WEBVTT 00:00:00.000 --> 00:00:02.490 align:middle line:84% The following content is provided under a Creative 00:00:02.490 --> 00:00:03.730 align:middle line:90% Commons license. 00:00:03.730 --> 00:00:06.120 align:middle line:84% Your support will help MIT OpenCourseWare 00:00:06.120 --> 00:00:10.090 align:middle line:84% continue to offer high-quality educational resources for free. 00:00:10.090 --> 00:00:12.720 align:middle line:84% To make a donation or to view additional materials 00:00:12.720 --> 00:00:16.185 align:middle line:84% from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:16.185 --> 00:00:17.560 align:middle line:90% at ocw.mit.edu. 00:00:17.560 --> 00:00:20.708 align:middle line:90% 00:00:20.708 --> 00:00:21.750 align:middle line:90% RAMESH RASKAR: All right. 00:00:21.750 --> 00:00:26.100 align:middle line:84% So let me switch over to this topic. 00:00:26.100 --> 00:00:27.360 align:middle line:90% And let's see some cool demos. 00:00:27.360 --> 00:00:32.729 align:middle line:90% 00:00:32.729 --> 00:00:36.930 align:middle line:84% So this particular project that Kimo Johnson and [INAUDIBLE] 00:00:36.930 --> 00:00:41.610 align:middle line:84% Adelson built, called retrographic sensor, 00:00:41.610 --> 00:00:46.360 align:middle line:84% also won the best demo award at CVPR. 00:00:46.360 --> 00:00:50.310 align:middle line:90% So we saw this chart last time. 00:00:50.310 --> 00:00:56.740 align:middle line:84% They're already selling 2.5 billion sensors 00:00:56.740 --> 00:01:00.320 align:middle line:84% in the next year or two per year. 00:01:00.320 --> 00:01:03.220 align:middle line:84% And this is the forecast made in 2006, 2007. 00:01:03.220 --> 00:01:06.990 align:middle line:90% So it might have changed. 00:01:06.990 --> 00:01:11.520 align:middle line:84% And if you see the integration of cameras in mobile phones, 00:01:11.520 --> 00:01:17.310 align:middle line:84% a natural question is, will we have-- 00:01:17.310 --> 00:01:22.350 align:middle line:84% will we even think about our digital still camera 00:01:22.350 --> 00:01:27.160 align:middle line:84% as a standalone entity in coming years? 00:01:27.160 --> 00:01:29.700 align:middle line:84% And if you think about the wristwatch, I think most of us 00:01:29.700 --> 00:01:31.320 align:middle line:90% don't wear wristwatches anymore. 00:01:31.320 --> 00:01:38.050 align:middle line:84% Knowing what time it is is not a difficult problem anymore. 00:01:38.050 --> 00:01:40.530 align:middle line:84% I don't need to spend a lot of money. 00:01:40.530 --> 00:01:45.780 align:middle line:84% Or I don't have to tether myself to an additional device 00:01:45.780 --> 00:01:48.360 align:middle line:90% to know what time it is. 00:01:48.360 --> 00:01:52.890 align:middle line:84% And will we be in a similar situation about image? 00:01:52.890 --> 00:01:58.480 align:middle line:84% Will we even have a standalone camera for any purpose? 00:01:58.480 --> 00:02:00.450 align:middle line:84% So I just want to take a couple of minutes 00:02:00.450 --> 00:02:03.990 align:middle line:90% to see what opinions you have. 00:02:03.990 --> 00:02:07.940 align:middle line:84% According to Nokia, more than 50% 00:02:07.940 --> 00:02:10.979 align:middle line:84% of the people who bought N95 got rid 00:02:10.979 --> 00:02:15.170 align:middle line:84% of their point-and-shoot camera And Nokia 00:02:15.170 --> 00:02:21.100 align:middle line:84% sells about a million phones every day. 00:02:21.100 --> 00:02:23.630 align:middle line:84% They sell 400 million phones a year. 00:02:23.630 --> 00:02:28.350 align:middle line:84% So they sell more than a million phones every day. 00:02:28.350 --> 00:02:31.085 align:middle line:90% So these are staggering numbers. 00:02:31.085 --> 00:02:35.838 align:middle line:90% 00:02:35.838 --> 00:02:37.880 align:middle line:84% Which is the largest camera company in the world? 00:02:37.880 --> 00:02:40.450 align:middle line:90% 00:02:40.450 --> 00:02:41.200 align:middle line:90% AUDIENCE: Nokia? 00:02:41.200 --> 00:02:41.560 align:middle line:90% RAMESH RASKAR: Sorry? 00:02:41.560 --> 00:02:42.440 align:middle line:90% AUDIENCE: Nokia [INAUDIBLE]. 00:02:42.440 --> 00:02:43.607 align:middle line:90% RAMESH RASKAR: Nokia, right. 00:02:43.607 --> 00:02:45.940 align:middle line:84% It's not a traditional camera company. 00:02:45.940 --> 00:02:50.920 align:middle line:84% And the same answer for, which is the largest computer company 00:02:50.920 --> 00:02:52.260 align:middle line:90% in the world? 00:02:52.260 --> 00:02:53.730 align:middle line:90% AUDIENCE: Flextronics. 00:02:53.730 --> 00:02:57.140 align:middle line:84% RAMESH RASKAR: Oh, yeah, Nokia doesn't build anything 00:02:57.140 --> 00:02:58.140 align:middle line:90% of their own, of course. 00:02:58.140 --> 00:03:00.600 align:middle line:90% They owe EM everything. 00:03:00.600 --> 00:03:04.290 align:middle line:84% But still, they're the largest camera and computer 00:03:04.290 --> 00:03:07.172 align:middle line:90% company in the world as well. 00:03:07.172 --> 00:03:08.340 align:middle line:90% So all right. 00:03:08.340 --> 00:03:11.850 align:middle line:84% When do you think of camera as a standalone device 00:03:11.850 --> 00:03:15.500 align:middle line:90% will completely disappear? 00:03:15.500 --> 00:03:17.330 align:middle line:84% AUDIENCE: It'll never completely disappear. 00:03:17.330 --> 00:03:18.280 align:middle line:84% RAMESH RASKAR: And by completely, I 00:03:18.280 --> 00:03:20.690 align:middle line:84% mean less than 0.1% of the population carries it, 00:03:20.690 --> 00:03:21.742 align:middle line:90% basically. 00:03:21.742 --> 00:03:24.200 align:middle line:84% Something that's-- it's like saying whether the film camera 00:03:24.200 --> 00:03:24.930 align:middle line:90% will disappear. 00:03:24.930 --> 00:03:25.940 align:middle line:90% And I believe it's gone. 00:03:25.940 --> 00:03:28.788 align:middle line:90% 00:03:28.788 --> 00:03:31.330 align:middle line:84% AUDIENCE: Oh, but they're going to be gone in different ways. 00:03:31.330 --> 00:03:33.455 align:middle line:84% So camera and digital cameras are going to be gone. 00:03:33.455 --> 00:03:35.400 align:middle line:84% And then, they're going to be everywhere. 00:03:35.400 --> 00:03:38.908 align:middle line:84% And film cameras are gone in that nobody uses them. 00:03:38.908 --> 00:03:40.180 align:middle line:90% AUDIENCE: What? 00:03:40.180 --> 00:03:41.180 align:middle line:90% RAMESH RASKAR: I'm sorry 00:03:41.180 --> 00:03:42.780 align:middle line:90% [INTERPOSING VOICES] 00:03:42.780 --> 00:03:47.002 align:middle line:90% [LAUGHTER] 00:03:47.002 --> 00:03:48.960 align:middle line:84% RAMESH RASKAR: I'll give you an invented thing. 00:03:48.960 --> 00:03:50.700 align:middle line:84% You don't want to answer the film camera question. 00:03:50.700 --> 00:03:51.325 align:middle line:90% You'll love it. 00:03:51.325 --> 00:03:53.740 align:middle line:84% But when will the digital camera disappear altogether? 00:03:53.740 --> 00:03:55.740 align:middle line:84% You don't have anything against digital cameras, 00:03:55.740 --> 00:03:57.522 align:middle line:90% though, do you? 00:03:57.522 --> 00:03:59.230 align:middle line:84% AUDIENCE: Well, no, I don't have anything 00:03:59.230 --> 00:04:00.230 align:middle line:90% against digital cameras. 00:04:00.230 --> 00:04:03.370 align:middle line:84% But it depends on your definition of a digital camera. 00:04:03.370 --> 00:04:05.620 align:middle line:84% If you're just talking about point-and-shoot cameras-- 00:04:05.620 --> 00:04:07.953 align:middle line:84% RAMESH RASKAR: A standalone-- a camera-- a device that's 00:04:07.953 --> 00:04:12.760 align:middle line:84% built specifically for the purpose of taking photos. 00:04:12.760 --> 00:04:14.950 align:middle line:84% AUDIENCE: So if you're not going to be 00:04:14.950 --> 00:04:18.790 align:middle line:84% a professional photographer, then the average consumer, 00:04:18.790 --> 00:04:24.090 align:middle line:84% it's probably just going to disappear maybe in 10 years? 00:04:24.090 --> 00:04:25.090 align:middle line:90% RAMESH RASKAR: 10 years. 00:04:25.090 --> 00:04:25.690 align:middle line:90% AUDIENCE: Five years? 00:04:25.690 --> 00:04:26.732 align:middle line:90% RAMESH RASKAR: All right. 00:04:26.732 --> 00:04:28.300 align:middle line:84% And either-- anybody thinks it'll 00:04:28.300 --> 00:04:31.960 align:middle line:90% happen in less than 10 years? 00:04:31.960 --> 00:04:33.368 align:middle line:90% Let's take someone over there. 00:04:33.368 --> 00:04:35.410 align:middle line:84% AUDIENCE: Yeah, I think around five [INAUDIBLE].. 00:04:35.410 --> 00:04:36.670 align:middle line:90% RAMESH RASKAR: In five years? 00:04:36.670 --> 00:04:40.028 align:middle line:84% There'll be no standalone consumer digital cameras. 00:04:40.028 --> 00:04:42.070 align:middle line:84% AUDIENCE: Yeah, I think in just a couple of years 00:04:42.070 --> 00:04:43.150 align:middle line:90% because if you think about it-- 00:04:43.150 --> 00:04:44.050 align:middle line:90% RAMESH RASKAR: In two years? 00:04:44.050 --> 00:04:44.350 align:middle line:90% AUDIENCE: Yeah. 00:04:44.350 --> 00:04:45.142 align:middle line:90% RAMESH RASKAR: Wow. 00:04:45.142 --> 00:04:47.770 align:middle line:90% [LAUGHTER] 00:04:47.770 --> 00:04:49.420 align:middle line:84% Let's call all the camera companies. 00:04:49.420 --> 00:04:50.710 align:middle line:90% [LAUGHTER] 00:04:50.710 --> 00:04:51.593 align:middle line:90% [INTERPOSING VOICES] 00:04:51.593 --> 00:04:53.260 align:middle line:84% AUDIENCE: --she's already switching over 00:04:53.260 --> 00:04:55.120 align:middle line:90% to emphasizing video. 00:04:55.120 --> 00:04:57.310 align:middle line:84% And then also, SLRs are starting to be video 00:04:57.310 --> 00:05:00.010 align:middle line:84% also because they're not just normal cameras anymore. 00:05:00.010 --> 00:05:00.250 align:middle line:90% RAMESH RASKAR: Right. 00:05:00.250 --> 00:05:02.930 align:middle line:84% But I meant, video cameras are still cameras, the same thing. 00:05:02.930 --> 00:05:07.660 align:middle line:84% I'm just saying, will we have standalone devices that 00:05:07.660 --> 00:05:08.650 align:middle line:90% only goes to-- 00:05:08.650 --> 00:05:09.538 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:05:09.538 --> 00:05:10.330 align:middle line:90% RAMESH RASKAR: Yes? 00:05:10.330 --> 00:05:13.540 align:middle line:84% AUDIENCE: There are cameras that allow you to upload your 00:05:13.540 --> 00:05:14.360 align:middle line:90% [INAUDIBLE]. 00:05:14.360 --> 00:05:17.195 align:middle line:90% 00:05:17.195 --> 00:05:18.070 align:middle line:90% RAMESH RASKAR: Right. 00:05:18.070 --> 00:05:18.970 align:middle line:84% AUDIENCE: And there's no reason why it just 00:05:18.970 --> 00:05:20.620 align:middle line:84% wouldn't be on a computer, the actual 00:05:20.620 --> 00:05:22.092 align:middle line:90% [INAUDIBLE] device [INAUDIBLE]. 00:05:22.092 --> 00:05:23.050 align:middle line:90% RAMESH RASKAR: Exactly. 00:05:23.050 --> 00:05:24.700 align:middle line:84% AUDIENCE: So basically, it's converting 00:05:24.700 --> 00:05:27.675 align:middle line:84% both the cameras [INAUDIBLE],, not just the acquisition. 00:05:27.675 --> 00:05:28.550 align:middle line:90% RAMESH RASKAR: Right. 00:05:28.550 --> 00:05:30.705 align:middle line:84% AUDIENCE: And the other [INAUDIBLE].. 00:05:30.705 --> 00:05:31.580 align:middle line:90% RAMESH RASKAR: Right. 00:05:31.580 --> 00:05:32.090 align:middle line:90% You're right. 00:05:32.090 --> 00:05:33.715 align:middle line:84% It's going to be some, kind of, fusion. 00:05:33.715 --> 00:05:35.800 align:middle line:84% But is the camera going to become a phone? 00:05:35.800 --> 00:05:37.508 align:middle line:84% Or the phone is going to become a camera? 00:05:37.508 --> 00:05:38.611 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:05:38.611 --> 00:05:40.084 align:middle line:90% [LAUGHTER] 00:05:40.084 --> 00:05:42.050 align:middle line:90% 00:05:42.050 --> 00:05:43.606 align:middle line:84% RAMESH RASKAR: I think in the back. 00:05:43.606 --> 00:05:45.170 align:middle line:90% AUDIENCE: I would say a century. 00:05:45.170 --> 00:05:46.360 align:middle line:90% RAMESH RASKAR: A century. 00:05:46.360 --> 00:05:46.860 align:middle line:90% Wow. 00:05:46.860 --> 00:05:49.010 align:middle line:84% AUDIENCE: Yeah, the same way that computers 00:05:49.010 --> 00:05:52.940 align:middle line:84% brought us the paperless office and the end of books. 00:05:52.940 --> 00:05:58.280 align:middle line:84% And we no longer have a stage, theater, or cinema. 00:05:58.280 --> 00:06:02.910 align:middle line:84% We've been predicting that for a long time [INAUDIBLE].. 00:06:02.910 --> 00:06:06.390 align:middle line:84% RAMESH RASKAR: So that's a very interesting comment. 00:06:06.390 --> 00:06:10.980 align:middle line:84% What's unique about paper or watching movies that may or may 00:06:10.980 --> 00:06:15.240 align:middle line:90% not apply to a device? 00:06:15.240 --> 00:06:19.110 align:middle line:84% AUDIENCE: But in response to what you said, 00:06:19.110 --> 00:06:22.410 align:middle line:84% the need to capture images won't disappear. 00:06:22.410 --> 00:06:24.060 align:middle line:84% But the fact that there is going to be 00:06:24.060 --> 00:06:27.480 align:middle line:84% one device that its sole purpose is to capture images 00:06:27.480 --> 00:06:28.530 align:middle line:90% will disappear. 00:06:28.530 --> 00:06:31.020 align:middle line:84% And it will be a device that it's either a phone, 00:06:31.020 --> 00:06:36.277 align:middle line:84% and capture this, and [INAUDIBLE],, and [INAUDIBLE].. 00:06:36.277 --> 00:06:38.740 align:middle line:84% AUDIENCE: Yeah, the way we print things has changed. 00:06:38.740 --> 00:06:39.990 align:middle line:90% But we still have [INAUDIBLE]. 00:06:39.990 --> 00:06:40.870 align:middle line:90% [INTERPOSING VOICES] 00:06:40.870 --> 00:06:41.700 align:middle line:84% AUDIENCE: That's why you're doing the image, 00:06:41.700 --> 00:06:44.002 align:middle line:84% though, because you're shooting stuff on a screen 00:06:44.002 --> 00:06:45.460 align:middle line:84% because you want the image quality. 00:06:45.460 --> 00:06:47.850 align:middle line:84% So maybe you [INAUDIBLE] enough or 00:06:47.850 --> 00:06:50.170 align:middle line:90% simulate it that [INAUDIBLE]. 00:06:50.170 --> 00:06:53.370 align:middle line:90% 00:06:53.370 --> 00:06:55.920 align:middle line:84% RAMESH RASKAR: Yeah, but it is a hypothetical scenario. 00:06:55.920 --> 00:07:00.870 align:middle line:84% If I can buy a phone whose image quality is sufficiently 00:07:00.870 --> 00:07:08.370 align:middle line:84% good compared to a standalone still camera, 00:07:08.370 --> 00:07:09.420 align:middle line:90% that's the question. 00:07:09.420 --> 00:07:11.100 align:middle line:84% Yeah, assuming that the image quality 00:07:11.100 --> 00:07:13.240 align:middle line:90% is going to be good enough. 00:07:13.240 --> 00:07:15.287 align:middle line:90% Let's ask somebody down here. 00:07:15.287 --> 00:07:16.120 align:middle line:90% AUDIENCE: Sorry, go. 00:07:16.120 --> 00:07:16.380 align:middle line:90% RAMESH RASKAR: Yeah. 00:07:16.380 --> 00:07:18.797 align:middle line:84% AUDIENCE: Yeah, I actually really agree with what he said. 00:07:18.797 --> 00:07:22.710 align:middle line:84% I actually think it'll never really disappear 00:07:22.710 --> 00:07:23.790 align:middle line:90% for a couple of reasons. 00:07:23.790 --> 00:07:24.290 align:middle line:90% I think-- 00:07:24.290 --> 00:07:26.347 align:middle line:84% RAMESH RASKAR: 0.1% of the population. 00:07:26.347 --> 00:07:27.180 align:middle line:90% AUDIENCE: All right. 00:07:27.180 --> 00:07:29.280 align:middle line:84% So I actually will be curious today how many people 00:07:29.280 --> 00:07:30.120 align:middle line:90% carry phone cameras. 00:07:30.120 --> 00:07:31.770 align:middle line:90% I'm curious. 00:07:31.770 --> 00:07:33.420 align:middle line:84% RAMESH RASKAR: One billion people 00:07:33.420 --> 00:07:35.340 align:middle line:90% carry a camera in the pocket. 00:07:35.340 --> 00:07:37.860 align:middle line:84% AUDIENCE: But so one reason I think 00:07:37.860 --> 00:07:41.190 align:middle line:84% is that there's not only image quality. 00:07:41.190 --> 00:07:43.857 align:middle line:84% But there's the ergonomic interaction. 00:07:43.857 --> 00:07:44.440 align:middle line:90% AUDIENCE: Yes. 00:07:44.440 --> 00:07:46.607 align:middle line:84% AUDIENCE: And I think the reason why a lot of people 00:07:46.607 --> 00:07:49.065 align:middle line:84% carry DSLR is not just for the image quality, 00:07:49.065 --> 00:07:50.940 align:middle line:84% but for having things available, and buttons, 00:07:50.940 --> 00:07:53.877 align:middle line:84% and having human hands that are certain sizes, and people 00:07:53.877 --> 00:07:54.710 align:middle line:90% [INTERPOSING VOICES] 00:07:54.710 --> 00:07:57.168 align:middle line:84% RAMESH RASKAR: Sometimes, that's heavy enough and something 00:07:57.168 --> 00:07:58.570 align:middle line:90% to look cool with and so on. 00:07:58.570 --> 00:07:58.670 align:middle line:90% [INTERPOSING VOICES] 00:07:58.670 --> 00:07:59.170 align:middle line:90% [LAUGHTER] 00:07:59.170 --> 00:07:59.790 align:middle line:90% AUDIENCE: So-- 00:07:59.790 --> 00:08:01.470 align:middle line:84% RAMESH RASKAR: Is it for function or just for 00:08:01.470 --> 00:08:01.700 align:middle line:90% [INTERPOSING VOICES] 00:08:01.700 --> 00:08:03.840 align:middle line:84% AUDIENCE: No, the heaviness does add a function 00:08:03.840 --> 00:08:06.588 align:middle line:84% because it stabilizes the image a little bit. 00:08:06.588 --> 00:08:08.880 align:middle line:84% AUDIENCE: Yeah, but I also think that there's something 00:08:08.880 --> 00:08:09.380 align:middle line:90% else too. 00:08:09.380 --> 00:08:11.430 align:middle line:84% I'm actually unconvinced in general 00:08:11.430 --> 00:08:15.540 align:middle line:84% that devices converging is necessarily good. 00:08:15.540 --> 00:08:17.230 align:middle line:84% Friends of mine that I tell this to say 00:08:17.230 --> 00:08:19.480 align:middle line:84% that I'm an exception because I carry two cell phones. 00:08:19.480 --> 00:08:21.440 align:middle line:84% And I think I'm carrying three cameras right now. 00:08:21.440 --> 00:08:23.815 align:middle line:84% RAMESH RASKAR: Yeah, two cell phones will never converge. 00:08:23.815 --> 00:08:25.156 align:middle line:90% [LAUGHTER] 00:08:25.156 --> 00:08:28.184 align:middle line:84% AUDIENCE: Well, the amount of phones that have two Sim cards 00:08:28.184 --> 00:08:29.017 align:middle line:90% [INTERPOSING VOICES] 00:08:29.017 --> 00:08:29.444 align:middle line:90% [LAUGHTER] 00:08:29.444 --> 00:08:30.300 align:middle line:90% RAMESH RASKAR: [INAUDIBLE] 00:08:30.300 --> 00:08:32.383 align:middle line:84% AUDIENCE: No, I think there are certain advantages 00:08:32.383 --> 00:08:35.940 align:middle line:84% to having, actually, redundancy in devices and things. 00:08:35.940 --> 00:08:38.250 align:middle line:84% So I definitely agree that digital cameras 00:08:38.250 --> 00:08:40.830 align:middle line:84% might have WiFi connectivity or other things. 00:08:40.830 --> 00:08:42.750 align:middle line:84% But I suspect that a lot of people 00:08:42.750 --> 00:08:45.210 align:middle line:84% will carry separate things that are 00:08:45.210 --> 00:08:48.285 align:middle line:84% their primary purpose as imaging and a separate thing itself. 00:08:48.285 --> 00:08:49.160 align:middle line:90% RAMESH RASKAR: Right. 00:08:49.160 --> 00:08:50.560 align:middle line:84% AUDIENCE: And then, they also have a camera in it. 00:08:50.560 --> 00:08:52.143 align:middle line:84% RAMESH RASKAR: Well, there will always 00:08:52.143 --> 00:08:54.030 align:middle line:84% be a film camera in [INAUDIBLE] house. 00:08:54.030 --> 00:08:56.100 align:middle line:90% [LAUGHTER] 00:08:56.100 --> 00:08:58.180 align:middle line:90% Even 20 years from now. 00:08:58.180 --> 00:09:00.940 align:middle line:90% But we're talking about 0.1%. 00:09:00.940 --> 00:09:02.800 align:middle line:84% So let's take a quick vote and move on. 00:09:02.800 --> 00:09:05.700 align:middle line:90% So how many people think it's-- 00:09:05.700 --> 00:09:06.580 align:middle line:90% let's do the reverse. 00:09:06.580 --> 00:09:10.770 align:middle line:84% So we'll say, two years from now, 00:09:10.770 --> 00:09:15.660 align:middle line:84% how many of you think will a standalone imaging 00:09:15.660 --> 00:09:17.802 align:middle line:90% device will disappear? 00:09:17.802 --> 00:09:18.650 align:middle line:90% Kevin? 00:09:18.650 --> 00:09:19.480 align:middle line:90% No? 00:09:19.480 --> 00:09:19.980 align:middle line:90% All right. 00:09:19.980 --> 00:09:20.670 align:middle line:90% We have one. 00:09:20.670 --> 00:09:22.365 align:middle line:90% AUDIENCE: No. 00:09:22.365 --> 00:09:24.490 align:middle line:84% RAMESH RASKAR: Do you want to say very quickly why? 00:09:24.490 --> 00:09:25.218 align:middle line:90% Because you-- 00:09:25.218 --> 00:09:26.154 align:middle line:90% [LAUGHTER] 00:09:26.154 --> 00:09:27.480 align:middle line:90% AUDIENCE: It would be-- 00:09:27.480 --> 00:09:30.175 align:middle line:84% so far, the image quality has been the problem. 00:09:30.175 --> 00:09:31.050 align:middle line:90% RAMESH RASKAR: Right. 00:09:31.050 --> 00:09:33.690 align:middle line:84% AUDIENCE: There's just not quite enough reliance on the sensors 00:09:33.690 --> 00:09:35.070 align:middle line:90% that you can fit into a phone. 00:09:35.070 --> 00:09:37.080 align:middle line:84% But I think people are finding ways 00:09:37.080 --> 00:09:41.015 align:middle line:84% to get around that with software and with optics. 00:09:41.015 --> 00:09:41.890 align:middle line:90% RAMESH RASKAR: Right. 00:09:41.890 --> 00:09:42.670 align:middle line:90% Oh, by the way, this-- 00:09:42.670 --> 00:09:43.540 align:middle line:84% AUDIENCE: And with the sensor, actually. 00:09:43.540 --> 00:09:45.707 align:middle line:84% There's a whole new generation of sensors coming out 00:09:45.707 --> 00:09:49.930 align:middle line:90% that allows you to [INAUDIBLE]. 00:09:49.930 --> 00:09:51.850 align:middle line:84% RAMESH RASKAR: Remember, right now, 00:09:51.850 --> 00:09:58.900 align:middle line:84% the total standalone devices for imaging is 10% already. 00:09:58.900 --> 00:10:02.090 align:middle line:84% This is optical mouse, which is not really a camera. 00:10:02.090 --> 00:10:03.560 align:middle line:90% We can ignore that. 00:10:03.560 --> 00:10:05.680 align:middle line:84% But mobile phone camera, [INAUDIBLE] cameras, 00:10:05.680 --> 00:10:06.590 align:middle line:90% gaming and so on. 00:10:06.590 --> 00:10:07.780 align:middle line:90% So it's already 10%. 00:10:07.780 --> 00:10:09.890 align:middle line:90% And we're talking about 0.1%. 00:10:09.890 --> 00:10:10.900 align:middle line:90% So let's move on. 00:10:10.900 --> 00:10:12.730 align:middle line:84% And just, let's take a quick poll. 00:10:12.730 --> 00:10:14.500 align:middle line:84% So two years, we have only [INAUDIBLE].. 00:10:14.500 --> 00:10:15.940 align:middle line:84% AUDIENCE: Quick question, I guess. 00:10:15.940 --> 00:10:18.040 align:middle line:90% Is that even a decent metric? 00:10:18.040 --> 00:10:21.520 align:middle line:84% Would it be better to say, how many photos are taken? 00:10:21.520 --> 00:10:23.650 align:middle line:84% Because a lot of people carry cell phone cameras, 00:10:23.650 --> 00:10:25.720 align:middle line:90% but are using the [INAUDIBLE]. 00:10:25.720 --> 00:10:27.610 align:middle line:84% My mom doesn't use her cell phone. 00:10:27.610 --> 00:10:27.910 align:middle line:90% RAMESH RASKAR: Right. 00:10:27.910 --> 00:10:28.765 align:middle line:90% AUDIENCE: But she has one. 00:10:28.765 --> 00:10:29.390 align:middle line:90% AUDIENCE: Yeah. 00:10:29.390 --> 00:10:31.450 align:middle line:84% RAMESH RASKAR: The next class, we'll talk about, 00:10:31.450 --> 00:10:33.950 align:middle line:84% how long will it take before photos disappear? 00:10:33.950 --> 00:10:34.450 align:middle line:90% But-- 00:10:34.450 --> 00:10:36.050 align:middle line:90% [LAUGHTER] 00:10:36.050 --> 00:10:38.298 align:middle line:90% --that's the next conversation. 00:10:38.298 --> 00:10:39.750 align:middle line:90% [LAUGHTER] 00:10:39.750 --> 00:10:41.690 align:middle line:90% 00:10:41.690 --> 00:10:44.470 align:middle line:84% So it's like saying, how many people 00:10:44.470 --> 00:10:46.658 align:middle line:84% write with their own handwriting? 00:10:46.658 --> 00:10:48.700 align:middle line:84% It's the same situation we want to see very soon. 00:10:48.700 --> 00:10:50.200 align:middle line:90% But that's for the next class. 00:10:50.200 --> 00:10:53.810 align:middle line:90% 00:10:53.810 --> 00:10:55.390 align:middle line:90% So two years, we have only one. 00:10:55.390 --> 00:10:56.065 align:middle line:90% Five years? 00:10:56.065 --> 00:10:59.400 align:middle line:90% 00:10:59.400 --> 00:11:00.966 align:middle line:90% Three or four. 00:11:00.966 --> 00:11:01.695 align:middle line:90% 10 years? 00:11:01.695 --> 00:11:05.370 align:middle line:90% 00:11:05.370 --> 00:11:07.980 align:middle line:90% 15 years? 00:11:07.980 --> 00:11:10.740 align:middle line:84% So I guess almost everybody because-- 00:11:10.740 --> 00:11:13.584 align:middle line:90% and never? 00:11:13.584 --> 00:11:14.988 align:middle line:90% Wow. 00:11:14.988 --> 00:11:16.860 align:middle line:90% Wow. 00:11:16.860 --> 00:11:18.120 align:middle line:90% All the power to you. 00:11:18.120 --> 00:11:19.920 align:middle line:84% AUDIENCE: I say one additional comment. 00:11:19.920 --> 00:11:21.870 align:middle line:84% We're thinking about this population. 00:11:21.870 --> 00:11:25.680 align:middle line:84% But here, in most of the world, people cannot go and afford 00:11:25.680 --> 00:11:26.917 align:middle line:90% this separate camera. 00:11:26.917 --> 00:11:28.125 align:middle line:90% They're going to buy a phone. 00:11:28.125 --> 00:11:28.860 align:middle line:90% RAMESH RASKAR: A similar point. 00:11:28.860 --> 00:11:30.060 align:middle line:90% An excellent point. 00:11:30.060 --> 00:11:32.070 align:middle line:90% AUDIENCE: [INAUDIBLE] 1%. 00:11:32.070 --> 00:11:32.970 align:middle line:90% RAMESH RASKAR: Yeah. 00:11:32.970 --> 00:11:33.780 align:middle line:84% AUDIENCE: It doesn't change that much. 00:11:33.780 --> 00:11:35.697 align:middle line:84% RAMESH RASKAR: When the remaining five billion 00:11:35.697 --> 00:11:36.390 align:middle line:90% come on board. 00:11:36.390 --> 00:11:37.530 align:middle line:90% AUDIENCE: Right. 00:11:37.530 --> 00:11:39.170 align:middle line:84% RAMESH RASKAR: Yeah, that's a good point, actually. 00:11:39.170 --> 00:11:40.420 align:middle line:90% AUDIENCE: So in camera phones. 00:11:40.420 --> 00:11:42.500 align:middle line:90% RAMESH RASKAR: Right. 00:11:42.500 --> 00:11:43.100 align:middle line:90% Yes, Jeremy? 00:11:43.100 --> 00:11:44.600 align:middle line:84% AUDIENCE: Yeah, I have a prediction. 00:11:44.600 --> 00:11:47.090 align:middle line:84% I think that people will still have many devices, 00:11:47.090 --> 00:11:48.520 align:middle line:90% like an Apple device. 00:11:48.520 --> 00:11:53.310 align:middle line:84% And if you look at Steve Jobs, you have the iPhone with camera 00:11:53.310 --> 00:11:55.085 align:middle line:90% and the iPod Touch with camera. 00:11:55.085 --> 00:11:55.960 align:middle line:90% RAMESH RASKAR: Right. 00:11:55.960 --> 00:11:58.635 align:middle line:84% AUDIENCE: Now they are an iPhone without [INAUDIBLE].. 00:11:58.635 --> 00:11:59.510 align:middle line:90% RAMESH RASKAR: Right. 00:11:59.510 --> 00:12:00.560 align:middle line:90% AUDIENCE: So it's interesting. 00:12:00.560 --> 00:12:01.640 align:middle line:84% RAMESH RASKAR: That's just a marketing gimmick. 00:12:01.640 --> 00:12:02.140 align:middle line:90% But-- 00:12:02.140 --> 00:12:04.310 align:middle line:84% AUDIENCE: Yeah, but it's interesting to think 00:12:04.310 --> 00:12:06.690 align:middle line:90% of marketing in this respect. 00:12:06.690 --> 00:12:08.586 align:middle line:84% And I think the point is they [INAUDIBLE].. 00:12:08.586 --> 00:12:09.461 align:middle line:90% RAMESH RASKAR: Right. 00:12:09.461 --> 00:12:11.715 align:middle line:84% AUDIENCE: Some people can [INAUDIBLE] device. 00:12:11.715 --> 00:12:12.590 align:middle line:90% RAMESH RASKAR: Right. 00:12:12.590 --> 00:12:14.090 align:middle line:84% AUDIENCE: And the mobile phone makes 00:12:14.090 --> 00:12:18.237 align:middle line:84% sense because communication is really [INAUDIBLE].. 00:12:18.237 --> 00:12:19.820 align:middle line:84% RAMESH RASKAR: Yeah, very soon, you'll 00:12:19.820 --> 00:12:21.920 align:middle line:84% be able to buy a burger with a camera in it. 00:12:21.920 --> 00:12:24.446 align:middle line:90% [LAUGHTER] 00:12:24.446 --> 00:12:25.760 align:middle line:90% That's your-- 00:12:25.760 --> 00:12:27.888 align:middle line:90% 00:12:27.888 --> 00:12:28.763 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:12:28.763 --> 00:12:30.805 align:middle line:84% RAMESH RASKAR: What's the toy called for chicken? 00:12:30.805 --> 00:12:31.648 align:middle line:90% The happy toy. 00:12:31.648 --> 00:12:32.690 align:middle line:90% AUDIENCE: The Happy Meal. 00:12:32.690 --> 00:12:33.740 align:middle line:90% RAMESH RASKAR: The Happy Meal. 00:12:33.740 --> 00:12:34.190 align:middle line:90% [LAUGHTER] 00:12:34.190 --> 00:12:35.990 align:middle line:84% And you'll know what you ate and how many calories 00:12:35.990 --> 00:12:36.823 align:middle line:90% [INTERPOSING VOICES] 00:12:36.823 --> 00:12:37.323 align:middle line:90% [LAUGHTER] 00:12:37.323 --> 00:12:38.406 align:middle line:90% RAMESH RASKAR: It's a lot. 00:12:38.406 --> 00:12:39.060 align:middle line:90% But-- 00:12:39.060 --> 00:12:39.400 align:middle line:90% [INTERPOSING VOICES] 00:12:39.400 --> 00:12:41.000 align:middle line:84% AUDIENCE: It will trace the burger 00:12:41.000 --> 00:12:42.560 align:middle line:90% through your digestive system? 00:12:42.560 --> 00:12:43.400 align:middle line:84% RAMESH RASKAR: Yeah, it will show you 00:12:43.400 --> 00:12:44.630 align:middle line:90% how happy you were and-- 00:12:44.630 --> 00:12:47.143 align:middle line:84% AUDIENCE: And then, when you get it out, it's [INAUDIBLE].. 00:12:47.143 --> 00:12:47.810 align:middle line:90% AUDIENCE: Great. 00:12:47.810 --> 00:12:48.290 align:middle line:90% RAMESH RASKAR: Right. 00:12:48.290 --> 00:12:48.770 align:middle line:90% [LAUGHTER] 00:12:48.770 --> 00:12:50.978 align:middle line:84% AUDIENCE: Yeah, in this graph, the number [INAUDIBLE] 00:12:50.978 --> 00:12:53.420 align:middle line:84% has also increased followed by the same proportion 00:12:53.420 --> 00:12:54.712 align:middle line:90% as the number of mobile phones. 00:12:54.712 --> 00:12:55.670 align:middle line:90% RAMESH RASKAR: Exactly. 00:12:55.670 --> 00:12:57.050 align:middle line:90% It's mostly mobile right now. 00:12:57.050 --> 00:12:59.342 align:middle line:84% AUDIENCE: Yeah, but see, the network of digital cameras 00:12:59.342 --> 00:13:00.320 align:middle line:90% is really part of the 00:13:00.320 --> 00:13:00.770 align:middle line:90% [INTERPOSING VOICES] 00:13:00.770 --> 00:13:02.840 align:middle line:84% RAMESH RASKAR: It's not growing at the same rate, 00:13:02.840 --> 00:13:05.360 align:middle line:90% unfortunately. 00:13:05.360 --> 00:13:06.758 align:middle line:90% So-- 00:13:06.758 --> 00:13:09.050 align:middle line:84% AUDIENCE: [INAUDIBLE] you can't buy a phone without it. 00:13:09.050 --> 00:13:10.820 align:middle line:84% It's hard to find a home without a camera. 00:13:10.820 --> 00:13:14.077 align:middle line:90% So it is unfair to look at that. 00:13:14.077 --> 00:13:16.410 align:middle line:84% RAMESH RASKAR: But that's the whole point of this class. 00:13:16.410 --> 00:13:19.090 align:middle line:84% I think we are at a stage where people are saying, 00:13:19.090 --> 00:13:20.340 align:middle line:90% oh, cameras are cheap enough. 00:13:20.340 --> 00:13:26.548 align:middle line:84% By the way, do you have an idea of how cheap a camera is today? 00:13:26.548 --> 00:13:28.290 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:13:28.290 --> 00:13:30.360 align:middle line:84% RAMESH RASKAR: For a device maker, 00:13:30.360 --> 00:13:33.557 align:middle line:84% how much does it cost them to pay for a camera? 00:13:33.557 --> 00:13:35.765 align:middle line:84% If they would attach a camera, how much does it cost? 00:13:35.765 --> 00:13:37.350 align:middle line:90% AUDIENCE: $1 [INAUDIBLE]. 00:13:37.350 --> 00:13:38.100 align:middle line:90% RAMESH RASKAR: $1. 00:13:38.100 --> 00:13:38.700 align:middle line:90% Anybody else? 00:13:38.700 --> 00:13:39.742 align:middle line:90% AUDIENCE: [INAUDIBLE] $5. 00:13:39.742 --> 00:13:41.765 align:middle line:90% RAMESH RASKAR: It's $0.20. 00:13:41.765 --> 00:13:42.390 align:middle line:90% AUDIENCE: What? 00:13:42.390 --> 00:13:42.920 align:middle line:90% [CHATTER] 00:13:42.920 --> 00:13:45.420 align:middle line:84% RAMESH RASKAR: So when I said it will come with your burger, 00:13:45.420 --> 00:13:47.480 align:middle line:90% I'm not joking. 00:13:47.480 --> 00:13:47.980 align:middle line:90% All right? 00:13:47.980 --> 00:13:49.320 align:middle line:84% Yeah, of course, it doesn't have power. 00:13:49.320 --> 00:13:50.112 align:middle line:90% It has electronics. 00:13:50.112 --> 00:13:50.760 align:middle line:90% It has objects. 00:13:50.760 --> 00:13:51.690 align:middle line:90% It has sensors. 00:13:51.690 --> 00:13:55.060 align:middle line:90% And it has a compression engine. 00:13:55.060 --> 00:13:55.560 align:middle line:90% All right? 00:13:55.560 --> 00:13:58.210 align:middle line:90% It's $0.20. 00:13:58.210 --> 00:13:59.760 align:middle line:90% So things are changing rapidly. 00:13:59.760 --> 00:14:01.590 align:middle line:84% And to answer your question on, OK, 00:14:01.590 --> 00:14:06.040 align:middle line:84% you cannot buy a device without a camera in it anymore, 00:14:06.040 --> 00:14:09.930 align:middle line:84% the problem is, right now, we don't have enough services that 00:14:09.930 --> 00:14:11.400 align:middle line:90% go on top of those cameras. 00:14:11.400 --> 00:14:13.650 align:middle line:84% But imagine if people in this room say, 00:14:13.650 --> 00:14:16.770 align:middle line:84% wow, a billion people have a camera. 00:14:16.770 --> 00:14:19.020 align:middle line:84% Let me try to see if I can build something 00:14:19.020 --> 00:14:21.330 align:middle line:90% that really exploits that. 00:14:21.330 --> 00:14:24.300 align:middle line:84% Or let me see if I can change the game a little bit, 00:14:24.300 --> 00:14:26.070 align:middle line:84% add a little bit extra features, maybe add 00:14:26.070 --> 00:14:29.340 align:middle line:84% a thermal sensor to it, maybe add UV light to it, 00:14:29.340 --> 00:14:30.180 align:middle line:90% whatever it is. 00:14:30.180 --> 00:14:33.540 align:middle line:84% Maybe I'll make it into a camera so that instead of $0.20, 00:14:33.540 --> 00:14:36.930 align:middle line:90% now it costs me $0.23. 00:14:36.930 --> 00:14:39.750 align:middle line:84% It'll completely change the game. 00:14:39.750 --> 00:14:42.150 align:middle line:84% And the same way we get addicted to our devices 00:14:42.150 --> 00:14:44.080 align:middle line:84% and we just can't live without them anymore. 00:14:44.080 --> 00:14:46.800 align:middle line:84% Hopefully, cameras will play a similar role, 00:14:46.800 --> 00:14:49.190 align:middle line:84% that you will be forced to use the camera 00:14:49.190 --> 00:14:51.000 align:middle line:84% and it won't be just an additional feature 00:14:51.000 --> 00:14:53.910 align:middle line:90% that never gets used. 00:14:53.910 --> 00:14:54.630 align:middle line:90% So-- 00:14:54.630 --> 00:14:56.670 align:middle line:84% AUDIENCE: Did you hope this or [INAUDIBLE]?? 00:14:56.670 --> 00:14:58.030 align:middle line:90% RAMESH RASKAR: About what? 00:14:58.030 --> 00:15:00.233 align:middle line:84% AUDIENCE: That you'll be forced to use a camera? 00:15:00.233 --> 00:15:02.400 align:middle line:84% RAMESH RASKAR: Not forced, but you'll be encouraged. 00:15:02.400 --> 00:15:03.156 align:middle line:90% How about that? 00:15:03.156 --> 00:15:05.100 align:middle line:90% [LAUGHTER] 00:15:05.100 --> 00:15:09.910 align:middle line:84% It's like saying, are you being forced to use a credit card? 00:15:09.910 --> 00:15:12.930 align:middle line:84% AUDIENCE: Yeah, that's what I actually felt like [INAUDIBLE].. 00:15:12.930 --> 00:15:13.763 align:middle line:90% RAMESH RASKAR: Yeah. 00:15:13.763 --> 00:15:14.638 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:15:14.638 --> 00:15:17.055 align:middle line:84% RAMESH RASKAR: That's another discussion I'm going to have 00:15:17.055 --> 00:15:17.745 align:middle line:90% [INAUDIBLE]. 00:15:17.745 --> 00:15:18.916 align:middle line:90% [LAUGHTER] 00:15:18.916 --> 00:15:20.170 align:middle line:90% 00:15:20.170 --> 00:15:20.670 align:middle line:90% All right. 00:15:20.670 --> 00:15:23.365 align:middle line:90% So taking notes. 00:15:23.365 --> 00:15:24.990 align:middle line:84% As you can see, a lot of our discussion 00:15:24.990 --> 00:15:27.330 align:middle line:90% will be not on the slides. 00:15:27.330 --> 00:15:30.570 align:middle line:90% So Sam's taking the notes today. 00:15:30.570 --> 00:15:34.290 align:middle line:84% And what you need to do is the slides will be online. 00:15:34.290 --> 00:15:36.460 align:middle line:84% So you don't need to capture what's on the slides. 00:15:36.460 --> 00:15:38.610 align:middle line:84% And you don't need to capture all the information that's 00:15:38.610 --> 00:15:39.110 align:middle line:90% coming up. 00:15:39.110 --> 00:15:41.980 align:middle line:84% For example, JB had a comment on a new camera 00:15:41.980 --> 00:15:43.840 align:middle line:84% or the other comments in the back. 00:15:43.840 --> 00:15:47.370 align:middle line:84% So you want to capture that in the slides or anything 00:15:47.370 --> 00:15:49.508 align:middle line:90% I mentioned. 00:15:49.508 --> 00:15:52.050 align:middle line:84% And also, summarize some of the question and answer sessions. 00:15:52.050 --> 00:15:53.940 align:middle line:90% And we'll draw some-- 00:15:53.940 --> 00:15:57.150 align:middle line:84% we'll have some demos, [INAUDIBLE] pictures. 00:15:57.150 --> 00:16:01.410 align:middle line:84% And I'll do some doodles on the board, and take a photo, 00:16:01.410 --> 00:16:02.070 align:middle line:90% and so on. 00:16:02.070 --> 00:16:05.820 align:middle line:84% And we'll have one student assigned for each class 00:16:05.820 --> 00:16:07.410 align:middle line:90% if you're taking for credit. 00:16:07.410 --> 00:16:11.400 align:middle line:84% And I encourage you to just use a laptop because you can, one, 00:16:11.400 --> 00:16:12.300 align:middle line:90% take notes faster. 00:16:12.300 --> 00:16:15.050 align:middle line:84% And you can also look up some information very quickly. 00:16:15.050 --> 00:16:17.050 align:middle line:84% And then, you can work with me over the weekend. 00:16:17.050 --> 00:16:21.090 align:middle line:84% And on one day, we'll post it on Stellar. 00:16:21.090 --> 00:16:25.170 align:middle line:84% Is everybody familiar with Stellar website for MIT? 00:16:25.170 --> 00:16:28.230 align:middle line:84% How many of you know about the Stellar webpage? 00:16:28.230 --> 00:16:29.580 align:middle line:90% Everybody is familiar? 00:16:29.580 --> 00:16:31.650 align:middle line:84% And if you don't have access, if you're not 00:16:31.650 --> 00:16:33.768 align:middle line:84% taking the class for credit, then you may not-- 00:16:33.768 --> 00:16:35.310 align:middle line:84% I think you should be able to access. 00:16:35.310 --> 00:16:37.483 align:middle line:84% It's open to anybody in the world. 00:16:37.483 --> 00:16:39.150 align:middle line:84% So you can even send it to your friends. 00:16:39.150 --> 00:16:41.550 align:middle line:84% And they can look at just the slides. 00:16:41.550 --> 00:16:45.455 align:middle line:84% They won't get a lot of the other information, of course. 00:16:45.455 --> 00:16:47.580 align:middle line:84% But if you cannot access it, just send me an email. 00:16:47.580 --> 00:16:53.850 align:middle line:84% And I can explicitly add you as a guest on the Stellar website. 00:16:53.850 --> 00:16:56.160 align:middle line:84% The one benefit of adding explicitly 00:16:56.160 --> 00:16:57.750 align:middle line:84% is that, sometimes, the announcements 00:16:57.750 --> 00:17:01.500 align:middle line:84% go only to the people who are taking the class for credit. 00:17:01.500 --> 00:17:04.180 align:middle line:84% It could be about exams or other things. 00:17:04.180 --> 00:17:06.210 align:middle line:84% And if you want to be part-- if you 00:17:06.210 --> 00:17:08.077 align:middle line:84% want to participate in that, then you'll 00:17:08.077 --> 00:17:09.119 align:middle line:90% get those emails as well. 00:17:09.119 --> 00:17:12.200 align:middle line:90% 00:17:12.200 --> 00:17:14.599 align:middle line:84% So assignment one was about lighting. 00:17:14.599 --> 00:17:17.030 align:middle line:84% And that's what we'll be talking about today, 00:17:17.030 --> 00:17:19.438 align:middle line:84% very different ways of capturing light. 00:17:19.438 --> 00:17:20.605 align:middle line:90% Does anybody have questions? 00:17:20.605 --> 00:17:22.605 align:middle line:84% Because there's have a few things you had to do. 00:17:22.605 --> 00:17:24.890 align:middle line:84% You had to have a committed source code. 00:17:24.890 --> 00:17:29.540 align:middle line:84% Or if you used a GUI-based approach, Photoshop 00:17:29.540 --> 00:17:31.340 align:middle line:84% or something, then you need to show 00:17:31.340 --> 00:17:33.620 align:middle line:84% me a lot of intermediate results, input and output 00:17:33.620 --> 00:17:34.400 align:middle line:90% images. 00:17:34.400 --> 00:17:38.750 align:middle line:84% The images should be captured by yourself with your own camera. 00:17:38.750 --> 00:17:40.022 align:middle line:90% If you don't have-- 00:17:40.022 --> 00:17:41.480 align:middle line:84% for the first assignment, maybe you 00:17:41.480 --> 00:17:42.710 align:middle line:90% can use a cell phone camera. 00:17:42.710 --> 00:17:46.580 align:middle line:84% But see if you can buy a cheap camera that has a manual mode. 00:17:46.580 --> 00:17:50.720 align:middle line:84% You can buy a pretty good camera for less than $150. 00:17:50.720 --> 00:17:54.200 align:middle line:90% I recommend the Canon A series. 00:17:54.200 --> 00:17:56.180 align:middle line:90% It usually has a manual mode. 00:17:56.180 --> 00:17:58.850 align:middle line:84% And another benefit of the Canon series 00:17:58.850 --> 00:18:01.160 align:middle line:90% is that we have very good SDK. 00:18:01.160 --> 00:18:04.940 align:middle line:84% So you can control the camera from your laptop 00:18:04.940 --> 00:18:06.200 align:middle line:90% or from your PC. 00:18:06.200 --> 00:18:11.060 align:middle line:84% So I recommend the Canon A series to buy. 00:18:11.060 --> 00:18:13.370 align:middle line:84% And then, you must create a webpage 00:18:13.370 --> 00:18:17.900 align:middle line:84% where you will host your solutions for your assignments. 00:18:17.900 --> 00:18:19.460 align:middle line:84% And then, you should send me a link. 00:18:19.460 --> 00:18:23.030 align:middle line:84% And what you update on the Stellar is just a link. 00:18:23.030 --> 00:18:27.240 align:middle line:84% You're not going to upload all the images and so on. 00:18:27.240 --> 00:18:34.070 align:middle line:84% And then, it's perfectly fine to use software you find online. 00:18:34.070 --> 00:18:35.550 align:middle line:90% It's perfectly fine. 00:18:35.550 --> 00:18:38.670 align:middle line:84% Don't use your colleague's software 00:18:38.670 --> 00:18:43.160 align:middle line:84% in this class or software people wrote in this class last year. 00:18:43.160 --> 00:18:44.330 align:middle line:90% But you can go online. 00:18:44.330 --> 00:18:46.040 align:middle line:90% Go to MATLAB repository. 00:18:46.040 --> 00:18:48.300 align:middle line:84% And if they have a function for doing x, y, z, 00:18:48.300 --> 00:18:49.550 align:middle line:90% it's perfectly fine to use it. 00:18:49.550 --> 00:18:52.080 align:middle line:90% 00:18:52.080 --> 00:18:54.560 align:middle line:84% And then, we have a Flickr group page. 00:18:54.560 --> 00:18:57.290 align:middle line:84% I believe you got an email from me about the Flickr group page. 00:18:57.290 --> 00:19:01.040 align:middle line:84% And all your final results, sorry, input and final results 00:19:01.040 --> 00:19:03.590 align:middle line:84% will be on the Flickr page so everybody can see it. 00:19:03.590 --> 00:19:07.320 align:middle line:84% And we can write comments about each other's work. 00:19:07.320 --> 00:19:13.220 align:middle line:84% And this will be a good trial run towards our final project, 00:19:13.220 --> 00:19:15.290 align:middle line:84% where we will be doing a lot of commenting 00:19:15.290 --> 00:19:18.125 align:middle line:84% on each other's work, giving a lot of feedback, and critiques, 00:19:18.125 --> 00:19:19.370 align:middle line:90% and so on. 00:19:19.370 --> 00:19:24.920 align:middle line:84% So three things, Stellar, your own webpage, 00:19:24.920 --> 00:19:28.320 align:middle line:90% and the Flickr page. 00:19:28.320 --> 00:19:31.130 align:middle line:90% Any questions? 00:19:31.130 --> 00:19:31.700 align:middle line:90% All right. 00:19:31.700 --> 00:19:35.750 align:middle line:90% So this was simple. 00:19:35.750 --> 00:19:38.330 align:middle line:84% And if you have more time, you probably 00:19:38.330 --> 00:19:40.790 align:middle line:84% want to do in your assignment something that 00:19:40.790 --> 00:19:45.960 align:middle line:84% looks like this, where you can do some beautiful light. 00:19:45.960 --> 00:19:46.460 align:middle line:90% All right. 00:19:46.460 --> 00:19:49.790 align:middle line:84% So some of the assignments, I think we already discussed it, 00:19:49.790 --> 00:19:51.140 align:middle line:90% will look like this. 00:19:51.140 --> 00:19:54.690 align:middle line:90% 00:19:54.690 --> 00:19:58.490 align:middle line:84% And let's start with some interesting ways you 00:19:58.490 --> 00:20:02.540 align:middle line:90% can think about illumination. 00:20:02.540 --> 00:20:08.150 align:middle line:84% So here's a photo from One Kendall Square. 00:20:08.150 --> 00:20:10.760 align:middle line:84% This is [INAUDIBLE] here and so on. 00:20:10.760 --> 00:20:15.950 align:middle line:84% And even if you have been to Kendall Square, this photo, 00:20:15.950 --> 00:20:19.020 align:middle line:84% it's somewhat difficult to understand what's going on. 00:20:19.020 --> 00:20:20.960 align:middle line:84% You don't even know how many stories this, 00:20:20.960 --> 00:20:23.000 align:middle line:84% I guess it's the [? Treper Labs ?] building, 00:20:23.000 --> 00:20:26.790 align:middle line:84% has, for example, from this photo and so on. 00:20:26.790 --> 00:20:28.850 align:middle line:84% So imagine if, magically, you can 00:20:28.850 --> 00:20:38.690 align:middle line:84% convert this photo into this photo from here to here? 00:20:38.690 --> 00:20:42.540 align:middle line:84% What are some tricks you can use to do that? 00:20:42.540 --> 00:20:43.540 align:middle line:90% AUDIENCE: Wait 12 hours? 00:20:43.540 --> 00:20:45.748 align:middle line:90% [LAUGHTER] 00:20:45.748 --> 00:20:46.831 align:middle line:90% RAMESH RASKAR: [INAUDIBLE] 00:20:46.831 --> 00:20:49.036 align:middle line:90% AUDIENCE: [INAUDIBLE]. 00:20:49.036 --> 00:20:51.878 align:middle line:84% They have Photoshop's Shadow Highlights feature. 00:20:51.878 --> 00:20:54.420 align:middle line:84% RAMESH RASKAR: Yeah, you can try to stretch the [INAUDIBLE].. 00:20:54.420 --> 00:20:57.690 align:middle line:84% Or you can try to take a really long exposure photo. 00:20:57.690 --> 00:20:59.662 align:middle line:84% But yeah, this is really, really dark. 00:20:59.662 --> 00:21:00.870 align:middle line:90% There's almost nothing there. 00:21:00.870 --> 00:21:03.750 align:middle line:84% AUDIENCE: [INAUDIBLE] another photo, you can [INAUDIBLE].. 00:21:03.750 --> 00:21:05.070 align:middle line:90% RAMESH RASKAR: Exactly. 00:21:05.070 --> 00:21:06.235 align:middle line:90% Who's saying that? 00:21:06.235 --> 00:21:07.110 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:21:07.110 --> 00:21:07.380 align:middle line:90% RAMESH RASKAR: All right. 00:21:07.380 --> 00:21:08.580 align:middle line:90% Good. 00:21:08.580 --> 00:21:10.060 align:middle line:90% So you just cheat. 00:21:10.060 --> 00:21:13.980 align:middle line:84% This was actually taken when I was working at MERL. 00:21:13.980 --> 00:21:15.870 align:middle line:84% And this will be from my office, actually. 00:21:15.870 --> 00:21:18.370 align:middle line:84% It's a much nicer building than the office I have right now. 00:21:18.370 --> 00:21:19.670 align:middle line:90% [LAUGHTER] 00:21:19.670 --> 00:21:21.005 align:middle line:90% AUDIENCE: This campus is nice. 00:21:21.005 --> 00:21:21.880 align:middle line:90% RAMESH RASKAR: Sorry? 00:21:21.880 --> 00:21:23.130 align:middle line:90% AUDIENCE: This campus is nice. 00:21:23.130 --> 00:21:24.990 align:middle line:84% RAMESH RASKAR: Yeah, I like the brick texture. 00:21:24.990 --> 00:21:25.650 align:middle line:90% It's beautiful. 00:21:25.650 --> 00:21:27.420 align:middle line:90% [LAUGHTER] 00:21:27.420 --> 00:21:32.500 align:middle line:84% And just take the photo after 12 hours, in the morning. 00:21:32.500 --> 00:21:36.330 align:middle line:84% Actually, I took a photo at 6:00 AM 00:21:36.330 --> 00:21:40.110 align:middle line:84% when there were very few cars on the street. 00:21:40.110 --> 00:21:43.800 align:middle line:84% And somehow, you want to fuse these two photos into a photo 00:21:43.800 --> 00:21:45.280 align:middle line:90% that looks like that. 00:21:45.280 --> 00:21:48.960 align:middle line:84% So the static parts are captured from daytime. 00:21:48.960 --> 00:21:53.310 align:middle line:84% And all the dynamic parts with the cars, and the Christmas 00:21:53.310 --> 00:21:56.350 align:middle line:84% lights, and so on are captured from the nighttime. 00:21:56.350 --> 00:21:58.710 align:middle line:84% So you want to make some intelligent decision, which 00:21:58.710 --> 00:22:02.430 align:middle line:84% when-- which pixel comes from where? 00:22:02.430 --> 00:22:04.110 align:middle line:84% AUDIENCE: The color temperature is off. 00:22:04.110 --> 00:22:06.110 align:middle line:84% RAMESH RASKAR: And the color temperature is off. 00:22:06.110 --> 00:22:10.230 align:middle line:84% And you ought to make some artistic decisions about that. 00:22:10.230 --> 00:22:13.350 align:middle line:84% And you'll always have some issues, like this one. 00:22:13.350 --> 00:22:15.360 align:middle line:84% I think this guy, I waited all night long. 00:22:15.360 --> 00:22:17.190 align:middle line:90% But he was part of my class. 00:22:17.190 --> 00:22:18.360 align:middle line:90% He just didn't move. 00:22:18.360 --> 00:22:19.812 align:middle line:90% So you can see that. 00:22:19.812 --> 00:22:20.312 align:middle line:90% [LAUGHTER] 00:22:20.312 --> 00:22:23.700 align:middle line:90% You can see that. 00:22:23.700 --> 00:22:26.550 align:middle line:84% So it's a context-enhanced image. 00:22:26.550 --> 00:22:29.490 align:middle line:84% And we're really exploiting natural illumination 00:22:29.490 --> 00:22:31.300 align:middle line:90% to capture a new photo. 00:22:31.300 --> 00:22:31.800 align:middle line:90% Yeah? 00:22:31.800 --> 00:22:34.650 align:middle line:84% AUDIENCE: So how was this [INAUDIBLE] to you what 00:22:34.650 --> 00:22:39.112 align:middle line:90% [INAUDIBLE] algorithm or-- 00:22:39.112 --> 00:22:41.070 align:middle line:84% RAMESH RASKAR: Yeah, it has to be an algorithm. 00:22:41.070 --> 00:22:45.280 align:middle line:84% So this is what you can do, just one approach. 00:22:45.280 --> 00:22:47.670 align:middle line:84% But maybe in your assignments, you'll 00:22:47.670 --> 00:22:50.520 align:middle line:90% come up with another approach. 00:22:50.520 --> 00:22:52.350 align:middle line:84% In this case, you can just take-- 00:22:52.350 --> 00:22:54.340 align:middle line:90% just compute the scene contrast. 00:22:54.340 --> 00:22:56.700 align:middle line:84% You can just take a five-by-five window 00:22:56.700 --> 00:22:59.770 align:middle line:84% and find pixels that have very high-variance. 00:22:59.770 --> 00:23:02.790 align:middle line:84% And you can create a stencil like this 00:23:02.790 --> 00:23:05.070 align:middle line:84% that says, oh, all these pixels look interesting. 00:23:05.070 --> 00:23:07.470 align:middle line:84% And this part here, clearly, there's nothing. 00:23:07.470 --> 00:23:11.070 align:middle line:84% So I should not take that from the nighttime image. 00:23:11.070 --> 00:23:13.290 align:middle line:90% And then, I can create-- 00:23:13.290 --> 00:23:14.850 align:middle line:90% I can use the stencil. 00:23:14.850 --> 00:23:16.770 align:middle line:84% The black pixels will come from daytime. 00:23:16.770 --> 00:23:18.870 align:middle line:84% And the white pixels will come from nighttime. 00:23:18.870 --> 00:23:20.670 align:middle line:84% And if you just combine that, combine 00:23:20.670 --> 00:23:23.430 align:middle line:84% those two using that stencil, you 00:23:23.430 --> 00:23:24.810 align:middle line:90% get a reasonable image already. 00:23:24.810 --> 00:23:28.920 align:middle line:84% But it will look extremely ugly because the transition 00:23:28.920 --> 00:23:31.740 align:middle line:84% from here to here, from nighttime to the daytime, 00:23:31.740 --> 00:23:35.048 align:middle line:90% will not look very natural. 00:23:35.048 --> 00:23:35.840 align:middle line:90% So what can you do? 00:23:35.840 --> 00:23:36.710 align:middle line:90% Any solutions? 00:23:36.710 --> 00:23:39.540 align:middle line:90% 00:23:39.540 --> 00:23:41.190 align:middle line:84% AUDIENCE: Blur the mask [INAUDIBLE].. 00:23:41.190 --> 00:23:41.880 align:middle line:90% RAMESH RASKAR: Blur the mask. 00:23:41.880 --> 00:23:42.547 align:middle line:90% You can do that. 00:23:42.547 --> 00:23:44.550 align:middle line:84% But then, the edges of the building 00:23:44.550 --> 00:23:47.370 align:middle line:84% will get very fuzzy because it looks like part of the building 00:23:47.370 --> 00:23:49.162 align:middle line:84% was in the daytime and part of the building 00:23:49.162 --> 00:23:50.715 align:middle line:90% was in the nighttime. 00:23:50.715 --> 00:23:52.840 align:middle line:84% AUDIENCE: Could you just start-- could you just say 00:23:52.840 --> 00:23:54.170 align:middle line:90% how you made the mask again? 00:23:54.170 --> 00:23:54.670 align:middle line:90% I-- 00:23:54.670 --> 00:23:55.180 align:middle line:90% RAMESH RASKAR: This mask? 00:23:55.180 --> 00:23:55.805 align:middle line:90% AUDIENCE: Yeah. 00:23:55.805 --> 00:23:58.370 align:middle line:84% RAMESH RASKAR: So in the simplest case, 00:23:58.370 --> 00:24:02.140 align:middle line:84% you can just say, given the pixels aren't bright enough. 00:24:02.140 --> 00:24:04.580 align:middle line:90% That is, imagine zero to 255. 00:24:04.580 --> 00:24:07.597 align:middle line:84% Anything over 150, mark it as [INAUDIBLE].. 00:24:07.597 --> 00:24:08.680 align:middle line:90% That's the simplest thing. 00:24:08.680 --> 00:24:11.170 align:middle line:90% But it's not so good. 00:24:11.170 --> 00:24:13.900 align:middle line:84% A slightly better way of doing that is you 00:24:13.900 --> 00:24:16.750 align:middle line:84% go in a window of, say, five-by-five pixels. 00:24:16.750 --> 00:24:19.990 align:middle line:84% And if all the 25 pixels are the same, then 00:24:19.990 --> 00:24:21.760 align:middle line:90% maybe it's not very interesting. 00:24:21.760 --> 00:24:23.410 align:middle line:84% Here, all 25 pixels are the same. 00:24:23.410 --> 00:24:26.485 align:middle line:84% Or in this region, all 25 pixels are the same. 00:24:26.485 --> 00:24:28.360 align:middle line:84% But if there's enough variation between them, 00:24:28.360 --> 00:24:31.270 align:middle line:84% if I just take the variance or the standard deviation 00:24:31.270 --> 00:24:33.910 align:middle line:84% of those 25 pixels and it's high enough, 00:24:33.910 --> 00:24:37.300 align:middle line:84% that looks like there's some information in those 25 pixels 00:24:37.300 --> 00:24:38.410 align:middle line:90% in a five-by-five window. 00:24:38.410 --> 00:24:42.220 align:middle line:84% And you'll have that center of that five-by-five window 00:24:42.220 --> 00:24:44.470 align:middle line:90% as something that's interesting. 00:24:44.470 --> 00:24:45.037 align:middle line:90% Yep? 00:24:45.037 --> 00:24:47.620 align:middle line:84% AUDIENCE: One thing you could do for [INAUDIBLE] the buildings 00:24:47.620 --> 00:24:52.170 align:middle line:84% is that you would shrink the components that are not 00:24:52.170 --> 00:24:54.653 align:middle line:90% [INAUDIBLE]. 00:24:54.653 --> 00:24:56.070 align:middle line:84% RAMESH RASKAR: So you can use some 00:24:56.070 --> 00:24:59.070 align:middle line:84% a graph card approach and find some connected components 00:24:59.070 --> 00:25:00.582 align:middle line:90% and so on, like that? 00:25:00.582 --> 00:25:02.040 align:middle line:84% AUDIENCE: [INAUDIBLE] a little bit, 00:25:02.040 --> 00:25:05.495 align:middle line:84% you could have taken any immersive photos and just, 00:25:05.495 --> 00:25:07.657 align:middle line:84% between them, defined motion or [INAUDIBLE].. 00:25:07.657 --> 00:25:08.740 align:middle line:90% RAMESH RASKAR: Great idea. 00:25:08.740 --> 00:25:10.550 align:middle line:84% Great idea, especially for something 00:25:10.550 --> 00:25:12.177 align:middle line:90% that's moving to find that. 00:25:12.177 --> 00:25:13.010 align:middle line:90% [INTERPOSING VOICES] 00:25:13.010 --> 00:25:14.385 align:middle line:84% RAMESH RASKAR: But you still have 00:25:14.385 --> 00:25:17.730 align:middle line:84% to take some data images to fill in the static parts that 00:25:17.730 --> 00:25:18.230 align:middle line:90% are dark. 00:25:18.230 --> 00:25:18.470 align:middle line:90% AUDIENCE: Yeah. 00:25:18.470 --> 00:25:19.803 align:middle line:90% RAMESH RASKAR: But that's right. 00:25:19.803 --> 00:25:22.070 align:middle line:84% In fact, that you have to do that for cars 00:25:22.070 --> 00:25:25.490 align:middle line:84% because some cars, the center of this car is too dark. 00:25:25.490 --> 00:25:28.350 align:middle line:90% But it's still important to us. 00:25:28.350 --> 00:25:31.340 align:middle line:84% So this is what simple processing would do. 00:25:31.340 --> 00:25:34.610 align:middle line:84% But maybe you want to do something smart. 00:25:34.610 --> 00:25:39.530 align:middle line:84% So this is just a zoomed in version of that. 00:25:39.530 --> 00:25:40.790 align:middle line:90% It's trying to capture-- 00:25:40.790 --> 00:25:46.130 align:middle line:84% all these windows did not have lights on, you see here. 00:25:46.130 --> 00:25:48.410 align:middle line:84% But there's one window with a light on. 00:25:48.410 --> 00:25:52.420 align:middle line:84% I think this is the scaled up after biogen building, Amgen 00:25:52.420 --> 00:25:54.130 align:middle line:90% building. 00:25:54.130 --> 00:25:56.830 align:middle line:90% And it captures that. 00:25:56.830 --> 00:25:59.565 align:middle line:84% But then, it creates this ugly artifact around that. 00:25:59.565 --> 00:26:01.690 align:middle line:84% So you want a very nice transition between the two. 00:26:01.690 --> 00:26:04.980 align:middle line:90% 00:26:04.980 --> 00:26:08.240 align:middle line:84% So the solution for that is actually 00:26:08.240 --> 00:26:11.360 align:middle line:90% called a gradient dom infusion. 00:26:11.360 --> 00:26:12.980 align:middle line:84% And that's just a fancy way of saying 00:26:12.980 --> 00:26:16.760 align:middle line:84% that instead of worrying about absolute intensities, 00:26:16.760 --> 00:26:19.340 align:middle line:84% I'm going to worry about differences 00:26:19.340 --> 00:26:21.480 align:middle line:90% between every pixel. 00:26:21.480 --> 00:26:24.950 align:middle line:84% So I'm going to take a pixel x plus one and pixel x 00:26:24.950 --> 00:26:26.760 align:middle line:84% and find the difference between them. 00:26:26.760 --> 00:26:29.630 align:middle line:84% And I'm going to blend those ones. 00:26:29.630 --> 00:26:33.500 align:middle line:84% So this is what would happen if you just blend the intensities. 00:26:33.500 --> 00:26:36.950 align:middle line:84% And this is what would happen if you blend the differences. 00:26:36.950 --> 00:26:39.690 align:middle line:84% Now if you blend the differences, which is just 00:26:39.690 --> 00:26:42.590 align:middle line:84% the forward difference, then the image that you will get 00:26:42.590 --> 00:26:44.910 align:middle line:84% will be also made up of forward differences. 00:26:44.910 --> 00:26:47.480 align:middle line:84% So to go from a forward difference back to the signal, 00:26:47.480 --> 00:26:49.520 align:middle line:90% we must do integration. 00:26:49.520 --> 00:26:51.930 align:middle line:90% And in 1D, it's very easy. 00:26:51.930 --> 00:26:53.880 align:middle line:90% And we'll study that later. 00:26:53.880 --> 00:26:55.980 align:middle line:84% But in 2D, it's quite challenging. 00:26:55.980 --> 00:26:58.250 align:middle line:84% So it's basically doing a 2D integration 00:26:58.250 --> 00:27:01.472 align:middle line:84% from all the forward differences and putting the two together. 00:27:01.472 --> 00:27:03.180 align:middle line:84% Again, I'm not going through the details. 00:27:03.180 --> 00:27:06.470 align:middle line:90% We'll study this more. 00:27:06.470 --> 00:27:09.950 align:middle line:84% It turns out you can create various very 00:27:09.950 --> 00:27:13.820 align:middle line:84% aesthetically pleasing fusions or blending 00:27:13.820 --> 00:27:14.970 align:middle line:90% between different parts. 00:27:14.970 --> 00:27:15.470 align:middle line:90% Yeah? 00:27:15.470 --> 00:27:19.610 align:middle line:84% AUDIENCE: What if-- this is just an example of a night and day. 00:27:19.610 --> 00:27:23.453 align:middle line:84% And what if you multiply the images and then normalize them? 00:27:23.453 --> 00:27:25.370 align:middle line:84% RAMESH RASKAR: You could try different things. 00:27:25.370 --> 00:27:27.820 align:middle line:84% AUDIENCE: Because the dark parts would come through. 00:27:27.820 --> 00:27:30.356 align:middle line:84% And the light parts from the day would also come through. 00:27:30.356 --> 00:27:32.690 align:middle line:84% And they might actually merge a little bit [INAUDIBLE].. 00:27:32.690 --> 00:27:33.565 align:middle line:90% RAMESH RASKAR: Right. 00:27:33.565 --> 00:27:35.388 align:middle line:84% So, Nina, these are some of the things-- 00:27:35.388 --> 00:27:37.430 align:middle line:84% it could work in one scenario, but not the other. 00:27:37.430 --> 00:27:38.030 align:middle line:90% You're right. 00:27:38.030 --> 00:27:40.790 align:middle line:84% And this is exactly what I want you to do in the assignment. 00:27:40.790 --> 00:27:46.580 align:middle line:84% The assignment gives you a very simple and very easy problem. 00:27:46.580 --> 00:27:51.530 align:middle line:84% But instead of taking pictures under two different flash table 00:27:51.530 --> 00:27:53.420 align:middle line:84% lab conditions, you could take a picture 00:27:53.420 --> 00:27:56.850 align:middle line:84% under two time instances, one in the daytime, 00:27:56.850 --> 00:27:58.400 align:middle line:84% one in the nighttime, or under two 00:27:58.400 --> 00:28:01.190 align:middle line:84% different polarizations or anything, and then fuse them. 00:28:01.190 --> 00:28:02.810 align:middle line:84% Instead of fusing color channels, 00:28:02.810 --> 00:28:05.450 align:middle line:90% you can fuse intensity channels. 00:28:05.450 --> 00:28:09.260 align:middle line:84% Or you can fuse the forward difference channels and so on. 00:28:09.260 --> 00:28:12.720 align:middle line:84% One great software, it's on the website, 00:28:12.720 --> 00:28:19.100 align:middle line:84% I would like you to use for testing is called HDR Shop. 00:28:19.100 --> 00:28:22.364 align:middle line:90% 00:28:22.364 --> 00:28:24.450 align:middle line:90% And it's available for free. 00:28:24.450 --> 00:28:29.640 align:middle line:84% It's from a group at USC led by Paul Debevec. 00:28:29.640 --> 00:28:31.640 align:middle line:84% And I highly recommend that because it certainly 00:28:31.640 --> 00:28:33.140 align:middle line:90% isn't Photoshop. 00:28:33.140 --> 00:28:36.620 align:middle line:84% If you use HDR Shop, again, even if you're 00:28:36.620 --> 00:28:39.620 align:middle line:84% doing MATLAB assignments, you can download HDR Shop. 00:28:39.620 --> 00:28:42.260 align:middle line:84% And it allows you to do some [INAUDIBLE].. 00:28:42.260 --> 00:28:44.870 align:middle line:84% You can sample images [INAUDIBLE].. 00:28:44.870 --> 00:28:46.310 align:middle line:90% You can subtract images. 00:28:46.310 --> 00:28:48.050 align:middle line:90% You can do forward differences. 00:28:48.050 --> 00:28:50.670 align:middle line:84% You can do volume densities and all those things. 00:28:50.670 --> 00:28:57.770 align:middle line:84% So it's a really good software for doing arithmetic on images. 00:28:57.770 --> 00:29:01.760 align:middle line:84% And the other [INAUDIBLE] is done in floating point. 00:29:01.760 --> 00:29:04.740 align:middle line:90% It's not trapped [INAUDIBLE]. 00:29:04.740 --> 00:29:07.100 align:middle line:84% So let's say you want to multiply two images. 00:29:07.100 --> 00:29:11.030 align:middle line:84% And the intensities are at 200 and 200. 00:29:11.030 --> 00:29:13.520 align:middle line:90% It's 40,000 [INAUDIBLE]. 00:29:13.520 --> 00:29:18.210 align:middle line:84% And Photoshop only supports floating point numbers. 00:29:18.210 --> 00:29:21.535 align:middle line:90% But HDR Shop will support that. 00:29:21.535 --> 00:29:22.390 align:middle line:90% [INAUDIBLE] 00:29:22.390 --> 00:29:25.640 align:middle line:90% 00:29:25.640 --> 00:29:27.460 align:middle line:84% So even before, sometimes, you're 00:29:27.460 --> 00:29:29.680 align:middle line:84% writing a MATLAB program, you can very quickly 00:29:29.680 --> 00:29:35.620 align:middle line:84% do some visual inspection on this and [INAUDIBLE].. 00:29:35.620 --> 00:29:37.120 align:middle line:84% Another thing a lot of people forget 00:29:37.120 --> 00:29:41.800 align:middle line:84% about when you're doing assignments or projects that 00:29:41.800 --> 00:29:45.910 align:middle line:84% involve images is don't work on the whole image. 00:29:45.910 --> 00:29:48.310 align:middle line:84% Work on a very tiny part of the image. 00:29:48.310 --> 00:29:51.190 align:middle line:84% Just take a 100 pixel by 100 pixel crop, 00:29:51.190 --> 00:29:52.990 align:middle line:90% largest size, but just a crop. 00:29:52.990 --> 00:29:56.050 align:middle line:84% So you have some two megapixel images. 00:29:56.050 --> 00:29:58.510 align:middle line:84% Don't try to run your core on a two-megapixel image. 00:29:58.510 --> 00:29:59.920 align:middle line:90% It might take forever. 00:29:59.920 --> 00:30:03.250 align:middle line:84% Just take a very tiny part from a very interesting part. 00:30:03.250 --> 00:30:06.520 align:middle line:84% [INAUDIBLE] run all your code on that. 00:30:06.520 --> 00:30:10.420 align:middle line:84% And when you feel that all your procedures are working 00:30:10.420 --> 00:30:13.750 align:middle line:84% [INAUDIBLE],, then you can go back to maybe 00:30:13.750 --> 00:30:15.270 align:middle line:90% some larger [? optical ?] image. 00:30:15.270 --> 00:30:21.220 align:middle line:90% 00:30:21.220 --> 00:30:25.120 align:middle line:84% So when I was working on this portrait, actually, 00:30:25.120 --> 00:30:33.490 align:middle line:84% I was very inspired by this surrealism, AR surrealism, 00:30:33.490 --> 00:30:36.640 align:middle line:84% where René Magritte and all that came up with Empire 00:30:36.640 --> 00:30:37.820 align:middle line:90% of the Lights. 00:30:37.820 --> 00:30:40.690 align:middle line:84% I'm sure a lot of you have seen this painting, where 00:30:40.690 --> 00:30:49.790 align:middle line:84% he created this very stark, disconcerting positioning. 00:30:49.790 --> 00:30:53.180 align:middle line:90% You have bright daylight here. 00:30:53.180 --> 00:30:59.030 align:middle line:84% But the bottom part of the image is a nighttime scene. 00:30:59.030 --> 00:31:04.700 align:middle line:84% And he played with a lot of this juxtaposing of disconcerting 00:31:04.700 --> 00:31:05.360 align:middle line:90% elements. 00:31:05.360 --> 00:31:08.720 align:middle line:84% And the question is, can photography 00:31:08.720 --> 00:31:13.970 align:middle line:84% create something that is surrrealistic, not always 00:31:13.970 --> 00:31:17.550 align:middle line:84% photorealistic, but surrrealistic? 00:31:17.550 --> 00:31:20.780 align:middle line:84% And when we come to a discussion about, 00:31:20.780 --> 00:31:23.150 align:middle line:84% will photos survive in the future, 00:31:23.150 --> 00:31:29.150 align:middle line:84% this is exactly the question that painters ask themselves 00:31:29.150 --> 00:31:30.600 align:middle line:90% when cameras are measured. 00:31:30.600 --> 00:31:33.110 align:middle line:84% There was a lot of interest in creating 00:31:33.110 --> 00:31:36.950 align:middle line:90% photorealistic paintings. 00:31:36.950 --> 00:31:39.860 align:middle line:84% And when camera came around, they all scratched their heads 00:31:39.860 --> 00:31:42.950 align:middle line:84% and said, wow, we need to move on and do something else. 00:31:42.950 --> 00:31:45.780 align:middle line:84% We cannot create photorealistic paintings anymore. 00:31:45.780 --> 00:31:48.290 align:middle line:84% And then, we had pointillism and surrealism. 00:31:48.290 --> 00:31:50.600 align:middle line:90% And now, it's just abstract. 00:31:50.600 --> 00:31:53.150 align:middle line:84% We don't even care about representing geometry, 00:31:53.150 --> 00:31:55.350 align:middle line:90% and reflectance, and so on. 00:31:55.350 --> 00:31:58.370 align:middle line:84% And the question is, should cameras 00:31:58.370 --> 00:32:04.220 align:middle line:84% worry about capturing intensity, and shapes, and geometry? 00:32:04.220 --> 00:32:05.750 align:middle line:84% When I check on Flickr, there are 00:32:05.750 --> 00:32:08.390 align:middle line:84% hundreds of images of buildings like this. 00:32:08.390 --> 00:32:12.290 align:middle line:84% What's so great about taking yet another photo of yet 00:32:12.290 --> 00:32:14.000 align:middle line:90% another building? 00:32:14.000 --> 00:32:17.240 align:middle line:84% What I want to create is images that are somehow 00:32:17.240 --> 00:32:19.760 align:middle line:90% more expressive and more real. 00:32:19.760 --> 00:32:24.620 align:middle line:84% So that's another debate, whether photos [INAUDIBLE].. 00:32:24.620 --> 00:32:27.170 align:middle line:84% I personally, by the way, have completely given up on camera. 00:32:27.170 --> 00:32:34.740 align:middle line:84% I don't-- I barely take photos when I travel and so on. 00:32:34.740 --> 00:32:35.990 align:middle line:90% It's just one of those things. 00:32:35.990 --> 00:32:39.070 align:middle line:84% I'm well beyond the stage of taking photos. 00:32:39.070 --> 00:32:43.067 align:middle line:84% And I rarely get impressed with beautiful photos. 00:32:43.067 --> 00:32:43.900 align:middle line:90% But these look nice. 00:32:43.900 --> 00:32:46.550 align:middle line:84% But I never get impressed with that. 00:32:46.550 --> 00:32:49.570 align:middle line:84% And hopefully, the remaining six billion 00:32:49.570 --> 00:32:53.320 align:middle line:84% people will also start challenging 00:32:53.320 --> 00:33:00.030 align:middle line:84% the notion of camera as a device that captures photorealism. 00:33:00.030 --> 00:33:02.040 align:middle line:84% People will challenge that notion 00:33:02.040 --> 00:33:07.100 align:middle line:84% just like the painters did during the Renaissance 00:33:07.100 --> 00:33:08.960 align:middle line:90% and [INAUDIBLE]. 00:33:08.960 --> 00:33:11.956 align:middle line:90% So any thoughts, [INAUDIBLE]? 00:33:11.956 --> 00:33:12.710 align:middle line:90% AUDIENCE: Huh? 00:33:12.710 --> 00:33:14.447 align:middle line:90% RAMESH RASKAR: Any thoughts? 00:33:14.447 --> 00:33:15.280 align:middle line:90% [INTERPOSING VOICES] 00:33:15.280 --> 00:33:19.980 align:middle line:84% AUDIENCE: There-- you've never seen 00:33:19.980 --> 00:33:23.065 align:middle line:84% a photo done by a photographer, let's 00:33:23.065 --> 00:33:26.785 align:middle line:84% say, Richard Avedon or Ansel Adams that you really liked, 00:33:26.785 --> 00:33:27.285 align:middle line:90% that 00:33:27.285 --> 00:33:27.560 align:middle line:90% [INTERPOSING VOICES] 00:33:27.560 --> 00:33:28.190 align:middle line:90% RAMESH RASKAR: It's beautiful. 00:33:28.190 --> 00:33:29.615 align:middle line:90% But I'm not impressed. 00:33:29.615 --> 00:33:31.118 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:33:31.118 --> 00:33:32.660 align:middle line:84% RAMESH RASKAR: Can you say it louder? 00:33:32.660 --> 00:33:36.260 align:middle line:84% She said, what about Ansel Adams and so on? 00:33:36.260 --> 00:33:37.347 align:middle line:90% Yeah, back then-- 00:33:37.347 --> 00:33:38.180 align:middle line:90% [INTERPOSING VOICES] 00:33:38.180 --> 00:33:40.388 align:middle line:84% RAMESH RASKAR: --when they did it, it was impressive. 00:33:40.388 --> 00:33:43.070 align:middle line:84% But now, if somebody shows me something as good as Ansel 00:33:43.070 --> 00:33:44.840 align:middle line:90% Adams, it's great. 00:33:44.840 --> 00:33:47.640 align:middle line:90% But it's not interesting. 00:33:47.640 --> 00:33:52.150 align:middle line:84% AUDIENCE: Well, you should look at some of Richard Avedon's 00:33:52.150 --> 00:33:54.550 align:middle line:90% more recent works. 00:33:54.550 --> 00:33:57.290 align:middle line:84% He does some pretty creative things. 00:33:57.290 --> 00:34:00.053 align:middle line:84% There's one, he use a skeleton in some of his photos. 00:34:00.053 --> 00:34:00.970 align:middle line:90% And it's very serious. 00:34:00.970 --> 00:34:03.007 align:middle line:84% RAMESH RASKAR: Yeah, so the content sometimes 00:34:03.007 --> 00:34:03.590 align:middle line:90% is impressive. 00:34:03.590 --> 00:34:04.507 align:middle line:90% AUDIENCE: Right, yeah. 00:34:04.507 --> 00:34:08.578 align:middle line:84% RAMESH RASKAR: But the art of photography. 00:34:08.578 --> 00:34:09.370 align:middle line:90% Comment back there? 00:34:09.370 --> 00:34:12.310 align:middle line:90% 00:34:12.310 --> 00:34:13.810 align:middle line:84% Somebody had a comment back there? 00:34:13.810 --> 00:34:14.310 align:middle line:90% All right. 00:34:14.310 --> 00:34:16.507 align:middle line:90% AUDIENCE: I had one [INAUDIBLE]. 00:34:16.507 --> 00:34:19.159 align:middle line:84% I'm not sure you can separate both the technique 00:34:19.159 --> 00:34:21.752 align:middle line:90% from the content. 00:34:21.752 --> 00:34:23.810 align:middle line:90% So this is really discretion. 00:34:23.810 --> 00:34:26.300 align:middle line:84% But I think they can come together 00:34:26.300 --> 00:34:29.460 align:middle line:84% in creating and influencing each other's thoughts. 00:34:29.460 --> 00:34:32.900 align:middle line:84% RAMESH RASKAR: You definitely need better and better tools 00:34:32.900 --> 00:34:36.170 align:middle line:84% and technologies to be able to create something 00:34:36.170 --> 00:34:38.040 align:middle line:84% that's really expressive, really, 00:34:38.040 --> 00:34:43.040 align:middle line:90% really beautiful and creative. 00:34:43.040 --> 00:34:48.409 align:middle line:84% But there was a time when photographers could impress us 00:34:48.409 --> 00:34:49.880 align:middle line:90% based on their techniques. 00:34:49.880 --> 00:34:52.820 align:middle line:84% The guy with the biggest camera was 00:34:52.820 --> 00:34:55.730 align:middle line:84% called a professional photographer, not the guy 00:34:55.730 --> 00:35:00.650 align:middle line:84% with the best eye and somebody who has the patience, 00:35:00.650 --> 00:35:02.750 align:middle line:84% and aesthetics, and all of those things. 00:35:02.750 --> 00:35:05.630 align:middle line:84% And hopefully, now we have separated the two. 00:35:05.630 --> 00:35:08.840 align:middle line:84% Just because the guy has the best gear 00:35:08.840 --> 00:35:11.455 align:middle line:84% doesn't mean he's a great photographer anymore. 00:35:11.455 --> 00:35:12.580 align:middle line:90% And that's what I mean by-- 00:35:12.580 --> 00:35:15.920 align:middle line:90% 00:35:15.920 --> 00:35:16.420 align:middle line:90% yes? 00:35:16.420 --> 00:35:17.350 align:middle line:84% AUDIENCE: [? Great book ?] on that topic 00:35:17.350 --> 00:35:19.225 align:middle line:84% as well, going back to old master techniques. 00:35:19.225 --> 00:35:21.527 align:middle line:84% It's called Secret Knowledge by David Hockney. 00:35:21.527 --> 00:35:23.360 align:middle line:84% I don't know if everybody else has read that 00:35:23.360 --> 00:35:24.780 align:middle line:90% [INAUDIBLE] all the different 00:35:24.780 --> 00:35:25.050 align:middle line:90% [INTERPOSING VOICES] 00:35:25.050 --> 00:35:26.508 align:middle line:84% RAMESH RASKAR: Can you just repeat? 00:35:26.508 --> 00:35:27.300 align:middle line:90% Is David Hockney-- 00:35:27.300 --> 00:35:28.300 align:middle line:90% AUDIENCE: David Hockney. 00:35:28.300 --> 00:35:29.790 align:middle line:90% It's called Secret Knowledge. 00:35:29.790 --> 00:35:32.130 align:middle line:90% And it's going through all-- 00:35:32.130 --> 00:35:34.385 align:middle line:84% he worked backwards based on weird artifacts 00:35:34.385 --> 00:35:35.760 align:middle line:84% and their paintings to figure out 00:35:35.760 --> 00:35:38.943 align:middle line:84% how they would make weird optical devices that could 00:35:38.943 --> 00:35:40.110 align:middle line:90% create the original artwork. 00:35:40.110 --> 00:35:41.950 align:middle line:84% And that goes back to the same concept 00:35:41.950 --> 00:35:44.437 align:middle line:84% of technique, and artists, and where those two 00:35:44.437 --> 00:35:45.270 align:middle line:90% [INTERPOSING VOICES] 00:35:45.270 --> 00:35:46.750 align:middle line:84% AUDIENCE: So there were from Ricoh? 00:35:46.750 --> 00:35:47.875 align:middle line:90% RAMESH RASKAR: What's that? 00:35:47.875 --> 00:35:48.764 align:middle line:90% AUDIENCE: From Ricoh? 00:35:48.764 --> 00:35:49.221 align:middle line:90% RAMESH RASKAR: What do you mean? 00:35:49.221 --> 00:35:49.721 align:middle line:90% What? 00:35:49.721 --> 00:35:52.049 align:middle line:84% AUDIENCE: The company Ricoh, he's from that. 00:35:52.049 --> 00:35:52.882 align:middle line:90% RAMESH RASKAR: What? 00:35:52.882 --> 00:35:53.706 align:middle line:90% David Hockney. 00:35:53.706 --> 00:35:54.120 align:middle line:90% AUDIENCE: The painter. 00:35:54.120 --> 00:35:54.540 align:middle line:90% RAMESH RASKAR: Yeah. 00:35:54.540 --> 00:35:55.873 align:middle line:90% AUDIENCE: He's a famous painter. 00:35:55.873 --> 00:35:57.870 align:middle line:90% RAMESH RASKAR: Oh, sorry. 00:35:57.870 --> 00:36:00.120 align:middle line:84% Who's the guy from Rico who did all the same analysis? 00:36:00.120 --> 00:36:03.700 align:middle line:90% 00:36:03.700 --> 00:36:07.803 align:middle line:84% But this is the one where they looked at-- 00:36:07.803 --> 00:36:09.220 align:middle line:84% AUDIENCE: It's an interesting read 00:36:09.220 --> 00:36:11.830 align:middle line:84% if anybody's interested on the history of how 00:36:11.830 --> 00:36:14.872 align:middle line:84% these optical techniques played into the role of artists 00:36:14.872 --> 00:36:15.372 align:middle line:90% [INAUDIBLE]. 00:36:15.372 --> 00:36:15.850 align:middle line:90% It's controversial 00:36:15.850 --> 00:36:16.330 align:middle line:90% [INTERPOSING VOICES] 00:36:16.330 --> 00:36:17.497 align:middle line:90% RAMESH RASKAR: Yes, exactly. 00:36:17.497 --> 00:36:20.100 align:middle line:84% I think we're talking about the same thing. 00:36:20.100 --> 00:36:21.100 align:middle line:90% It's very controversial. 00:36:21.100 --> 00:36:24.040 align:middle line:84% And he claims that the painters knew 00:36:24.040 --> 00:36:28.720 align:middle line:84% about perspective and so on even before the Renaissance 00:36:28.720 --> 00:36:29.380 align:middle line:90% and so on. 00:36:29.380 --> 00:36:31.630 align:middle line:84% AUDIENCE: Yeah, there's a lot of interesting backwards 00:36:31.630 --> 00:36:33.047 align:middle line:84% scientific discoveries that he did 00:36:33.047 --> 00:36:35.453 align:middle line:84% based off your artifacts in perspective. 00:36:35.453 --> 00:36:37.400 align:middle line:84% AUDIENCE: Yeah, I think it's David Stork. 00:36:37.400 --> 00:36:37.880 align:middle line:90% RAMESH RASKAR: David Stork 00:36:37.880 --> 00:36:39.050 align:middle line:90% [INTERPOSING VOICES] 00:36:39.050 --> 00:36:39.440 align:middle line:90% AUDIENCE: --Ricoh. 00:36:39.440 --> 00:36:39.725 align:middle line:90% RAMESH RASKAR: Exactly. 00:36:39.725 --> 00:36:40.340 align:middle line:90% AUDIENCE: Yeah. 00:36:40.340 --> 00:36:42.050 align:middle line:84% RAMESH RASKAR: Is that the same thing we're talking about? 00:36:42.050 --> 00:36:43.940 align:middle line:84% AUDIENCE: Different but maybe related? 00:36:43.940 --> 00:36:46.380 align:middle line:84% AUDIENCE: I think they do similar things. 00:36:46.380 --> 00:36:49.895 align:middle line:84% David Stork writes about optical projections for planes. 00:36:49.895 --> 00:36:52.590 align:middle line:90% And so it's probably similar. 00:36:52.590 --> 00:36:54.942 align:middle line:84% RAMESH RASKAR: So David Stork and David Hockney. 00:36:54.942 --> 00:36:57.890 align:middle line:90% All right. 00:36:57.890 --> 00:36:59.230 align:middle line:90% My laptop is recording again. 00:36:59.230 --> 00:37:01.790 align:middle line:90% That's fine. 00:37:01.790 --> 00:37:05.910 align:middle line:84% So yeah, so I think that's a great challenge 00:37:05.910 --> 00:37:07.520 align:middle line:90% we have about how-- 00:37:07.520 --> 00:37:15.470 align:middle line:84% the same way a camera put photorealistic paintings. 00:37:15.470 --> 00:37:19.880 align:middle line:84% They made them boring and basically put that breed 00:37:19.880 --> 00:37:22.430 align:middle line:90% of painters out of business. 00:37:22.430 --> 00:37:27.170 align:middle line:84% What will be the next imaging paradigm 00:37:27.170 --> 00:37:31.570 align:middle line:84% that we'll put to this camera that 00:37:31.570 --> 00:37:37.800 align:middle line:84% would make these cameras boring, obsolete, and out of business? 00:37:37.800 --> 00:37:39.840 align:middle line:84% Because there's only so far you can 00:37:39.840 --> 00:37:46.770 align:middle line:84% go with a device that records light and tries 00:37:46.770 --> 00:37:51.450 align:middle line:84% to be realistic with a good signal-to-noise ratio 00:37:51.450 --> 00:37:54.340 align:middle line:84% converting photons into electrons. 00:37:54.340 --> 00:37:57.240 align:middle line:90% That's not what humans do. 00:37:57.240 --> 00:38:00.410 align:middle line:84% We don't see with our eyes at all. 00:38:00.410 --> 00:38:02.260 align:middle line:90% We just record with our eyes. 00:38:02.260 --> 00:38:04.870 align:middle line:90% And we see with our brain. 00:38:04.870 --> 00:38:07.540 align:middle line:84% And right now, there is very little 00:38:07.540 --> 00:38:10.600 align:middle line:84% suppression in our imaging devices 00:38:10.600 --> 00:38:16.450 align:middle line:90% about sensing and seeing. 00:38:16.450 --> 00:38:20.590 align:middle line:84% But the whole concept behind computational photography 00:38:20.590 --> 00:38:23.740 align:middle line:84% is that these two processes are completely separate. 00:38:23.740 --> 00:38:25.240 align:middle line:90% They can be decoupled. 00:38:25.240 --> 00:38:28.840 align:middle line:84% You can sense in some crazy way with some crazy mechanism 00:38:28.840 --> 00:38:30.910 align:middle line:84% and then use a lot of computation 00:38:30.910 --> 00:38:36.250 align:middle line:84% to create images or create visual experiences that 00:38:36.250 --> 00:38:39.350 align:middle line:90% are very different. 00:38:39.350 --> 00:38:41.650 align:middle line:90% And if you look at-- 00:38:41.650 --> 00:38:43.780 align:middle line:84% Dan in my group and others have put up 00:38:43.780 --> 00:38:48.250 align:middle line:84% together this really beautiful digital totem in our area. 00:38:48.250 --> 00:38:51.340 align:middle line:84% And it's constantly streaming the most interesting images 00:38:51.340 --> 00:38:52.510 align:middle line:90% on Flickr. 00:38:52.510 --> 00:38:54.490 align:middle line:84% And you stand in front of that, you 00:38:54.490 --> 00:38:58.690 align:middle line:84% will realize that if anybody is saying 00:38:58.690 --> 00:39:02.510 align:middle line:84% that the goal of a camera is to mimic a human eye, 00:39:02.510 --> 00:39:06.067 align:middle line:84% and that's what makes it a good camera, 00:39:06.067 --> 00:39:07.900 align:middle line:84% that notion will be challenged, because most 00:39:07.900 --> 00:39:12.070 align:middle line:84% of those photos which are tagged interesting by thousands 00:39:12.070 --> 00:39:15.040 align:middle line:84% of people on Flickr are actually the photos 00:39:15.040 --> 00:39:18.310 align:middle line:90% you don't see with your eyes. 00:39:18.310 --> 00:39:21.880 align:middle line:90% It has some really crazy focus. 00:39:21.880 --> 00:39:28.820 align:middle line:84% It has very different color response curves. 00:39:28.820 --> 00:39:32.360 align:middle line:84% It has very different zoom and so on. 00:39:32.360 --> 00:39:34.150 align:middle line:84% It's basically things-- people think 00:39:34.150 --> 00:39:37.750 align:middle line:84% it's interesting when it's not what you 00:39:37.750 --> 00:39:39.730 align:middle line:90% see with your own naked eye. 00:39:39.730 --> 00:39:43.900 align:middle line:84% And that's what the goal-- if you take the concept for-- 00:39:43.900 --> 00:39:45.850 align:middle line:84% right now, people are trying to use cameras 00:39:45.850 --> 00:39:47.860 align:middle line:84% that are built to mimic the human eye 00:39:47.860 --> 00:39:50.050 align:middle line:84% and pushing them to create things 00:39:50.050 --> 00:39:52.180 align:middle line:90% that don't mimic the human eye. 00:39:52.180 --> 00:39:55.420 align:middle line:84% If you take that further, why not 00:39:55.420 --> 00:39:57.760 align:middle line:84% start building imaging platforms that 00:39:57.760 --> 00:40:02.570 align:middle line:84% have nothing to do with capturing a 2D perspective 00:40:02.570 --> 00:40:04.130 align:middle line:90% photo? 00:40:04.130 --> 00:40:05.630 align:middle line:84% And that's what we'll see real soon. 00:40:05.630 --> 00:40:09.190 align:middle line:84% So I'm looking towards an imaging platform that can just 00:40:09.190 --> 00:40:11.348 align:middle line:90% create abstract visual art. 00:40:11.348 --> 00:40:12.640 align:middle line:90% I don't know what that will be. 00:40:12.640 --> 00:40:14.434 align:middle line:90% But go ahead. 00:40:14.434 --> 00:40:17.302 align:middle line:84% AUDIENCE: I guess this is more of a question 00:40:17.302 --> 00:40:19.214 align:middle line:90% or a topic for your discussion. 00:40:19.214 --> 00:40:22.706 align:middle line:90% But what do you imagine the-- 00:40:22.706 --> 00:40:26.545 align:middle line:84% what is the fundamental photography or class 00:40:26.545 --> 00:40:33.700 align:middle line:84% of images that makes it lasting as opposed to the art tied 00:40:33.700 --> 00:40:38.080 align:middle line:90% to technology, which is batty. 00:40:38.080 --> 00:40:41.318 align:middle line:84% In the past year or so, [INAUDIBLE] photography. 00:40:41.318 --> 00:40:42.527 align:middle line:90% And they 00:40:42.527 --> 00:40:43.360 align:middle line:90% [INTERPOSING VOICES] 00:40:43.360 --> 00:40:44.480 align:middle line:90% [LAUGHTER] 00:40:44.480 --> 00:40:44.610 align:middle line:90% RAMESH RASKAR: Right. 00:40:44.610 --> 00:40:46.550 align:middle line:84% AUDIENCE: So it's entering the life of kitsch. 00:40:46.550 --> 00:40:47.780 align:middle line:90% RAMESH RASKAR: Right. 00:40:47.780 --> 00:40:51.800 align:middle line:84% AUDIENCE: Or even in HDR, you see a lot of-- 00:40:51.800 --> 00:40:55.070 align:middle line:90% it was in the beginning. 00:40:55.070 --> 00:40:58.868 align:middle line:84% And now, it feels like it's run its course. 00:40:58.868 --> 00:40:59.660 align:middle line:90% RAMESH RASKAR: Yes. 00:40:59.660 --> 00:41:01.130 align:middle line:84% AUDIENCE: And so but there's still 00:41:01.130 --> 00:41:05.180 align:middle line:84% photography in the traditional sense of capturing 00:41:05.180 --> 00:41:10.030 align:middle line:84% what the human eye sees is fundamental in some way, which 00:41:10.030 --> 00:41:16.960 align:middle line:84% is separate from the technique, things like [INAUDIBLE].. 00:41:16.960 --> 00:41:19.390 align:middle line:84% So I guess the question that I have is, 00:41:19.390 --> 00:41:23.350 align:middle line:84% what is the fundamental of the photographic medium? 00:41:23.350 --> 00:41:26.230 align:middle line:84% And in terms of technology, what would 00:41:26.230 --> 00:41:30.050 align:middle line:84% be the fundamental technological [? ability ?] as opposed 00:41:30.050 --> 00:41:31.882 align:middle line:90% to separating that from kitsch? 00:41:31.882 --> 00:41:32.840 align:middle line:90% RAMESH RASKAR: I agree. 00:41:32.840 --> 00:41:33.360 align:middle line:90% I agree. 00:41:33.360 --> 00:41:38.630 align:middle line:84% I think the camera, to me, it's just a sensor. 00:41:38.630 --> 00:41:40.400 align:middle line:90% It's just a transducer. 00:41:40.400 --> 00:41:43.970 align:middle line:84% And people say, wow, if the camera 00:41:43.970 --> 00:41:47.330 align:middle line:84% is going to change, fundamentally, 00:41:47.330 --> 00:41:49.610 align:middle line:84% it's still going to convert electrons 00:41:49.610 --> 00:41:52.180 align:middle line:90% into photons and photons into-- 00:41:52.180 --> 00:41:54.530 align:middle line:84% sorry, photons into electrons, and electrons 00:41:54.530 --> 00:41:57.170 align:middle line:90% into pixels, and so on. 00:41:57.170 --> 00:42:01.200 align:middle line:84% The laws of physics are not going to change anytime soon. 00:42:01.200 --> 00:42:08.810 align:middle line:84% So how we use it, though, it's going to change altogether. 00:42:08.810 --> 00:42:13.040 align:middle line:84% And this notion that once you take a picture, that's set, 00:42:13.040 --> 00:42:16.850 align:middle line:84% use the best possible optics and sensor 00:42:16.850 --> 00:42:19.880 align:middle line:84% so that it captured image and that you can see later 00:42:19.880 --> 00:42:20.720 align:middle line:90% is changing. 00:42:20.720 --> 00:42:22.147 align:middle line:90% So it will shift. 00:42:22.147 --> 00:42:24.230 align:middle line:84% I'm glad you brought it up because that's the very 00:42:24.230 --> 00:42:25.522 align:middle line:90% next thing I was going to show. 00:42:25.522 --> 00:42:28.370 align:middle line:90% 00:42:28.370 --> 00:42:29.990 align:middle line:90% This is going before that. 00:42:29.990 --> 00:42:32.340 align:middle line:90% I'll come to this. 00:42:32.340 --> 00:42:35.000 align:middle line:84% [INAUDIBLE] some other interesting images 00:42:35.000 --> 00:42:36.110 align:middle line:90% are time lapse mosaics. 00:42:36.110 --> 00:42:41.330 align:middle line:84% So again, I sat in my office and took these time lapse pictures. 00:42:41.330 --> 00:42:43.380 align:middle line:84% And on the left side, it's daytime. 00:42:43.380 --> 00:42:45.620 align:middle line:90% Bright time, it's nighttime. 00:42:45.620 --> 00:42:50.720 align:middle line:84% And all I'm doing is just picking up strips of the images 00:42:50.720 --> 00:42:52.520 align:middle line:90% from different times of day. 00:42:52.520 --> 00:42:55.250 align:middle line:84% And [INAUDIBLE] annoying artifacts 00:42:55.250 --> 00:42:57.410 align:middle line:84% because it's on some other exposure. 00:42:57.410 --> 00:43:00.228 align:middle line:84% But using a gradient domain technique, 00:43:00.228 --> 00:43:01.895 align:middle line:84% you can create a very smooth transition. 00:43:01.895 --> 00:43:03.950 align:middle line:90% AUDIENCE: [INAUDIBLE] cop car. 00:43:03.950 --> 00:43:06.372 align:middle line:90% AUDIENCE: Yeah, cop cars and-- 00:43:06.372 --> 00:43:07.580 align:middle line:90% RAMESH RASKAR: Yeah, exactly. 00:43:07.580 --> 00:43:09.740 align:middle line:90% It still has those problems. 00:43:09.740 --> 00:43:11.180 align:middle line:84% So I don't have as much patience. 00:43:11.180 --> 00:43:13.550 align:middle line:84% And I'm only doing it sitting in my office. 00:43:13.550 --> 00:43:19.700 align:middle line:84% But it's just a tool that photographers could create, 00:43:19.700 --> 00:43:23.780 align:middle line:84% create images that have a very different notion than what 00:43:23.780 --> 00:43:26.185 align:middle line:84% your eye will see if you're there sitting in an office. 00:43:26.185 --> 00:43:27.560 align:middle line:84% Of course, hopefully, this can be 00:43:27.560 --> 00:43:31.160 align:middle line:84% done in Times Square or some really interesting places. 00:43:31.160 --> 00:43:34.280 align:middle line:84% And by the way, this one is also catching up now on Flickr, 00:43:34.280 --> 00:43:36.230 align:middle line:84% a lot of people who are trying to do this. 00:43:36.230 --> 00:43:38.918 align:middle line:90% We did this in 2002. 00:43:38.918 --> 00:43:39.830 align:middle line:90% But-- 00:43:39.830 --> 00:43:43.040 align:middle line:84% AUDIENCE: So in lots of examples we saw here, 00:43:43.040 --> 00:43:48.860 align:middle line:84% the final result is an image like a final master in sound, 00:43:48.860 --> 00:43:52.640 align:middle line:90% where you do all equalizing. 00:43:52.640 --> 00:43:53.720 align:middle line:90% There is one sound. 00:43:53.720 --> 00:43:56.540 align:middle line:84% And lots of techniques now could be also interactive. 00:43:56.540 --> 00:44:00.115 align:middle line:84% You could actually go and those will be made. 00:44:00.115 --> 00:44:01.490 align:middle line:84% It goes through every part of it, 00:44:01.490 --> 00:44:05.310 align:middle line:84% because that is the final goal is to understand it. 00:44:05.310 --> 00:44:07.940 align:middle line:84% There is this idea that there is something complex about who 00:44:07.940 --> 00:44:09.326 align:middle line:90% [INAUDIBLE]. 00:44:09.326 --> 00:44:10.845 align:middle line:84% And it could be interactive as well. 00:44:10.845 --> 00:44:11.720 align:middle line:90% RAMESH RASKAR: Right. 00:44:11.720 --> 00:44:13.760 align:middle line:84% AUDIENCE: So in one sense, the [INAUDIBLE] goal 00:44:13.760 --> 00:44:19.280 align:middle line:84% is always to create a new static image or media. 00:44:19.280 --> 00:44:22.292 align:middle line:84% The interactive parts would be so interesting. 00:44:22.292 --> 00:44:23.750 align:middle line:84% RAMESH RASKAR: Yeah, I think you're 00:44:23.750 --> 00:44:27.290 align:middle line:84% pointing to a very important fundamental, again, 00:44:27.290 --> 00:44:28.790 align:middle line:84% constraint we have put on ourselves. 00:44:28.790 --> 00:44:33.715 align:middle line:84% That an imaging device should create, eventually, a 2D photo. 00:44:33.715 --> 00:44:34.340 align:middle line:90% AUDIENCE: Yeah. 00:44:34.340 --> 00:44:37.550 align:middle line:84% RAMESH RASKAR: And that's very limiting, extremely limiting. 00:44:37.550 --> 00:44:40.910 align:middle line:84% And we will be spending a little bit of time 00:44:40.910 --> 00:44:43.160 align:middle line:84% near the end of the semester talking about display 00:44:43.160 --> 00:44:46.700 align:middle line:84% technologies, because unless there 00:44:46.700 --> 00:44:51.320 align:middle line:84% is corresponding innovation, and research, and products, 00:44:51.320 --> 00:44:54.710 align:middle line:84% and services in display, it doesn't make sense to spend 00:44:54.710 --> 00:44:57.830 align:middle line:90% a lot of time on sensors. 00:44:57.830 --> 00:44:59.745 align:middle line:84% So yeah, the two have to go hand-in-hand. 00:44:59.745 --> 00:45:01.620 align:middle line:84% And there are some exciting directions there. 00:45:01.620 --> 00:45:03.710 align:middle line:84% So the last chapter of our book talks 00:45:03.710 --> 00:45:06.560 align:middle line:84% all about these other parallel developments 00:45:06.560 --> 00:45:08.660 align:middle line:84% that's going to change how we-- so for example, 00:45:08.660 --> 00:45:09.770 align:middle line:90% we have 60 displayed now. 00:45:09.770 --> 00:45:12.920 align:middle line:84% AUDIENCE: Because even in the simple 2D display, 00:45:12.920 --> 00:45:15.260 align:middle line:84% there is this project called Photos Projector. 00:45:15.260 --> 00:45:15.885 align:middle line:90% Maybe you know. 00:45:15.885 --> 00:45:21.710 align:middle line:84% That's also in this, where you can find any [INAUDIBLE] 00:45:21.710 --> 00:45:22.970 align:middle line:90% in time. 00:45:22.970 --> 00:45:24.990 align:middle line:84% So it's another way of also exploring this. 00:45:24.990 --> 00:45:25.490 align:middle line:90% It's 00:45:25.490 --> 00:45:25.922 align:middle line:90% [INTERPOSING VOICES] 00:45:25.922 --> 00:45:26.880 align:middle line:90% RAMESH RASKAR: Exactly. 00:45:26.880 --> 00:45:28.010 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:45:28.010 --> 00:45:28.968 align:middle line:90% RAMESH RASKAR: Exactly. 00:45:28.968 --> 00:45:30.230 align:middle line:90% AUDIENCE: And by taking-- 00:45:30.230 --> 00:45:33.035 align:middle line:84% giving the controls to the user, it will be used to empower him. 00:45:33.035 --> 00:45:33.910 align:middle line:90% RAMESH RASKAR: Right. 00:45:33.910 --> 00:45:36.680 align:middle line:84% AUDIENCE: Then just to one final comment 00:45:36.680 --> 00:45:37.942 align:middle line:90% representation [INAUDIBLE]. 00:45:37.942 --> 00:45:38.900 align:middle line:90% RAMESH RASKAR: Exactly. 00:45:38.900 --> 00:45:40.010 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:45:40.010 --> 00:45:40.968 align:middle line:90% RAMESH RASKAR: Exactly. 00:45:40.968 --> 00:45:44.870 align:middle line:84% So navigating through photos is a great way. 00:45:44.870 --> 00:45:46.100 align:middle line:90% But at the same time-- 00:45:46.100 --> 00:45:46.933 align:middle line:90% I love this process. 00:45:46.933 --> 00:45:48.350 align:middle line:84% At the same time, we want to think 00:45:48.350 --> 00:45:49.550 align:middle line:90% about something that scales. 00:45:49.550 --> 00:45:49.880 align:middle line:90% AUDIENCE: Yes. 00:45:49.880 --> 00:45:51.320 align:middle line:84% RAMESH RASKAR: And build it so people can use it. 00:45:51.320 --> 00:45:51.990 align:middle line:90% AUDIENCE: Yes. 00:45:51.990 --> 00:45:53.870 align:middle line:84% RAMESH RASKAR: So that's another challenge. 00:45:53.870 --> 00:45:55.700 align:middle line:90% So let's shift. 00:45:55.700 --> 00:45:57.440 align:middle line:90% I'm glad you brought it up. 00:45:57.440 --> 00:46:01.490 align:middle line:84% This is the latest fad on Flickr. 00:46:01.490 --> 00:46:06.500 align:middle line:84% I think, yeah, the HDR, it's reaching its-- 00:46:06.500 --> 00:46:09.140 align:middle line:90% it's bottoming out now. 00:46:09.140 --> 00:46:14.480 align:middle line:84% But this is tilt-shift imaging, where this looks 00:46:14.480 --> 00:46:19.610 align:middle line:90% like a tabletop toy scene. 00:46:19.610 --> 00:46:23.000 align:middle line:84% But it's actually taken from hundreds of meters away. 00:46:23.000 --> 00:46:27.290 align:middle line:84% This is in Rajasthan, the city of Jodhpur, the Blue City. 00:46:27.290 --> 00:46:32.180 align:middle line:84% And that's when you haven't used the tilt-shift class. 00:46:32.180 --> 00:46:39.250 align:middle line:84% So in the basic principles, you want to create-- 00:46:39.250 --> 00:46:42.310 align:middle line:84% instead of making-- keeping the lens, and the image plane, 00:46:42.310 --> 00:46:44.830 align:middle line:84% and the plane of focus parallel to each other, 00:46:44.830 --> 00:46:47.740 align:middle line:84% if you tilt the lens with respect to the sensor, 00:46:47.740 --> 00:46:51.170 align:middle line:84% then it turns out the plane at which things are in focus, 00:46:51.170 --> 00:46:53.650 align:middle line:90% the plane of focus, also tilts. 00:46:53.650 --> 00:46:56.110 align:middle line:84% And this is based on the shift-tilt principle. 00:46:56.110 --> 00:46:59.020 align:middle line:84% And credit this in detail when we come back to optics 00:46:59.020 --> 00:47:01.060 align:middle line:90% and light [INAUDIBLE]. 00:47:01.060 --> 00:47:06.010 align:middle line:84% But I just wanted to bring this up because it's fun. 00:47:06.010 --> 00:47:08.650 align:middle line:90% And you can play with it. 00:47:08.650 --> 00:47:09.670 align:middle line:90% All right. 00:47:09.670 --> 00:47:16.610 align:middle line:84% So let's talk about some other things. 00:47:16.610 --> 00:47:18.490 align:middle line:90% So illumination. 00:47:18.490 --> 00:47:21.970 align:middle line:90% 00:47:21.970 --> 00:47:25.430 align:middle line:84% You can use it in very interesting ways. 00:47:25.430 --> 00:47:30.670 align:middle line:84% Here's the project, I believe, from Georgia Tech, 00:47:30.670 --> 00:47:33.655 align:middle line:84% where they created an anti-paparazzi flash. 00:47:33.655 --> 00:47:34.700 align:middle line:90% [LAUGHTER] 00:47:34.700 --> 00:47:35.910 align:middle line:90% 00:47:35.910 --> 00:47:37.890 align:middle line:84% And as you can see, the way it works 00:47:37.890 --> 00:47:43.110 align:middle line:84% is the paparazzi will take a photo of a celebrity coming 00:47:43.110 --> 00:47:44.550 align:middle line:90% out. 00:47:44.550 --> 00:47:47.490 align:middle line:84% As soon as the flash goes off, it's 00:47:47.490 --> 00:47:51.600 align:middle line:84% detected by some electronic device. 00:47:51.600 --> 00:47:57.030 align:middle line:84% And it will blast more light so that what 00:47:57.030 --> 00:48:01.620 align:middle line:84% the paparazzi will get in the camera is some blown out photo. 00:48:01.620 --> 00:48:04.730 align:middle line:90% 00:48:04.730 --> 00:48:07.700 align:middle line:90% How does it work? 00:48:07.700 --> 00:48:11.180 align:middle line:90% You can-- so let's say this is-- 00:48:11.180 --> 00:48:12.470 align:middle line:90% let me make sure I understand. 00:48:12.470 --> 00:48:15.470 align:middle line:84% So it comes out a traditional camera. 00:48:15.470 --> 00:48:20.270 align:middle line:84% Especially a [INAUDIBLE] camera is actually a retroreflector. 00:48:20.270 --> 00:48:23.130 align:middle line:90% I'll explain that in a minute. 00:48:23.130 --> 00:48:30.980 align:middle line:84% So if you take a flash photo, especially a cell phone camera, 00:48:30.980 --> 00:48:37.160 align:middle line:84% then it appears as a bright spot in the flash photography. 00:48:37.160 --> 00:48:40.940 align:middle line:84% So what we're going to do is this guy here is actually-- 00:48:40.940 --> 00:48:45.380 align:middle line:84% it's going to be a camera with a ring of LED lights, 00:48:45.380 --> 00:48:48.330 align:middle line:90% which is this one here. 00:48:48.330 --> 00:48:51.740 align:middle line:84% And right next to it is going to be a projector. 00:48:51.740 --> 00:48:56.240 align:middle line:84% So this is mounted in the ceiling, a camera with LED 00:48:56.240 --> 00:48:58.940 align:middle line:90% lights and a projector. 00:48:58.940 --> 00:49:01.070 align:middle line:84% Now these LED lights are on all the time. 00:49:01.070 --> 00:49:09.890 align:middle line:84% And a paparazzi's camera will appear as a bright spot 00:49:09.890 --> 00:49:12.800 align:middle line:90% because it's a retroreflector. 00:49:12.800 --> 00:49:15.200 align:middle line:84% And this projector then, which is 00:49:15.200 --> 00:49:20.060 align:middle line:84% right below the mounted camera, will turn on the lights, 00:49:20.060 --> 00:49:26.330 align:middle line:84% turn on some pixels that are pointed towards this paparazzi. 00:49:26.330 --> 00:49:29.600 align:middle line:84% And so basically, you'll always see this very bright light, 00:49:29.600 --> 00:49:32.662 align:middle line:84% bright spotlight, being put on [? hand. ?] At the same time, 00:49:32.662 --> 00:49:34.370 align:middle line:84% the rest of the scene is going to be dark 00:49:34.370 --> 00:49:38.510 align:middle line:84% because only those pixels of the projector are being turned on. 00:49:38.510 --> 00:49:42.650 align:middle line:84% And that's why your camera, when it takes a picture, where 00:49:42.650 --> 00:49:46.610 align:middle line:84% it will just see this really bright source that's blowing 00:49:46.610 --> 00:49:49.170 align:middle line:90% out the rest of the scene. 00:49:49.170 --> 00:49:49.850 align:middle line:90% Yes? 00:49:49.850 --> 00:49:53.150 align:middle line:84% AUDIENCE: What about surveillance cameras? 00:49:53.150 --> 00:49:56.117 align:middle line:90% We don't want to bomb them? 00:49:56.117 --> 00:49:57.950 align:middle line:84% RAMESH RASKAR: So surveillance cameras are-- 00:49:57.950 --> 00:50:04.880 align:middle line:90% 00:50:04.880 --> 00:50:05.790 align:middle line:90% what's the question? 00:50:05.790 --> 00:50:06.496 align:middle line:90% AUDIENCE: I-- 00:50:06.496 --> 00:50:08.507 align:middle line:90% [LAUGHTER] 00:50:08.507 --> 00:50:12.080 align:middle line:84% Are they-- if, for example, we want surveillance cameras 00:50:12.080 --> 00:50:14.780 align:middle line:84% to just monitor that region, and then 00:50:14.780 --> 00:50:17.990 align:middle line:84% we have this device there that bombs every camera that it 00:50:17.990 --> 00:50:20.930 align:middle line:84% finds, then probably we are not capturing any information. 00:50:20.930 --> 00:50:21.605 align:middle line:90% RAMESH RASKAR: Yeah, exactly. 00:50:21.605 --> 00:50:22.105 align:middle line:90% All right. 00:50:22.105 --> 00:50:24.312 align:middle line:90% So the point is, can you-- 00:50:24.312 --> 00:50:25.145 align:middle line:90% [INTERPOSING VOICES] 00:50:25.145 --> 00:50:26.978 align:middle line:84% RAMESH RASKAR: --defeat surveillance cameras 00:50:26.978 --> 00:50:28.040 align:middle line:90% [INAUDIBLE]? 00:50:28.040 --> 00:50:31.250 align:middle line:84% Yeah, there are-- we will learn a lot of interesting ways 00:50:31.250 --> 00:50:35.150 align:middle line:84% you can beat traffic cameras or those cameras in this class. 00:50:35.150 --> 00:50:37.460 align:middle line:84% But that will all happen at the very [? chance. ?] 00:50:37.460 --> 00:50:39.384 align:middle line:90% [LAUGHTER] 00:50:39.384 --> 00:50:42.270 align:middle line:90% [INAUDIBLE] 00:50:42.270 --> 00:50:44.840 align:middle line:90% So what is there to reflect? 00:50:44.840 --> 00:50:47.450 align:middle line:90% And we'll look at those. 00:50:47.450 --> 00:50:49.505 align:middle line:84% Can you guys all see this over here? 00:50:49.505 --> 00:50:50.390 align:middle line:90% [INAUDIBLE] 00:50:50.390 --> 00:50:56.890 align:middle line:90% 00:50:56.890 --> 00:51:00.600 align:middle line:84% So if you have a traditional surface, like a wall, 00:51:00.600 --> 00:51:02.380 align:middle line:90% light comes in. 00:51:02.380 --> 00:51:05.410 align:middle line:84% And it gets reflected in our direction. 00:51:05.410 --> 00:51:08.520 align:middle line:84% And you've got a diffuse [INAUDIBLE].. 00:51:08.520 --> 00:51:11.910 align:middle line:84% If you have a mirror, then light comes in and gets 00:51:11.910 --> 00:51:18.850 align:middle line:84% reflected symmetric around the [INAUDIBLE].. 00:51:18.850 --> 00:51:23.720 align:middle line:84% But most surfaces, like this surface 00:51:23.720 --> 00:51:25.750 align:middle line:90% here, over here, light comes in. 00:51:25.750 --> 00:51:31.510 align:middle line:84% And it doesn't reflect in a uniform way. 00:51:31.510 --> 00:51:33.040 align:middle line:84% And it doesn't behave like a mirror. 00:51:33.040 --> 00:51:34.310 align:middle line:90% It's somewhere in between. 00:51:34.310 --> 00:51:36.310 align:middle line:90% So you get a bright spot. 00:51:36.310 --> 00:51:40.660 align:middle line:84% But you also get some other [INAUDIBLE] and so on. 00:51:40.660 --> 00:51:42.640 align:middle line:90% Is that clear? 00:51:42.640 --> 00:51:46.240 align:middle line:84% So diffuse, light comes in, gets tilted in all directions. 00:51:46.240 --> 00:51:48.070 align:middle line:90% Mirror goes in one direction. 00:51:48.070 --> 00:51:51.130 align:middle line:84% And a shiny surface is somewhere in between. 00:51:51.130 --> 00:51:53.860 align:middle line:90% 00:51:53.860 --> 00:51:57.950 align:middle line:84% There's other surfaces where light comes in 00:51:57.950 --> 00:52:00.220 align:middle line:84% and it reflects back in the same direction. 00:52:00.220 --> 00:52:03.100 align:middle line:90% 00:52:03.100 --> 00:52:05.950 align:middle line:84% And same surface, the light comes in and reflects 00:52:05.950 --> 00:52:09.100 align:middle line:90% back again to the same place. 00:52:09.100 --> 00:52:10.780 align:middle line:90% And this is retroreflective. 00:52:10.780 --> 00:52:15.640 align:middle line:90% 00:52:15.640 --> 00:52:19.800 align:middle line:84% And this is very useful material that you can use in many cases. 00:52:19.800 --> 00:52:22.335 align:middle line:90% Anybody know where this is used? 00:52:22.335 --> 00:52:23.910 align:middle line:90% [INTERPOSING VOICES] 00:52:23.910 --> 00:52:28.650 align:middle line:84% RAMESH RASKAR: The back of your backpack [INAUDIBLE].. 00:52:28.650 --> 00:52:30.710 align:middle line:84% That's reflector of the [INAUDIBLE].. 00:52:30.710 --> 00:52:33.530 align:middle line:90% 00:52:33.530 --> 00:52:37.247 align:middle line:84% AUDIENCE: [INAUDIBLE] system [INAUDIBLE].. 00:52:37.247 --> 00:52:38.580 align:middle line:90% RAMESH RASKAR: Oh, that's right. 00:52:38.580 --> 00:52:40.650 align:middle line:90% [INAUDIBLE], yeah. 00:52:40.650 --> 00:52:43.100 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:52:43.100 --> 00:52:46.080 align:middle line:84% RAMESH RASKAR: [INAUDIBLE] on same principle as the camera 00:52:46.080 --> 00:52:48.480 align:middle line:90% and the signal [INAUDIBLE]. 00:52:48.480 --> 00:52:52.050 align:middle line:90% So human eyes also reflect. 00:52:52.050 --> 00:52:53.770 align:middle line:90% And we'll see what happens. 00:52:53.770 --> 00:52:56.473 align:middle line:84% So this is a principle of retroreflection. 00:52:56.473 --> 00:52:58.140 align:middle line:84% How it may work, there are multiple ways 00:52:58.140 --> 00:52:59.098 align:middle line:90% you can make it happen. 00:52:59.098 --> 00:53:02.550 align:middle line:84% You can just have a so-called [INAUDIBLE] situation. 00:53:02.550 --> 00:53:06.480 align:middle line:84% If I put a [INAUDIBLE],, which is the mirror, 00:53:06.480 --> 00:53:10.300 align:middle line:84% then it comes out no matter which direction light comes in. 00:53:10.300 --> 00:53:12.730 align:middle line:84% It goes back in the same direction. 00:53:12.730 --> 00:53:14.332 align:middle line:90% It comes from here. 00:53:14.332 --> 00:53:18.140 align:middle line:84% And here, it goes back the same direction. 00:53:18.140 --> 00:53:21.290 align:middle line:84% So [INAUDIBLE] is a very useful retroreflector. 00:53:21.290 --> 00:53:25.790 align:middle line:84% In fact, if you go to many shops, especially tiny shops 00:53:25.790 --> 00:53:27.920 align:middle line:84% which put mirrors on their walls to make 00:53:27.920 --> 00:53:33.320 align:middle line:84% it look like it's much larger [INAUDIBLE] reflections, 00:53:33.320 --> 00:53:39.370 align:middle line:84% and you stand at the corner of such [INAUDIBLE] shops, 00:53:39.370 --> 00:53:42.140 align:middle line:84% you look at the corner, you will see your own mirror 00:53:42.140 --> 00:53:47.270 align:middle line:84% image, which is strange because they say this is a shop. 00:53:47.270 --> 00:53:49.010 align:middle line:90% And there's a mirror here. 00:53:49.010 --> 00:53:50.680 align:middle line:90% And you're standing here. 00:53:50.680 --> 00:53:53.240 align:middle line:84% You'll see your own reflection here in this mirror. 00:53:53.240 --> 00:53:56.670 align:middle line:84% And you'll also see your own reflection here in this mirror. 00:53:56.670 --> 00:54:01.650 align:middle line:84% But you'll also see you have one image here in the corner. 00:54:01.650 --> 00:54:06.080 align:middle line:84% And as you move around, this image will move with you. 00:54:06.080 --> 00:54:07.460 align:middle line:90% This image will move with you. 00:54:07.460 --> 00:54:11.930 align:middle line:84% But this image will always be in the corner. 00:54:11.930 --> 00:54:13.350 align:middle line:90% It's retroreflection. 00:54:13.350 --> 00:54:18.650 align:middle line:84% I look at the corner, I always see myself. 00:54:18.650 --> 00:54:21.202 align:middle line:84% So I'm surprised we don't have mirrors like this in our house 00:54:21.202 --> 00:54:23.660 align:middle line:84% so that I don't have to actually go in front of the mirror. 00:54:23.660 --> 00:54:25.160 align:middle line:90% I could be anywhere in the room. 00:54:25.160 --> 00:54:28.947 align:middle line:84% And I will still see myself no matter what I do. 00:54:28.947 --> 00:54:30.530 align:middle line:84% So you could build a mirror like that. 00:54:30.530 --> 00:54:33.850 align:middle line:90% It's just that you [INAUDIBLE]. 00:54:33.850 --> 00:54:34.350 align:middle line:90% Yes? 00:54:34.350 --> 00:54:36.570 align:middle line:84% AUDIENCE: Can you use it also for measuring [INAUDIBLE] 00:54:36.570 --> 00:54:36.990 align:middle line:90% laser? 00:54:36.990 --> 00:54:37.782 align:middle line:90% RAMESH RASKAR: Yes. 00:54:37.782 --> 00:54:38.760 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:54:38.760 --> 00:54:39.015 align:middle line:90% RAMESH RASKAR: Exactly. 00:54:39.015 --> 00:54:39.848 align:middle line:90% [INTERPOSING VOICES] 00:54:39.848 --> 00:54:42.750 align:middle line:90% RAMESH RASKAR: [INAUDIBLE] 00:54:42.750 --> 00:54:47.710 align:middle line:84% AUDIENCE: Very good angles [INAUDIBLE] for 00:54:47.710 --> 00:54:48.750 align:middle line:90% [INTERPOSING VOICES] 00:54:48.750 --> 00:54:49.375 align:middle line:90% AUDIENCE: Yeah. 00:54:49.375 --> 00:54:51.600 align:middle line:84% RAMESH RASKAR: Yeah, so that's one way of doing it. 00:54:51.600 --> 00:54:51.980 align:middle line:90% Yes? 00:54:51.980 --> 00:54:54.022 align:middle line:84% AUDIENCE: Is [INAUDIBLE] reduction also required? 00:54:54.022 --> 00:54:55.110 align:middle line:90% Because rhetorically-- 00:54:55.110 --> 00:54:55.770 align:middle line:90% RAMESH RASKAR: Very good point. 00:54:55.770 --> 00:54:56.437 align:middle line:90% Very good point. 00:54:56.437 --> 00:54:59.760 align:middle line:84% So whether reduction is like [INAUDIBLE],, you see-- 00:54:59.760 --> 00:55:06.890 align:middle line:84% and the color red is because the chemicals are [INAUDIBLE] 00:55:06.890 --> 00:55:08.668 align:middle line:84% AUDIENCE: [INAUDIBLE] also, [INAUDIBLE].. 00:55:08.668 --> 00:55:09.460 align:middle line:90% RAMESH RASKAR: Huh? 00:55:09.460 --> 00:55:11.422 align:middle line:84% AUDIENCE: That could [INAUDIBLE].. 00:55:11.422 --> 00:55:12.380 align:middle line:90% RAMESH RASKAR: Exactly. 00:55:12.380 --> 00:55:14.590 align:middle line:84% Now both of those are another way of doing it. 00:55:14.590 --> 00:55:19.150 align:middle line:84% So you-- so once [INAUDIBLE] the other way is-- 00:55:19.150 --> 00:55:21.160 align:middle line:84% [INAUDIBLE] property of this one is 00:55:21.160 --> 00:55:23.777 align:middle line:84% you know how people will say, in the mirror, you get confused. 00:55:23.777 --> 00:55:25.860 align:middle line:84% If you move your left hand, your right hand moves. 00:55:25.860 --> 00:55:28.210 align:middle line:84% And you move the right hand, the right hand moves. 00:55:28.210 --> 00:55:30.020 align:middle line:84% The [INAUDIBLE],, that doesn't happen. 00:55:30.020 --> 00:55:33.540 align:middle line:84% You can move this hand [INAUDIBLE].. 00:55:33.540 --> 00:55:40.210 align:middle line:84% You can move this hand so your image, 00:55:40.210 --> 00:55:41.815 align:middle line:90% it's not clear [INAUDIBLE]. 00:55:41.815 --> 00:55:46.205 align:middle line:90% 00:55:46.205 --> 00:55:48.330 align:middle line:84% Is there a shop around here with mirrors like this? 00:55:48.330 --> 00:55:51.220 align:middle line:90% 00:55:51.220 --> 00:55:52.505 align:middle line:90% You use the [INAUDIBLE]. 00:55:52.505 --> 00:55:55.820 align:middle line:90% 00:55:55.820 --> 00:55:56.320 align:middle line:90% All right. 00:55:56.320 --> 00:55:58.400 align:middle line:90% So other one is false. 00:55:58.400 --> 00:56:02.260 align:middle line:90% If I just take a shiny-- 00:56:02.260 --> 00:56:04.660 align:middle line:84% this is the last [INAUDIBLE],, actually. 00:56:04.660 --> 00:56:06.600 align:middle line:90% Night comes in. 00:56:06.600 --> 00:56:12.940 align:middle line:84% It reflects because of this refraction. 00:56:12.940 --> 00:56:14.900 align:middle line:84% And then, it reflects again because, of course, 00:56:14.900 --> 00:56:16.025 align:middle line:90% we need another reflection. 00:56:16.025 --> 00:56:20.185 align:middle line:84% There's a refractive index here [INAUDIBLE].. 00:56:20.185 --> 00:56:22.670 align:middle line:90% And then, it goes back out. 00:56:22.670 --> 00:56:26.430 align:middle line:90% And then [INAUDIBLE]. 00:56:26.430 --> 00:56:31.720 align:middle line:84% So that's how a glass bead-based retroreflector would work. 00:56:31.720 --> 00:56:34.390 align:middle line:84% And then, of course, you'll have a bunch of these. 00:56:34.390 --> 00:56:37.020 align:middle line:90% So you can [INAUDIBLE]. 00:56:37.020 --> 00:56:37.520 align:middle line:90% All right? 00:56:37.520 --> 00:56:40.733 align:middle line:84% And this is exactly how a rainbow works. 00:56:40.733 --> 00:56:45.660 align:middle line:84% So you see a rainbow after you've had this rain. 00:56:45.660 --> 00:56:47.940 align:middle line:90% This is how a rainbow works. 00:56:47.940 --> 00:56:53.232 align:middle line:84% If you ever see a rainbow with maximum-- 00:56:53.232 --> 00:56:57.060 align:middle line:90% 00:56:57.060 --> 00:57:01.470 align:middle line:84% most vivid colors, what conditions you are satisfying? 00:57:01.470 --> 00:57:03.030 align:middle line:90% This is for the sun. 00:57:03.030 --> 00:57:11.260 align:middle line:84% And this is for the high-humidity here. 00:57:11.260 --> 00:57:14.688 align:middle line:84% What's the [INAUDIBLE] condition [INAUDIBLE]?? 00:57:14.688 --> 00:57:16.450 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:57:16.450 --> 00:57:17.533 align:middle line:90% RAMESH RASKAR: [INAUDIBLE] 00:57:17.533 --> 00:57:18.810 align:middle line:90% AUDIENCE: [INAUDIBLE], yeah. 00:57:18.810 --> 00:57:21.550 align:middle line:84% RAMESH RASKAR: So this is the sky. 00:57:21.550 --> 00:57:23.970 align:middle line:90% The sun is just setting. 00:57:23.970 --> 00:57:24.600 align:middle line:90% You are here. 00:57:24.600 --> 00:57:27.390 align:middle line:90% 00:57:27.390 --> 00:57:29.098 align:middle line:84% And you're going to see it exactly 00:57:29.098 --> 00:57:30.390 align:middle line:90% in the opposite direction here. 00:57:30.390 --> 00:57:32.520 align:middle line:90% So the rainbow is here. 00:57:32.520 --> 00:57:33.840 align:middle line:90% So very common. 00:57:33.840 --> 00:57:37.050 align:middle line:84% You would go to some place with a waterfall. 00:57:37.050 --> 00:57:38.638 align:middle line:90% And you want to see the rainbow. 00:57:38.638 --> 00:57:40.680 align:middle line:84% In case of a challenge, you can see it sometimes. 00:57:40.680 --> 00:57:41.850 align:middle line:90% You don't see it sometimes. 00:57:41.850 --> 00:57:45.225 align:middle line:84% All you have to make sure is that the sun-- so usually, it's 00:57:45.225 --> 00:57:46.730 align:middle line:90% high enough. 00:57:46.730 --> 00:57:47.730 align:middle line:90% You can see the rainbow. 00:57:47.730 --> 00:57:52.950 align:middle line:84% Because you can imagine if the sun is pretty high in the sky, 00:57:52.950 --> 00:57:55.410 align:middle line:84% you'll never see a rainbow because it's occluded 00:57:55.410 --> 00:57:57.690 align:middle line:90% by the surface of the Earth. 00:57:57.690 --> 00:58:01.540 align:middle line:84% But when the sun is low enough, you will see a rainbow. 00:58:01.540 --> 00:58:04.650 align:middle line:84% On the other hand, you just have a mountain, 00:58:04.650 --> 00:58:06.690 align:middle line:84% then even in the daytime, you could see 00:58:06.690 --> 00:58:10.770 align:middle line:90% the rainbow if the [INAUDIBLE]. 00:58:10.770 --> 00:58:13.500 align:middle line:84% The lowest point by this are allowing 00:58:13.500 --> 00:58:16.110 align:middle line:90% you to see the retroreflection. 00:58:16.110 --> 00:58:18.000 align:middle line:84% And we're coming to, why this rainbow? 00:58:18.000 --> 00:58:21.845 align:middle line:84% Because the refractive index for different times [INAUDIBLE].. 00:58:21.845 --> 00:58:24.185 align:middle line:84% So that's why we see [INAUDIBLE].. 00:58:24.185 --> 00:58:27.110 align:middle line:90% 00:58:27.110 --> 00:58:30.840 align:middle line:84% So it's all fascinating [INAUDIBLE],, retroreflection. 00:58:30.840 --> 00:58:37.970 align:middle line:84% So in case of, why does the human eye have red eye or a cat 00:58:37.970 --> 00:58:42.435 align:middle line:90% has these bright spots? 00:58:42.435 --> 00:58:43.310 align:middle line:90% It's the same reason. 00:58:43.310 --> 00:58:46.668 align:middle line:90% 00:58:46.668 --> 00:58:47.960 align:middle line:90% So all you want is [INAUDIBLE]. 00:58:47.960 --> 00:58:51.080 align:middle line:84% Light goes in, comes back in the same direction. 00:58:51.080 --> 00:58:58.310 align:middle line:84% So if you have now a [INAUDIBLE].. 00:58:58.310 --> 00:59:01.630 align:middle line:90% 00:59:01.630 --> 00:59:06.640 align:middle line:84% So now, if you have, let's say, a camera for now. 00:59:06.640 --> 00:59:09.440 align:middle line:84% And human eye was almost similar [INAUDIBLE].. 00:59:09.440 --> 00:59:12.260 align:middle line:84% And there's a bright spot, bright light. 00:59:12.260 --> 00:59:14.590 align:middle line:84% I'm going to stand right next to it. 00:59:14.590 --> 00:59:16.660 align:middle line:90% Light goes in. 00:59:16.660 --> 00:59:18.430 align:middle line:90% And this gets focused. 00:59:18.430 --> 00:59:19.350 align:middle line:90% That's the point. 00:59:19.350 --> 00:59:23.500 align:middle line:90% 00:59:23.500 --> 00:59:27.040 align:middle line:84% And as you can imagine, if you're a sensor, 00:59:27.040 --> 00:59:29.435 align:middle line:84% it actually has a shiny [INAUDIBLE] in front of it. 00:59:29.435 --> 00:59:31.060 align:middle line:84% If you look at a sensor, you'll realize 00:59:31.060 --> 00:59:35.300 align:middle line:84% it's pretty shiny because [INAUDIBLE].. 00:59:35.300 --> 00:59:37.870 align:middle line:90% Light goes in and reflects. 00:59:37.870 --> 00:59:40.585 align:middle line:84% And no matter where light comes out from, because 00:59:40.585 --> 00:59:42.180 align:middle line:84% of the geometry of the light, you 00:59:42.180 --> 00:59:43.510 align:middle line:90% can go back to the same spot. 00:59:43.510 --> 00:59:46.770 align:middle line:90% 00:59:46.770 --> 00:59:49.410 align:middle line:84% Now [INAUDIBLE] for the focus, you do not 00:59:49.410 --> 00:59:52.170 align:middle line:90% see it because [INAUDIBLE]. 00:59:52.170 --> 00:59:54.990 align:middle line:84% But if the point was in sharp focus, on the sensor, 00:59:54.990 --> 00:59:56.220 align:middle line:90% it will reflect back. 00:59:56.220 --> 00:59:59.080 align:middle line:84% And it will come back in the same place. 00:59:59.080 --> 01:00:01.140 align:middle line:84% And so this is the same reason why 01:00:01.140 --> 01:00:05.180 align:middle line:84% for a human eye, again, in a situation like [INAUDIBLE],, 01:00:05.180 --> 01:00:09.294 align:middle line:84% it reflects and comes back in the same direction. 01:00:09.294 --> 01:00:10.270 align:middle line:90% [INAUDIBLE] 01:00:10.270 --> 01:00:13.020 align:middle line:84% So the reason why I tell you, what's 01:00:13.020 --> 01:00:16.470 align:middle line:84% the new strategy for removing a red light? 01:00:16.470 --> 01:00:18.020 align:middle line:90% What should you do? 01:00:18.020 --> 01:00:19.860 align:middle line:84% AUDIENCE: Flash phase or [INAUDIBLE].. 01:00:19.860 --> 01:00:20.902 align:middle line:90% RAMESH RASKAR: Flash out. 01:00:20.902 --> 01:00:22.908 align:middle line:84% Or if you have a cheap camera, where you-- 01:00:22.908 --> 01:00:23.784 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:00:23.784 --> 01:00:25.890 align:middle line:84% RAMESH RASKAR: [INAUDIBLE] camera. 01:00:25.890 --> 01:00:27.750 align:middle line:84% Because you're not looking at the camera, 01:00:27.750 --> 01:00:29.610 align:middle line:84% you have-- the image that we formed 01:00:29.610 --> 01:00:33.930 align:middle line:84% is forming something that's made of the light [INAUDIBLE].. 01:00:33.930 --> 01:00:36.940 align:middle line:84% And so you're not referring back to the camera. 01:00:36.940 --> 01:00:39.300 align:middle line:84% AUDIENCE: So the anti-paparazzi system will also 01:00:39.300 --> 01:00:41.412 align:middle line:90% blind everybody [INAUDIBLE]. 01:00:41.412 --> 01:00:44.020 align:middle line:90% RAMESH RASKAR: If it was-- 01:00:44.020 --> 01:00:45.020 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:00:45.020 --> 01:00:45.978 align:middle line:90% RAMESH RASKAR: If the-- 01:00:45.978 --> 01:00:47.520 align:middle line:90% [LAUGHTER] 01:00:47.520 --> 01:00:49.250 align:middle line:90% 01:00:49.250 --> 01:00:53.970 align:middle line:84% If the city-mounted light was omnidirectional, 01:00:53.970 --> 01:00:55.140 align:middle line:90% then that would matter. 01:00:55.140 --> 01:00:57.570 align:middle line:84% But because the projector had constrained 01:00:57.570 --> 01:01:01.690 align:middle line:84% light only at [INAUDIBLE] points [INAUDIBLE] this point. 01:01:01.690 --> 01:01:03.190 align:middle line:90% [INTERPOSING VOICES] 01:01:03.190 --> 01:01:04.638 align:middle line:90% 01:01:04.638 --> 01:01:05.430 align:middle line:90% RAMESH RASKAR: Yes. 01:01:05.430 --> 01:01:06.305 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:01:06.305 --> 01:01:09.120 align:middle line:90% RAMESH RASKAR: Or this camera. 01:01:09.120 --> 01:01:10.713 align:middle line:90% We don't want to [INAUDIBLE]. 01:01:10.713 --> 01:01:12.630 align:middle line:84% AUDIENCE: I think the point JD was bringing up 01:01:12.630 --> 01:01:14.463 align:middle line:84% is that human eyes are also retroreflective. 01:01:14.463 --> 01:01:17.580 align:middle line:84% So if you look at a spot, it will potentially flash you. 01:01:17.580 --> 01:01:18.210 align:middle line:84% RAMESH RASKAR: Is that what you're saying? 01:01:18.210 --> 01:01:18.600 align:middle line:90% AUDIENCE: Yes. 01:01:18.600 --> 01:01:19.410 align:middle line:90% RAMESH RASKAR: Yeah, so-- 01:01:19.410 --> 01:01:20.370 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:01:20.370 --> 01:01:20.940 align:middle line:90% RAMESH RASKAR: It's true. 01:01:20.940 --> 01:01:21.660 align:middle line:90% It's true. 01:01:21.660 --> 01:01:22.940 align:middle line:90% So this is among the rules. 01:01:22.940 --> 01:01:25.000 align:middle line:90% [LAUGHTER] 01:01:25.000 --> 01:01:27.615 align:middle line:84% So just don't go there with [INAUDIBLE].. 01:01:27.615 --> 01:01:30.720 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:01:30.720 --> 01:01:34.620 align:middle line:84% RAMESH RASKAR: So also, you know that [INAUDIBLE] are 01:01:34.620 --> 01:01:36.595 align:middle line:90% a very technical environment. 01:01:36.595 --> 01:01:38.220 align:middle line:84% Usually, [INAUDIBLE] have more problems 01:01:38.220 --> 01:01:42.480 align:middle line:84% with [INAUDIBLE] don't have a problem with [INAUDIBLE].. 01:01:42.480 --> 01:01:46.150 align:middle line:84% And that's just because of the pigment and so on. 01:01:46.150 --> 01:01:49.588 align:middle line:84% But you're right, that If the system works really well, 01:01:49.588 --> 01:01:51.255 align:middle line:84% it will also blind all the [INAUDIBLE].. 01:01:51.255 --> 01:01:51.930 align:middle line:90% [LAUGHTER] 01:01:51.930 --> 01:01:54.180 align:middle line:84% AUDIENCE: So if you have the masks, something 01:01:54.180 --> 01:01:58.560 align:middle line:84% to put into your lens, then this will seem, obviously, 01:01:58.560 --> 01:02:00.750 align:middle line:90% [INAUDIBLE]. 01:02:00.750 --> 01:02:03.420 align:middle line:84% RAMESH RASKAR: Oh, yeah, that's the next [INAUDIBLE],, 01:02:03.420 --> 01:02:05.458 align:middle line:90% how to beat [INAUDIBLE]. 01:02:05.458 --> 01:02:06.000 align:middle line:90% AUDIENCE: And 01:02:06.000 --> 01:02:06.360 align:middle line:90% [INTERPOSING VOICES] 01:02:06.360 --> 01:02:07.693 align:middle line:90% AUDIENCE: You can use the flash. 01:02:07.693 --> 01:02:08.566 align:middle line:90% [LAUGHTER] 01:02:08.566 --> 01:02:12.020 align:middle line:84% Use a camera with a high-ISO [INAUDIBLE].. 01:02:12.020 --> 01:02:13.770 align:middle line:84% RAMESH RASKAR: Yeah, there's a lot of ways 01:02:13.770 --> 01:02:16.410 align:middle line:90% to get around it, always. 01:02:16.410 --> 01:02:19.410 align:middle line:84% If you want to have a very nice discussion on how 01:02:19.410 --> 01:02:22.488 align:middle line:84% to change your license plate so that the-- 01:02:22.488 --> 01:02:24.720 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:02:24.720 --> 01:02:27.145 align:middle line:84% RAMESH RASKAR: --high-speed-- whatever, the-- 01:02:27.145 --> 01:02:28.020 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:02:28.020 --> 01:02:29.395 align:middle line:84% RAMESH RASKAR: --speeding cameras 01:02:29.395 --> 01:02:31.170 align:middle line:90% will not be able to see it. 01:02:31.170 --> 01:02:32.533 align:middle line:90% You ought to buy me a beer. 01:02:32.533 --> 01:02:33.460 align:middle line:90% [LAUGHTER] 01:02:33.460 --> 01:02:34.780 align:middle line:90% 01:02:34.780 --> 01:02:35.280 align:middle line:90% All right? 01:02:35.280 --> 01:02:35.670 align:middle line:90% AUDIENCE: Yeah. 01:02:35.670 --> 01:02:36.300 align:middle line:90% RAMESH RASKAR: You're up for it? 01:02:36.300 --> 01:02:36.925 align:middle line:90% AUDIENCE: Yeah. 01:02:36.925 --> 01:02:38.410 align:middle line:90% RAMESH RASKAR: You're over 18? 01:02:38.410 --> 01:02:38.910 align:middle line:90% [LAUGHTER] 01:02:38.910 --> 01:02:40.650 align:middle line:90% AUDIENCE: Yeah, 21 [INAUDIBLE]. 01:02:40.650 --> 01:02:42.840 align:middle line:84% RAMESH RASKAR: I did not say age. 01:02:42.840 --> 01:02:43.640 align:middle line:90% All right. 01:02:43.640 --> 01:02:44.140 align:middle line:90% Good. 01:02:44.140 --> 01:02:48.120 align:middle line:84% So before we take a short break and go 01:02:48.120 --> 01:02:50.130 align:middle line:84% into the illumination, more discussion, 01:02:50.130 --> 01:02:52.860 align:middle line:84% I just want to give you the final projects 01:02:52.860 --> 01:02:53.950 align:middle line:90% you could be working on. 01:02:53.950 --> 01:02:57.330 align:middle line:84% Remember, we are really here to do something that's cool 01:02:57.330 --> 01:02:59.790 align:middle line:90% as well as novel. 01:02:59.790 --> 01:03:01.470 align:middle line:84% So many times, I'm going to bring up 01:03:01.470 --> 01:03:04.844 align:middle line:84% a list of the projects that are not cool. 01:03:04.844 --> 01:03:05.610 align:middle line:90% All right? 01:03:05.610 --> 01:03:07.830 align:middle line:84% And again, no offense to somebody who 01:03:07.830 --> 01:03:09.660 align:middle line:84% might be working in a similar field. 01:03:09.660 --> 01:03:11.490 align:middle line:90% But just, let's be honest. 01:03:11.490 --> 01:03:13.120 align:middle line:90% Let's do something nice. 01:03:13.120 --> 01:03:17.280 align:middle line:84% So here are some suggestions for types of fun projects 01:03:17.280 --> 01:03:19.890 align:middle line:90% you could be doing. 01:03:19.890 --> 01:03:23.220 align:middle line:84% You have beautiful user interaction devices 01:03:23.220 --> 01:03:24.830 align:middle line:90% that are always fun. 01:03:24.830 --> 01:03:27.660 align:middle line:84% You start using a 2D sensor camera. 01:03:27.660 --> 01:03:31.800 align:middle line:84% Maybe use the [INAUDIBLE] camera for the finished camera, 01:03:31.800 --> 01:03:34.440 align:middle line:84% which can be, by the way, created from a flatbed scanner. 01:03:34.440 --> 01:03:37.560 align:middle line:84% You can buy a flatbed scanner for under $100. 01:03:37.560 --> 01:03:40.830 align:middle line:84% And it's actually a multithousand frame rate 01:03:40.830 --> 01:03:42.630 align:middle line:90% camera. 01:03:42.630 --> 01:03:44.560 align:middle line:84% You can just hold it in one place. 01:03:44.560 --> 01:03:46.470 align:middle line:84% And if you walk in front of it, you'll 01:03:46.470 --> 01:03:48.540 align:middle line:84% basically create a photo finished camera. 01:03:48.540 --> 01:03:50.582 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:03:50.582 --> 01:03:52.540 align:middle line:84% RAMESH RASKAR: Yeah, there are a lot of issues. 01:03:52.540 --> 01:03:54.130 align:middle line:90% But we'll help you on that. 01:03:54.130 --> 01:03:56.830 align:middle line:90% We'll help you on that. 01:03:56.830 --> 01:03:59.050 align:middle line:84% Yeah, like Dino was saying, it's not 01:03:59.050 --> 01:04:01.760 align:middle line:84% something you can just buy and just plug it in. 01:04:01.760 --> 01:04:06.430 align:middle line:84% You have to get your hands dirty in [INAUDIBLE].. 01:04:06.430 --> 01:04:09.100 align:middle line:84% Or just use photodetectors, single-pixel cameras. 01:04:09.100 --> 01:04:11.740 align:middle line:84% You can do a lot of interesting things with it. 01:04:11.740 --> 01:04:14.560 align:middle line:84% You can include some interesting illumination. 01:04:14.560 --> 01:04:15.670 align:middle line:90% Capture the invisible. 01:04:15.670 --> 01:04:18.310 align:middle line:84% It's always my fascination on, how can we capture something 01:04:18.310 --> 01:04:20.930 align:middle line:84% that cannot be seen with the naked eye? 01:04:20.930 --> 01:04:22.870 align:middle line:84% So maybe bring some tomography machine 01:04:22.870 --> 01:04:26.320 align:middle line:84% that can see inside the body, some structure like 3D 01:04:26.320 --> 01:04:28.450 align:middle line:90% scanning or fluorescence. 01:04:28.450 --> 01:04:32.090 align:middle line:84% And we'll see that experiment here just in a minute. 01:04:32.090 --> 01:04:34.120 align:middle line:84% And this is a question you were asking earlier. 01:04:34.120 --> 01:04:35.950 align:middle line:84% Are there cameras in other domains, 01:04:35.950 --> 01:04:38.500 align:middle line:84% electromagnetic, or audio, or resistant. 01:04:38.500 --> 01:04:42.250 align:middle line:84% So how about an audio camera, or a magnetic camera, 01:04:42.250 --> 01:04:43.480 align:middle line:90% or a capacitive camera? 01:04:43.480 --> 01:04:46.450 align:middle line:84% So we won't be discussing it a lot in the class. 01:04:46.450 --> 01:04:50.720 align:middle line:84% But I think it's a great way to do a final project. 01:04:50.720 --> 01:04:52.660 align:middle line:84% Thermal lab camera, we can help you. 01:04:52.660 --> 01:04:55.840 align:middle line:84% Maybe a thermal lab camera that detects emotions. 01:04:55.840 --> 01:04:58.360 align:middle line:84% Multispectral camera, cameras that 01:04:58.360 --> 01:05:02.230 align:middle line:84% can distinguish between two very similar colored objects. 01:05:02.230 --> 01:05:05.410 align:middle line:84% There's a lot of us interested in distinguishing camel 01:05:05.410 --> 01:05:06.670 align:middle line:90% from sand. 01:05:06.670 --> 01:05:10.270 align:middle line:84% So can you create some mechanisms 01:05:10.270 --> 01:05:13.820 align:middle line:84% so that two very similar looking objects can 01:05:13.820 --> 01:05:16.610 align:middle line:90% be completely distinguished? 01:05:16.610 --> 01:05:18.800 align:middle line:84% There's a lot of market in the golf business, where 01:05:18.800 --> 01:05:25.730 align:middle line:84% people want to spot their golf ball on a green background. 01:05:25.730 --> 01:05:27.020 align:middle line:90% So you just put this-- 01:05:27.020 --> 01:05:29.408 align:middle line:90% you know what to do? 01:05:29.408 --> 01:05:31.200 align:middle line:84% Yeah, they sell it for hundreds of dollars. 01:05:31.200 --> 01:05:32.765 align:middle line:90% All you need is a blue filter. 01:05:32.765 --> 01:05:35.615 align:middle line:84% If you put a blue filter on the grass, which is green, 01:05:35.615 --> 01:05:37.550 align:middle line:90% it looks black. 01:05:37.550 --> 01:05:42.798 align:middle line:84% But your ball, which is not green, stands up. 01:05:42.798 --> 01:05:44.090 align:middle line:90% So you have a black background. 01:05:44.090 --> 01:05:48.860 align:middle line:84% And anything that's not green will really stand out. 01:05:48.860 --> 01:05:53.660 align:middle line:84% You can-- there are lots of other businesses [INAUDIBLE].. 01:05:53.660 --> 01:05:55.025 align:middle line:90% Illumination. 01:05:55.025 --> 01:05:58.010 align:middle line:84% [INAUDIBLE] photography, I think we saw earlier. 01:05:58.010 --> 01:05:58.930 align:middle line:90% A lot of fun. 01:05:58.930 --> 01:05:59.930 align:middle line:90% All, kinds of, strobing. 01:05:59.930 --> 01:06:03.350 align:middle line:90% 01:06:03.350 --> 01:06:09.410 align:middle line:84% Nonimaging elements such as gyros, GPS, interaction 01:06:09.410 --> 01:06:10.330 align:middle line:90% between two cameras. 01:06:10.330 --> 01:06:12.080 align:middle line:84% Maybe there is some lighting communication 01:06:12.080 --> 01:06:14.750 align:middle line:84% or Wi-Fi communication between them. 01:06:14.750 --> 01:06:18.650 align:middle line:84% Optics is always a lot of fun, camera areas, light fields, 01:06:18.650 --> 01:06:20.300 align:middle line:90% portrait aperture. 01:06:20.300 --> 01:06:25.460 align:middle line:84% Maybe try to mimic a vision of one of the animals. 01:06:25.460 --> 01:06:26.630 align:middle line:90% Or study. 01:06:26.630 --> 01:06:28.760 align:middle line:84% Bring some worms or bring some cats. 01:06:28.760 --> 01:06:32.180 align:middle line:84% And cats are, by the way, really beautiful mechanisms 01:06:32.180 --> 01:06:34.460 align:middle line:90% of how they work. 01:06:34.460 --> 01:06:36.830 align:middle line:84% And we'll have a whole lecture on animal eyes. 01:06:36.830 --> 01:06:39.330 align:middle line:84% I think seventh or eighth lecture. 01:06:39.330 --> 01:06:41.540 align:middle line:90% So we'll study that. 01:06:41.540 --> 01:06:42.350 align:middle line:90% And time. 01:06:42.350 --> 01:06:44.990 align:middle line:84% Timelapse photos, there's so much 01:06:44.990 --> 01:06:46.820 align:middle line:84% information in time lapse photos. 01:06:46.820 --> 01:06:50.238 align:middle line:84% I wish I could go on Google Earth or Google Street View 01:06:50.238 --> 01:06:51.530 align:middle line:90% and it will show me time lapse. 01:06:51.530 --> 01:06:53.270 align:middle line:84% It can't be that difficult. They'd 01:06:53.270 --> 01:06:55.040 align:middle line:84% have to scale their database only 01:06:55.040 --> 01:06:59.247 align:middle line:84% by a factor of 24, which is not that much more, I think. 01:06:59.247 --> 01:07:00.830 align:middle line:84% But it'd be nice to see the same place 01:07:00.830 --> 01:07:03.050 align:middle line:84% in the daytime and nighttime, and what, 01:07:03.050 --> 01:07:06.110 align:middle line:84% kind of, traffic it has, and what, kind of, people it has. 01:07:06.110 --> 01:07:10.340 align:middle line:84% Right now, it's very static about its appearance. 01:07:10.340 --> 01:07:13.160 align:middle line:84% Somebody had a very nice idea for time lapse photos 01:07:13.160 --> 01:07:16.640 align:middle line:84% for Google Street Map, which is-- 01:07:16.640 --> 01:07:19.400 align:middle line:84% sometimes, you don't need a video of what's going on there. 01:07:19.400 --> 01:07:22.910 align:middle line:84% But the world, at least man-made world, 01:07:22.910 --> 01:07:25.370 align:middle line:84% usually, it will only have a few discrete steps. 01:07:25.370 --> 01:07:29.120 align:middle line:84% If there's a traffic light, it just 01:07:29.120 --> 01:07:31.160 align:middle line:84% switches between three different colors. 01:07:31.160 --> 01:07:35.420 align:middle line:84% If there's a bridge on the river, that drawbridge, 01:07:35.420 --> 01:07:37.700 align:middle line:84% it's either this position or this position. 01:07:37.700 --> 01:07:40.040 align:middle line:84% There are a lot of these discrete things 01:07:40.040 --> 01:07:41.670 align:middle line:90% that happen in man-made world. 01:07:41.670 --> 01:07:43.430 align:middle line:84% So it'd be nice if in Google Street, 01:07:43.430 --> 01:07:47.570 align:middle line:84% you can see all those discrete steps, discrete situations 01:07:47.570 --> 01:07:48.860 align:middle line:90% in [INAUDIBLE]. 01:07:48.860 --> 01:07:52.220 align:middle line:90% 01:07:52.220 --> 01:07:55.640 align:middle line:84% You can create this direct global separation. 01:07:55.640 --> 01:07:57.380 align:middle line:90% I think we saw that earlier. 01:07:57.380 --> 01:08:00.350 align:middle line:84% Create new types of cameras that are over 10-- 01:08:00.350 --> 01:08:03.410 align:middle line:84% these are all suggestions for final products. 01:08:03.410 --> 01:08:05.660 align:middle line:90% Mimic animal eyes. 01:08:05.660 --> 01:08:06.830 align:middle line:90% Play with photonic crystals. 01:08:06.830 --> 01:08:09.200 align:middle line:84% They're are very easy to make now. 01:08:09.200 --> 01:08:12.740 align:middle line:84% Photonic crystals, basically, in simple words, 01:08:12.740 --> 01:08:21.950 align:middle line:84% these are ordered arrays of materials that have 01:08:21.950 --> 01:08:23.870 align:middle line:90% different refractive indexes. 01:08:23.870 --> 01:08:27.800 align:middle line:84% In the simplest case, it could be just glass and air. 01:08:27.800 --> 01:08:31.490 align:middle line:84% So if you could just create micron-scaled holes in glass, 01:08:31.490 --> 01:08:33.300 align:middle line:84% it behaves in very interesting ways. 01:08:33.300 --> 01:08:37.310 align:middle line:84% So maybe somebody will play with that [INAUDIBLE] photography 01:08:37.310 --> 01:08:38.043 align:middle line:90% and [INAUDIBLE]. 01:08:38.043 --> 01:08:38.297 align:middle line:90% AUDIENCE: Ramesh? 01:08:38.297 --> 01:08:38.810 align:middle line:90% RAMESH RASKAR: Yeah? 01:08:38.810 --> 01:08:40.268 align:middle line:84% AUDIENCE: You say it's very simple. 01:08:40.268 --> 01:08:42.163 align:middle line:90% How do you do that? 01:08:42.163 --> 01:08:43.580 align:middle line:84% RAMESH RASKAR: By simple, I mean-- 01:08:43.580 --> 01:08:43.819 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:08:43.819 --> 01:08:46.130 align:middle line:84% RAMESH RASKAR: Yeah, the idea is fairly simple. 01:08:46.130 --> 01:08:49.790 align:middle line:84% Actually, when photonic crystals were invented in Bell Labs, 01:08:49.790 --> 01:08:53.630 align:middle line:84% they literally took CNC C machines in metal. 01:08:53.630 --> 01:08:55.052 align:middle line:90% And they just punched holes. 01:08:55.052 --> 01:08:57.260 align:middle line:84% AUDIENCE: You can get access to that or [INAUDIBLE]?? 01:08:57.260 --> 01:08:59.630 align:middle line:84% RAMESH RASKAR: We have a microelectronics laboratory. 01:08:59.630 --> 01:09:03.560 align:middle line:90% I think Quinn knows about it. 01:09:03.560 --> 01:09:05.540 align:middle line:84% And we can-- I think there's some charge. 01:09:05.540 --> 01:09:10.130 align:middle line:84% But I can easily take care of the funding for that. 01:09:10.130 --> 01:09:14.643 align:middle line:84% And I think you pay a fixed one-hour rental for it. 01:09:14.643 --> 01:09:16.310 align:middle line:84% But if it's a research project, maybe we 01:09:16.310 --> 01:09:17.819 align:middle line:90% can even get around that. 01:09:17.819 --> 01:09:25.010 align:middle line:84% So in fact, you can buy now nanomaterials and photonic 01:09:25.010 --> 01:09:26.960 align:middle line:90% crystals just online. 01:09:26.960 --> 01:09:30.680 align:middle line:84% You can get two-micron beads and six-micron beads-- 01:09:30.680 --> 01:09:35.630 align:middle line:84% sorry, two-nanometer beads and six-nanometer beads online. 01:09:35.630 --> 01:09:36.410 align:middle line:90% Very toxic. 01:09:36.410 --> 01:09:38.330 align:middle line:90% A little bit dangerous. 01:09:38.330 --> 01:09:43.064 align:middle line:84% But there are certain varieties that you can just use. 01:09:43.064 --> 01:09:44.689 align:middle line:84% So it's a lot of fun to play with that. 01:09:44.689 --> 01:09:47.165 align:middle line:90% 01:09:47.165 --> 01:09:48.540 align:middle line:84% And here are some sample projects 01:09:48.540 --> 01:09:50.460 align:middle line:90% from last year's class. 01:09:50.460 --> 01:09:52.080 align:middle line:84% In our [INAUDIBLE] photography, again, 01:09:52.080 --> 01:09:54.270 align:middle line:84% this was the only undergrad in the class. 01:09:54.270 --> 01:09:58.170 align:middle line:84% And he won the best award, Best Project Award, from Nokia. 01:09:58.170 --> 01:10:00.930 align:middle line:90% 01:10:00.930 --> 01:10:02.550 align:middle line:84% Jessie from mechanical engineering 01:10:02.550 --> 01:10:05.820 align:middle line:84% did a camera array for particle image velocimetry. 01:10:05.820 --> 01:10:08.820 align:middle line:84% And now, it has become his PhD thesis. 01:10:08.820 --> 01:10:12.060 align:middle line:84% And he's applying for hundreds of thousands 01:10:12.060 --> 01:10:13.860 align:middle line:90% of dollars in grants. 01:10:13.860 --> 01:10:17.160 align:middle line:84% Bidirectional screen was a project by Matt, Matt Hirsch. 01:10:17.160 --> 01:10:19.410 align:middle line:84% Again, it has become his master's thesis, 01:10:19.410 --> 01:10:22.971 align:middle line:90% SIGGRAPH paper, and so on. 01:10:22.971 --> 01:10:24.508 align:middle line:84% [INAUDIBLE] Kermani, in my group, 01:10:24.508 --> 01:10:26.550 align:middle line:84% he just looked at the theory, he didn't have time 01:10:26.550 --> 01:10:28.500 align:middle line:84% to build anything at that time, but looking 01:10:28.500 --> 01:10:31.890 align:middle line:84% around a corner, which is, again, a paper this year. 01:10:31.890 --> 01:10:34.300 align:middle line:84% Somebody is building a tomography machine and so on. 01:10:34.300 --> 01:10:35.850 align:middle line:90% Lots of cool things. 01:10:35.850 --> 01:10:38.130 align:middle line:84% As you can imagine, last year, the emphasis 01:10:38.130 --> 01:10:41.220 align:middle line:84% was a little bit more hardware-intensive 01:10:41.220 --> 01:10:43.860 align:middle line:84% because the people who are taking the classes 01:10:43.860 --> 01:10:45.750 align:middle line:90% were interested in that. 01:10:45.750 --> 01:10:49.110 align:middle line:84% But this year, we have a more diverse crowd here. 01:10:49.110 --> 01:10:53.070 align:middle line:84% So I'm expecting projects in just art or photography, 01:10:53.070 --> 01:11:02.040 align:middle line:84% beautiful synthesis of images, real-time XCS systems, maybe 01:11:02.040 --> 01:11:04.410 align:middle line:84% some different types of scientific imaging, 01:11:04.410 --> 01:11:09.930 align:middle line:84% maybe a new microscope, anything of that variety. 01:11:09.930 --> 01:11:11.590 align:middle line:84% And we'll spend a lot of time together. 01:11:11.590 --> 01:11:13.800 align:middle line:84% We have four or five mentors assigned. 01:11:13.800 --> 01:11:18.040 align:middle line:84% We're working with you on the final projects. 01:11:18.040 --> 01:11:18.850 align:middle line:90% All right. 01:11:18.850 --> 01:11:20.095 align:middle line:90% So let's take a short break. 01:11:20.095 --> 01:11:21.970 align:middle line:84% And the short break, we are going to show you 01:11:21.970 --> 01:11:27.430 align:middle line:84% some cool fluorescent demos here. 01:11:27.430 --> 01:11:29.740 align:middle line:84% And then, we'll start back in about 10 minutes 01:11:29.740 --> 01:11:32.890 align:middle line:84% to talk about other types of image. 01:11:32.890 --> 01:11:36.630 align:middle line:90% 01:11:36.630 --> 01:11:42.080 align:middle line:84% So here's a big question for photography. 01:11:42.080 --> 01:11:43.967 align:middle line:90% 1930s, we had big cameras. 01:11:43.967 --> 01:11:46.175 align:middle line:84% The guy had to get under the cloth to take a picture. 01:11:46.175 --> 01:11:48.260 align:middle line:84% AUDIENCE: That's actually a press camera. 01:11:48.260 --> 01:11:49.340 align:middle line:90% It's pretty small. 01:11:49.340 --> 01:11:49.940 align:middle line:90% And people 01:11:49.940 --> 01:11:50.440 align:middle line:90% [INTERPOSING VOICES] 01:11:50.440 --> 01:11:52.070 align:middle line:84% RAMESH RASKAR: This is more recent. 01:11:52.070 --> 01:11:53.925 align:middle line:84% But I'm guessing you have cameras 01:11:53.925 --> 01:11:55.040 align:middle line:90% that are bigger than this? 01:11:55.040 --> 01:11:55.665 align:middle line:90% AUDIENCE: Yeah. 01:11:55.665 --> 01:11:58.280 align:middle line:90% 01:11:58.280 --> 01:12:00.200 align:middle line:84% RAMESH RASKAR: And there's nothing wrong 01:12:00.200 --> 01:12:03.620 align:middle line:84% in having large cameras [INAUDIBLE].. 01:12:03.620 --> 01:12:10.430 align:middle line:84% And that has shrunk down to something, this, and this, 01:12:10.430 --> 01:12:13.920 align:middle line:84% and $0.20 with all the electronics, and processing, 01:12:13.920 --> 01:12:14.850 align:middle line:90% and so on. 01:12:14.850 --> 01:12:20.300 align:middle line:84% But when you think about lighting, not much has changed. 01:12:20.300 --> 01:12:22.520 align:middle line:84% It has definitely not reduced in size. 01:12:22.520 --> 01:12:27.870 align:middle line:84% It definitely has not reduced in cost or convenience and so on. 01:12:27.870 --> 01:12:34.370 align:middle line:84% And in a way, what really distinguishes consumer 01:12:34.370 --> 01:12:36.410 align:middle line:84% photography from professional photography 01:12:36.410 --> 01:12:40.740 align:middle line:84% is not the camera anymore, but the lighting. 01:12:40.740 --> 01:12:44.373 align:middle line:84% So this is a challenge for our community, who 01:12:44.373 --> 01:12:46.040 align:middle line:84% we like to think about making everything 01:12:46.040 --> 01:12:49.370 align:middle line:90% programmable and easy to use. 01:12:49.370 --> 01:12:52.515 align:middle line:84% The same revolution will not happen for illumination. 01:12:52.515 --> 01:12:56.210 align:middle line:90% 01:12:56.210 --> 01:12:59.030 align:middle line:84% So we're going to explore that a little bit into today's 01:12:59.030 --> 01:13:03.510 align:middle line:84% class, then come back and talk some more in the next class. 01:13:03.510 --> 01:13:05.870 align:middle line:84% So when it's all about cameras, we 01:13:05.870 --> 01:13:08.520 align:middle line:84% try to make it very smart with the lenses, 01:13:08.520 --> 01:13:10.910 align:middle line:84% and different sensors, and new processing. 01:13:10.910 --> 01:13:12.830 align:middle line:84% But the light source is still a flash. 01:13:12.830 --> 01:13:15.030 align:middle line:84% Maybe it has fancy umbrellas and so on. 01:13:15.030 --> 01:13:18.200 align:middle line:90% But it's mostly a flash. 01:13:18.200 --> 01:13:20.570 align:middle line:84% So what we want to do is replace that 01:13:20.570 --> 01:13:23.600 align:middle line:84% into a more programmable illumination 01:13:23.600 --> 01:13:25.220 align:middle line:90% in a very high-dimensional way. 01:13:25.220 --> 01:13:27.800 align:middle line:84% It's geometry recorded for the illumination 01:13:27.800 --> 01:13:31.410 align:middle line:90% field in time and color. 01:13:31.410 --> 01:13:32.150 align:middle line:90% All right? 01:13:32.150 --> 01:13:35.990 align:middle line:84% So some of the earliest examples, and this whole field 01:13:35.990 --> 01:13:38.197 align:middle line:84% is called computational illumination, 01:13:38.197 --> 01:13:40.280 align:middle line:84% we're going to use computing even in the lighting. 01:13:40.280 --> 01:13:42.950 align:middle line:90% 01:13:42.950 --> 01:13:47.240 align:middle line:84% And maybe the pioneer was Edgerton, Dr. Edgerton 01:13:47.240 --> 01:13:48.650 align:middle line:90% right here. 01:13:48.650 --> 01:13:53.840 align:middle line:84% Very famous examples of bullets going through through an apple. 01:13:53.840 --> 01:13:56.840 align:middle line:84% And Santiago saying, you have this one or-- 01:13:56.840 --> 01:13:58.160 align:middle line:90% AUDIENCE: The three balloons. 01:13:58.160 --> 01:13:59.660 align:middle line:84% RAMESH RASKAR: Three balloons, yeah. 01:13:59.660 --> 01:14:03.290 align:middle line:84% So [INAUDIBLE] has beautiful, beautiful pictures. 01:14:03.290 --> 01:14:04.480 align:middle line:90% Are they in your dorm room? 01:14:04.480 --> 01:14:05.480 align:middle line:90% AUDIENCE: I've seen one. 01:14:05.480 --> 01:14:06.200 align:middle line:90% RAMESH RASKAR: The same one? 01:14:06.200 --> 01:14:06.860 align:middle line:90% Yeah, great. 01:14:06.860 --> 01:14:08.090 align:middle line:90% So beautiful pictures. 01:14:08.090 --> 01:14:11.505 align:middle line:84% But they were not captured with smart cameras. 01:14:11.505 --> 01:14:13.130 align:middle line:84% They were captured with smart lighting. 01:14:13.130 --> 01:14:16.790 align:middle line:90% 01:14:16.790 --> 01:14:18.860 align:middle line:84% You have an ordinary camera, where the shutter 01:14:18.860 --> 01:14:21.230 align:middle line:90% is open for a certain duration. 01:14:21.230 --> 01:14:26.330 align:middle line:84% But then, you have a flash that freezes that motion, which, 01:14:26.330 --> 01:14:28.580 align:middle line:90% as I recall, was troublesome. 01:14:28.580 --> 01:14:33.440 align:middle line:84% And in this particular case, you have a sequential strobe that 01:14:33.440 --> 01:14:35.458 align:middle line:90% captures this whole circle. 01:14:35.458 --> 01:14:36.750 align:middle line:90% Nothing smart about the camera. 01:14:36.750 --> 01:14:40.970 align:middle line:90% Really smart strobed lighting. 01:14:40.970 --> 01:14:42.410 align:middle line:84% And of course, [? Doc ?] Edgerton 01:14:42.410 --> 01:14:47.360 align:middle line:84% came up with chemical processes to create very sharp duration 01:14:47.360 --> 01:14:49.170 align:middle line:90% light sources. 01:14:49.170 --> 01:14:53.180 align:middle line:84% But now, we have LEDs and solid state devices. 01:14:53.180 --> 01:14:57.980 align:middle line:84% We can control light down to a few picoseconds or nanoseconds. 01:14:57.980 --> 01:15:04.760 align:middle line:84% All this can be done very easily in your house. 01:15:04.760 --> 01:15:08.420 align:middle line:84% So again, this is what distinguishes consumers 01:15:08.420 --> 01:15:10.640 align:middle line:90% from professionals. 01:15:10.640 --> 01:15:15.120 align:middle line:84% And we look at this step-by-step. 01:15:15.120 --> 01:15:17.240 align:middle line:84% And this will be a more technical discussion 01:15:17.240 --> 01:15:22.940 align:middle line:84% about what parameters of light we can change. 01:15:22.940 --> 01:15:25.310 align:middle line:84% So let's think about, what are the things we 01:15:25.310 --> 01:15:26.650 align:middle line:90% can change about light? 01:15:26.650 --> 01:15:29.340 align:middle line:90% One is clearly the brightness. 01:15:29.340 --> 01:15:32.870 align:middle line:84% What are some other things we can change? 01:15:32.870 --> 01:15:33.750 align:middle line:90% AUDIENCE: Color. 01:15:33.750 --> 01:15:35.436 align:middle line:90% RAMESH RASKAR: Color. 01:15:35.436 --> 01:15:36.660 align:middle line:90% AUDIENCE: The coordination. 01:15:36.660 --> 01:15:38.142 align:middle line:90% RAMESH RASKAR: The cone. 01:15:38.142 --> 01:15:39.870 align:middle line:84% AUDIENCE: [INAUDIBLE] polarizing. 01:15:39.870 --> 01:15:41.970 align:middle line:90% RAMESH RASKAR: Polarization. 01:15:41.970 --> 01:15:43.200 align:middle line:90% AUDIENCE: Coherence length. 01:15:43.200 --> 01:15:45.325 align:middle line:84% RAMESH RASKAR: Go ahead and slant a little bit more 01:15:45.325 --> 01:15:46.340 align:middle line:90% exotic, yes. 01:15:46.340 --> 01:15:47.340 align:middle line:90% AUDIENCE: Diffusion. 01:15:47.340 --> 01:15:48.360 align:middle line:90% AUDIENCE: The direction. 01:15:48.360 --> 01:15:49.515 align:middle line:84% RAMESH RASKAR: The direction, yes. 01:15:49.515 --> 01:15:50.182 align:middle line:90% AUDIENCE: Color. 01:15:50.182 --> 01:15:52.650 align:middle line:84% RAMESH RASKAR: Maybe it's not even a single cone, but just 01:15:52.650 --> 01:15:53.430 align:middle line:90% a projector. 01:15:53.430 --> 01:15:57.880 align:middle line:84% Different directions have a different intensity. 01:15:57.880 --> 01:15:58.570 align:middle line:90% AUDIENCE: Time. 01:15:58.570 --> 01:15:59.403 align:middle line:90% RAMESH RASKAR: Time. 01:15:59.403 --> 01:16:03.550 align:middle line:84% So strobing, or duration, or synchronization 01:16:03.550 --> 01:16:06.940 align:middle line:90% with the captured image. 01:16:06.940 --> 01:16:08.340 align:middle line:90% AUDIENCE: Relative fades. 01:16:08.340 --> 01:16:10.120 align:middle line:84% RAMESH RASKAR: Relative fades, of course. 01:16:10.120 --> 01:16:12.550 align:middle line:84% AUDIENCE: The lenses on your lighting [INAUDIBLE].. 01:16:12.550 --> 01:16:14.050 align:middle line:84% RAMESH RASKAR: For lenses, exactly. 01:16:14.050 --> 01:16:16.360 align:middle line:84% Don't think of your flash as just a light source. 01:16:16.360 --> 01:16:21.040 align:middle line:84% But put fancy optics or masks in front of them to control, 01:16:21.040 --> 01:16:22.270 align:middle line:90% again, direction and so on. 01:16:22.270 --> 01:16:25.353 align:middle line:84% AUDIENCE: You can put inside the object [INAUDIBLE].. 01:16:25.353 --> 01:16:27.520 align:middle line:84% RAMESH RASKAR: Yeah, you can change the environment, 01:16:27.520 --> 01:16:29.050 align:middle line:84% not just illuminate the environment, 01:16:29.050 --> 01:16:30.592 align:middle line:84% but change the environment by putting 01:16:30.592 --> 01:16:33.940 align:middle line:84% some interesting lighting inside there. 01:16:33.940 --> 01:16:37.090 align:middle line:84% So this is, by the way, another great way 01:16:37.090 --> 01:16:40.450 align:middle line:90% to come up with new ideas. 01:16:40.450 --> 01:16:45.130 align:middle line:84% And you say, this picture really bothers me. 01:16:45.130 --> 01:16:45.997 align:middle line:90% What can I do? 01:16:45.997 --> 01:16:47.830 align:middle line:84% And if you want to do research in this area, 01:16:47.830 --> 01:16:49.690 align:middle line:84% you ask this question yourself, what 01:16:49.690 --> 01:16:52.990 align:middle line:84% are all the ways I can control the lighting parameters? 01:16:52.990 --> 01:16:55.450 align:middle line:84% And I'm going to go through one-by-one 01:16:55.450 --> 01:16:58.000 align:middle line:90% and see how I can attack them. 01:16:58.000 --> 01:17:01.690 align:middle line:84% And we'll see examples of exactly how that has happened 01:17:01.690 --> 01:17:03.770 align:middle line:90% over the last several years. 01:17:03.770 --> 01:17:08.260 align:middle line:84% So the simplest one is I can have light or no light. 01:17:08.260 --> 01:17:10.140 align:middle line:90% The simplest [INAUDIBLE]. 01:17:10.140 --> 01:17:12.730 align:middle line:84% The next parameter is duration, how long 01:17:12.730 --> 01:17:18.310 align:middle line:84% it's on or its brightness and so on, its position, its color, 01:17:18.310 --> 01:17:22.180 align:middle line:84% using it as a projector, especially angle, 01:17:22.180 --> 01:17:24.310 align:middle line:84% modulation in space, modulation in time, 01:17:24.310 --> 01:17:26.860 align:middle line:84% and sometimes just natural light, just time lapse, 01:17:26.860 --> 01:17:27.710 align:middle line:90% which we saw. 01:17:27.710 --> 01:17:28.210 align:middle line:90% All right? 01:17:28.210 --> 01:17:32.710 align:middle line:84% So we'll just look at a few projects to get us motivated. 01:17:32.710 --> 01:17:35.650 align:middle line:84% And we saw this one last time, a multiflash camera, 01:17:35.650 --> 01:17:40.240 align:middle line:84% where we're really trying to solve a new problem. 01:17:40.240 --> 01:17:43.420 align:middle line:84% If I want to tell you what's inside my car, 01:17:43.420 --> 01:17:47.200 align:middle line:84% I can just take a photo and send it to you. 01:17:47.200 --> 01:17:49.810 align:middle line:84% But when car companies want to do the same thing, 01:17:49.810 --> 01:17:52.620 align:middle line:90% they hire artists. 01:17:52.620 --> 01:17:56.970 align:middle line:84% So the question is, why do we hire artists to draw something 01:17:56.970 --> 01:17:58.384 align:middle line:90% that can be photographed? 01:17:58.384 --> 01:18:02.740 align:middle line:90% 01:18:02.740 --> 01:18:03.268 align:middle line:90% Yes? 01:18:03.268 --> 01:18:05.560 align:middle line:84% AUDIENCE: They might know what's more important or less 01:18:05.560 --> 01:18:06.820 align:middle line:90% important [INAUDIBLE]. 01:18:06.820 --> 01:18:10.560 align:middle line:84% RAMESH RASKAR: The semantics is very important, yes. 01:18:10.560 --> 01:18:11.370 align:middle line:90% Yes? 01:18:11.370 --> 01:18:14.728 align:middle line:84% AUDIENCE: [INAUDIBLE] reflection, so like edges. 01:18:14.728 --> 01:18:16.770 align:middle line:84% RAMESH RASKAR: So yeah, reflections are annoying. 01:18:16.770 --> 01:18:18.670 align:middle line:84% And they don't really add any information. 01:18:18.670 --> 01:18:21.135 align:middle line:84% And edges just felt more important. 01:18:21.135 --> 01:18:25.320 align:middle line:84% AUDIENCE: In that picture, so the central elements 01:18:25.320 --> 01:18:26.822 align:middle line:90% are highlighted. 01:18:26.822 --> 01:18:27.780 align:middle line:90% RAMESH RASKAR: Exactly. 01:18:27.780 --> 01:18:30.853 align:middle line:84% The hand is brighter than the rest of the car. 01:18:30.853 --> 01:18:31.770 align:middle line:90% Is that what you mean? 01:18:31.770 --> 01:18:32.395 align:middle line:90% AUDIENCE: Yeah. 01:18:32.395 --> 01:18:33.780 align:middle line:90% RAMESH RASKAR: Yeah. 01:18:33.780 --> 01:18:37.210 align:middle line:84% AUDIENCE: Some unnecessary wires or [INAUDIBLE] can be-- 01:18:37.210 --> 01:18:37.710 align:middle line:90% [LAUGHTER] 01:18:37.710 --> 01:18:40.710 align:middle line:84% RAMESH RASKAR: Yeah, all these things, it's just clutter. 01:18:40.710 --> 01:18:45.430 align:middle line:84% It's not critical to explaining what's in the car. 01:18:45.430 --> 01:18:46.540 align:middle line:90% It just looks wonderful. 01:18:46.540 --> 01:18:49.690 align:middle line:84% So yeah, unnecessary shadows, clutters, 01:18:49.690 --> 01:18:52.940 align:middle line:84% too many colors as opposed to highlighting the shape, 01:18:52.940 --> 01:18:55.360 align:middle line:84% and just marking what's moving, and using very simple, 01:18:55.360 --> 01:18:56.290 align:middle line:90% basic colors. 01:18:56.290 --> 01:18:56.790 align:middle line:90% Yes? 01:18:56.790 --> 01:18:59.650 align:middle line:84% AUDIENCE: [INAUDIBLE] strategy [INAUDIBLE]?? 01:18:59.650 --> 01:19:00.454 align:middle line:90% [LAUGHTER] 01:19:00.454 --> 01:19:01.660 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:19:01.660 --> 01:19:03.702 align:middle line:84% RAMESH RASKAR: There are some other names for it. 01:19:03.702 --> 01:19:05.060 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:19:05.060 --> 01:19:06.560 align:middle line:84% RAMESH RASKAR: Not for [INAUDIBLE].. 01:19:06.560 --> 01:19:09.970 align:middle line:84% So imagine if you can just build a camera that 01:19:09.970 --> 01:19:12.340 align:middle line:84% gives you an image on the white as opposed to black. 01:19:12.340 --> 01:19:15.620 align:middle line:84% We are not trying to put the artists out of business. 01:19:15.620 --> 01:19:17.110 align:middle line:84% But we are trying to create tools 01:19:17.110 --> 01:19:20.830 align:middle line:84% so that they don't end up time rotoscoping and drawing 01:19:20.830 --> 01:19:21.580 align:middle line:90% the lines. 01:19:21.580 --> 01:19:23.980 align:middle line:84% They get all the line maps very quickly. 01:19:23.980 --> 01:19:28.130 align:middle line:84% And then beyond that, the creative process starts. 01:19:28.130 --> 01:19:31.960 align:middle line:84% So get rid of this tedious task of finding the edges, 01:19:31.960 --> 01:19:36.160 align:middle line:84% and marking, and so on, like all the magic scissors and so on. 01:19:36.160 --> 01:19:39.690 align:middle line:90% 01:19:39.690 --> 01:19:42.920 align:middle line:84% So of course, the first solution would be just take the photo 01:19:42.920 --> 01:19:44.970 align:middle line:90% and try to find intensity edges. 01:19:44.970 --> 01:19:48.450 align:middle line:84% It turns out it doesn't work very well. 01:19:48.450 --> 01:19:52.160 align:middle line:84% And this is the problem you face in XCI all the time. 01:19:52.160 --> 01:19:55.650 align:middle line:84% You have some gestures with your hand. 01:19:55.650 --> 01:19:59.060 align:middle line:84% We might have some challenging background lighting. 01:19:59.060 --> 01:20:02.600 align:middle line:84% Then a typical method is to maybe do some edge detection, 01:20:02.600 --> 01:20:03.650 align:middle line:90% color detection. 01:20:03.650 --> 01:20:07.640 align:middle line:84% And you just hope that they are not-- 01:20:07.640 --> 01:20:10.460 align:middle line:84% the lighting is just right when you're showing the demo 01:20:10.460 --> 01:20:12.110 align:middle line:90% and you have matched-- 01:20:12.110 --> 01:20:13.790 align:middle line:84% controlled the background just enough so 01:20:13.790 --> 01:20:15.260 align:middle line:90% that everything works out. 01:20:15.260 --> 01:20:19.850 align:middle line:84% Or you can be smart about it and use-- 01:20:19.850 --> 01:20:21.950 align:middle line:84% distinguish within reflectance edges, which 01:20:21.950 --> 01:20:25.640 align:middle line:84% are edges between different materials, 01:20:25.640 --> 01:20:29.240 align:middle line:84% and distinguish them from depth edges, 01:20:29.240 --> 01:20:32.180 align:middle line:90% which are real geometric edges. 01:20:32.180 --> 01:20:34.760 align:middle line:84% So these are all intensity edges. 01:20:34.760 --> 01:20:37.070 align:middle line:84% And these are all just geometry edges. 01:20:37.070 --> 01:20:40.570 align:middle line:84% So imagine if I could take a photo of this and get this out. 01:20:40.570 --> 01:20:46.430 align:middle line:84% It would be much easier for me to write an interactive system 01:20:46.430 --> 01:20:48.390 align:middle line:90% that's based on it. 01:20:48.390 --> 01:20:51.410 align:middle line:84% AUDIENCE: Well, this is really like a computer-based question, 01:20:51.410 --> 01:20:51.920 align:middle line:90% actually. 01:20:51.920 --> 01:20:54.050 align:middle line:84% The example with the car [INAUDIBLE] 01:20:54.050 --> 01:20:57.230 align:middle line:84% maybe a professional meeting because a human being knows 01:20:57.230 --> 01:20:58.070 align:middle line:90% what to abstract 01:20:58.070 --> 01:20:59.330 align:middle line:90% [INTERPOSING VOICES] 01:20:59.330 --> 01:21:01.288 align:middle line:84% AUDIENCE: --to normally emphasize [INAUDIBLE].. 01:21:01.288 --> 01:21:04.530 align:middle line:84% RAMESH RASKAR: Yes, it's multiple goals. 01:21:04.530 --> 01:21:07.250 align:middle line:84% So one goal is a diminished reality-- 01:21:07.250 --> 01:21:08.600 align:middle line:90% [LAUGHTER] 01:21:08.600 --> 01:21:10.126 align:middle line:90% --trademarked by-- 01:21:10.126 --> 01:21:11.481 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:21:11.481 --> 01:21:12.356 align:middle line:90% RAMESH RASKAR: Sorry? 01:21:12.356 --> 01:21:14.240 align:middle line:84% AUDIENCE: [INAUDIBLE] the internet. 01:21:14.240 --> 01:21:15.560 align:middle line:90% RAMESH RASKAR: All right. 01:21:15.560 --> 01:21:19.890 align:middle line:84% So [INAUDIBLE],, who's a scientist in our group, 01:21:19.890 --> 01:21:23.910 align:middle line:84% has a very interesting way of putting this. 01:21:23.910 --> 01:21:27.770 align:middle line:84% He says, photography, it's well-known. 01:21:27.770 --> 01:21:35.030 align:middle line:84% Photography is the art of exclusion, 01:21:35.030 --> 01:21:37.100 align:middle line:90% de-emphasizing things that-- 01:21:37.100 --> 01:21:38.540 align:middle line:84% the photographer makes a decision 01:21:38.540 --> 01:21:41.150 align:middle line:84% on what to de-emphasize either by using focus 01:21:41.150 --> 01:21:44.990 align:middle line:84% or by cropping it and so on so that the photographer can 01:21:44.990 --> 01:21:48.980 align:middle line:84% convey to the viewer what's most important. 01:21:48.980 --> 01:21:52.310 align:middle line:84% But what we're doing computationally now 01:21:52.310 --> 01:21:55.910 align:middle line:84% is adding another level of exclusion. 01:21:55.910 --> 01:21:58.370 align:middle line:84% In this case, we're excluding the intensity edges 01:21:58.370 --> 01:22:01.740 align:middle line:84% and just keeping depth edges and all these other things. 01:22:01.740 --> 01:22:04.640 align:middle line:84% So computationally, we would extend 01:22:04.640 --> 01:22:07.002 align:middle line:90% the concept of exclusion. 01:22:07.002 --> 01:22:08.960 align:middle line:84% So one could be for nonphotorealistic rendering 01:22:08.960 --> 01:22:10.490 align:middle line:90% for creating cartoons. 01:22:10.490 --> 01:22:11.990 align:middle line:90% One could be for XCI. 01:22:11.990 --> 01:22:13.970 align:middle line:84% And there are lots of other things. 01:22:13.970 --> 01:22:16.400 align:middle line:84% Right now, this is being used in Mitsubishi products 01:22:16.400 --> 01:22:18.410 align:middle line:90% for some really bizarre things. 01:22:18.410 --> 01:22:22.410 align:middle line:90% 01:22:22.410 --> 01:22:23.840 align:middle line:90% So we saw the idea last time. 01:22:23.840 --> 01:22:26.030 align:middle line:90% You get the slivers of shadows. 01:22:26.030 --> 01:22:28.520 align:middle line:84% And by analyzing the shadows, you 01:22:28.520 --> 01:22:32.180 align:middle line:84% can continue-- compute the depth discontinuities. 01:22:32.180 --> 01:22:34.910 align:middle line:84% And some people like to call it slivers. 01:22:34.910 --> 01:22:37.700 align:middle line:84% Some people like to call it occluding contours or shape 01:22:37.700 --> 01:22:38.480 align:middle line:90% boundaries. 01:22:38.480 --> 01:22:40.460 align:middle line:90% But they're not so precise. 01:22:40.460 --> 01:22:42.140 align:middle line:84% And depth discontinuities is what 01:22:42.140 --> 01:22:43.320 align:middle line:90% we're really talking about. 01:22:43.320 --> 01:22:46.670 align:middle line:84% So when I take a photo, then the depth 01:22:46.670 --> 01:22:49.370 align:middle line:84% of this pixel and depth of the pixel 01:22:49.370 --> 01:22:53.390 align:middle line:84% on the other side of this screen has a [INAUDIBLE] 01:22:53.390 --> 01:22:54.560 align:middle line:90% discontinuity. 01:22:54.560 --> 01:22:57.320 align:middle line:84% And that's really what we're talking about as 01:22:57.320 --> 01:23:01.010 align:middle line:84% opposed to slivers, where it's usually 01:23:01.010 --> 01:23:02.580 align:middle line:90% about just the external part. 01:23:02.580 --> 01:23:04.310 align:middle line:84% Well, if I put my hand over here, 01:23:04.310 --> 01:23:06.800 align:middle line:84% this is still a depth discontinuity 01:23:06.800 --> 01:23:08.510 align:middle line:90% from my hand to my body. 01:23:08.510 --> 01:23:11.630 align:middle line:90% So I would like to capture that. 01:23:11.630 --> 01:23:15.240 align:middle line:84% And shape boundaries is also not very precise. 01:23:15.240 --> 01:23:18.200 align:middle line:84% So depth discontinuity is really what we're talking about. 01:23:18.200 --> 01:23:23.380 align:middle line:84% And imagine if this is in the Sony EyeToy or the Microsoft 01:23:23.380 --> 01:23:23.900 align:middle line:90% Natal. 01:23:23.900 --> 01:23:27.260 align:middle line:84% Then if you have a depth camera, if I put my hand really close 01:23:27.260 --> 01:23:29.830 align:middle line:84% to my body, it cannot distinguish the depth with 01:23:29.830 --> 01:23:32.420 align:middle line:84% respect to the camera of my body versus my hand. 01:23:32.420 --> 01:23:34.070 align:middle line:84% But with this camera, I can still 01:23:34.070 --> 01:23:38.480 align:middle line:84% continue to detect that there's a shadow that's 01:23:38.480 --> 01:23:40.010 align:middle line:90% created from that. 01:23:40.010 --> 01:23:42.053 align:middle line:84% AUDIENCE: How about that laser from-- 01:23:42.053 --> 01:23:42.960 align:middle line:90% RAMESH RASKAR: Sorry? 01:23:42.960 --> 01:23:44.450 align:middle line:84% AUDIENCE: The laser projector camera? 01:23:44.450 --> 01:23:46.658 align:middle line:84% RAMESH RASKAR: Yeah, so actually, the Microsoft Natal 01:23:46.658 --> 01:23:50.150 align:middle line:84% uses this PrimeSense sensor, which is not time of flight, 01:23:50.150 --> 01:23:52.700 align:middle line:90% but based on triangulation. 01:23:52.700 --> 01:23:54.800 align:middle line:84% But anything that senses depth as 01:23:54.800 --> 01:23:57.830 align:middle line:84% opposed to depth discontinuity will have the same problem 01:23:57.830 --> 01:23:59.630 align:middle line:84% because, as you get closer and closer, 01:23:59.630 --> 01:24:00.740 align:middle line:84% you'll not be able to detect that. 01:24:00.740 --> 01:24:02.280 align:middle line:84% AUDIENCE: If you were testing for a piece of paper, 01:24:02.280 --> 01:24:03.620 align:middle line:90% it seems pretty good. 01:24:03.620 --> 01:24:04.310 align:middle line:90% RAMESH RASKAR: It was very good. 01:24:04.310 --> 01:24:05.268 align:middle line:90% Yeah, it was very good. 01:24:05.268 --> 01:24:08.540 align:middle line:84% Yeah, so if it's a really good depth-sensing camera, 01:24:08.540 --> 01:24:09.900 align:middle line:90% then you're all set. 01:24:09.900 --> 01:24:11.090 align:middle line:90% But it has poor resolution. 01:24:11.090 --> 01:24:13.670 align:middle line:84% And it's a much more complex device. 01:24:13.670 --> 01:24:18.430 align:middle line:90% 01:24:18.430 --> 01:24:23.180 align:middle line:84% So again, shadows to the right, shadows to the left, and so on. 01:24:23.180 --> 01:24:24.430 align:middle line:90% But then, look at the shadows. 01:24:24.430 --> 01:24:27.490 align:middle line:84% They are not just one-pixel-wide. 01:24:27.490 --> 01:24:29.200 align:middle line:84% Depending on the depth difference, 01:24:29.200 --> 01:24:31.690 align:middle line:84% they could be very wide or very narrow. 01:24:31.690 --> 01:24:35.110 align:middle line:84% So you want to create an output that's exactly one-pixel-wide. 01:24:35.110 --> 01:24:36.860 align:middle line:90% So how does this work? 01:24:36.860 --> 01:24:39.550 align:middle line:84% So before we go there, I guess, is there intensity in this? 01:24:39.550 --> 01:24:41.347 align:middle line:90% These are [INAUDIBLE]. 01:24:41.347 --> 01:24:43.180 align:middle line:84% And we can just put that together and create 01:24:43.180 --> 01:24:45.930 align:middle line:90% a cartoon in real-time. 01:24:45.930 --> 01:24:49.030 align:middle line:90% In fact, we demonstrated this. 01:24:49.030 --> 01:24:51.400 align:middle line:84% So yeah, this is another example, my old '93 Honda 01:24:51.400 --> 01:24:52.240 align:middle line:90% Civic. 01:24:52.240 --> 01:24:56.380 align:middle line:84% And this is what you will get if you use the intensity detector. 01:24:56.380 --> 01:24:58.780 align:middle line:84% This is what you get from the four flashes. 01:24:58.780 --> 01:25:00.430 align:middle line:84% And you will see the Honda sign here. 01:25:00.430 --> 01:25:02.720 align:middle line:90% That's barely visible here. 01:25:02.720 --> 01:25:08.080 align:middle line:84% That's because the Honda sign has a relief. 01:25:08.080 --> 01:25:09.320 align:middle line:90% It's a height field. 01:25:09.320 --> 01:25:11.470 align:middle line:84% It's a geometric rather than texture. 01:25:11.470 --> 01:25:17.720 align:middle line:84% And all these scratch marks and rusting is just texture. 01:25:17.720 --> 01:25:20.320 align:middle line:84% It does not contribute to the geometry. 01:25:20.320 --> 01:25:25.000 align:middle line:84% And if you try to locate the spark plugs or the dipstick, 01:25:25.000 --> 01:25:27.490 align:middle line:90% it's barely visible here. 01:25:27.490 --> 01:25:31.030 align:middle line:84% But again, here, you can see it right there. 01:25:31.030 --> 01:25:34.340 align:middle line:84% So the shapes are what we're really looking for. 01:25:34.340 --> 01:25:37.210 align:middle line:84% And we've got the four images, input images, just four images. 01:25:37.210 --> 01:25:40.480 align:middle line:84% You can see to the naked eye, they 01:25:40.480 --> 01:25:43.720 align:middle line:84% look identical to a naive observer. 01:25:43.720 --> 01:25:45.850 align:middle line:90% But you look at the shadows. 01:25:45.850 --> 01:25:48.910 align:middle line:84% Here, the shadows are to the right of these pipes. 01:25:48.910 --> 01:25:51.370 align:middle line:84% Here, the shadow are to the left of these pipes. 01:25:51.370 --> 01:25:53.600 align:middle line:84% Here, the shadows are at the top. 01:25:53.600 --> 01:25:55.690 align:middle line:84% And here, the shadows are at the bottom. 01:25:55.690 --> 01:26:00.250 align:middle line:84% And by analyzing the shadows, you can find the [INAUDIBLE].. 01:26:00.250 --> 01:26:00.880 align:middle line:90% Yeah? 01:26:00.880 --> 01:26:03.940 align:middle line:84% AUDIENCE: In practice, honestly, the farther 01:26:03.940 --> 01:26:06.310 align:middle line:84% continuous from 12 feet away, it's 01:26:06.310 --> 01:26:09.107 align:middle line:84% not as effective because your flashes are pretty close to the 01:26:09.107 --> 01:26:09.940 align:middle line:90% [INTERPOSING VOICES] 01:26:09.940 --> 01:26:10.360 align:middle line:90% RAMESH RASKAR: Right. 01:26:10.360 --> 01:26:10.840 align:middle line:90% Exactly. 01:26:10.840 --> 01:26:12.840 align:middle line:84% So we'll talk a little bit about what parameters 01:26:12.840 --> 01:26:15.610 align:middle line:84% you are to choose to get an optimal performance. 01:26:15.610 --> 01:26:17.440 align:middle line:84% And yeah, there's definitely a sweet spot 01:26:17.440 --> 01:26:19.873 align:middle line:90% where it works in the system. 01:26:19.873 --> 01:26:23.240 align:middle line:90% 01:26:23.240 --> 01:26:25.580 align:middle line:90% Some more examples here. 01:26:25.580 --> 01:26:28.610 align:middle line:84% And I just want to describe to you very quickly 01:26:28.610 --> 01:26:31.550 align:middle line:84% the geometric parameters that make it happen. 01:26:31.550 --> 01:26:35.060 align:middle line:84% But we were showing this live at SIGGRAPH. 01:26:35.060 --> 01:26:37.640 align:middle line:84% And you can stand in front of this camera, where 01:26:37.640 --> 01:26:42.730 align:middle line:84% every iconic fourth frame had a different flashlight going off. 01:26:42.730 --> 01:26:43.730 align:middle line:90% They put in the lead-in. 01:26:43.730 --> 01:26:44.938 align:middle line:90% You can stand in front of it. 01:26:44.938 --> 01:26:46.790 align:middle line:90% And it's a cartoon. 01:26:46.790 --> 01:26:52.220 align:middle line:84% And I'm a big fan of the A-ha video from the 1980s, 01:26:52.220 --> 01:26:53.100 align:middle line:90% "Take On Me." 01:26:53.100 --> 01:26:53.600 align:middle line:90% [LAUGHTER] 01:26:53.600 --> 01:26:54.290 align:middle line:90% You remember that? 01:26:54.290 --> 01:26:54.915 align:middle line:90% AUDIENCE: Yeah. 01:26:54.915 --> 01:26:56.013 align:middle line:90% AUDIENCE: Wow. 01:26:56.013 --> 01:26:58.430 align:middle line:84% RAMESH RASKAR: I thought I was too old to appreciate that. 01:26:58.430 --> 01:27:02.330 align:middle line:84% But so in the A-ha video, there's 01:27:02.330 --> 01:27:06.410 align:middle line:84% this very beautiful effect where on one side of the glass 01:27:06.410 --> 01:27:08.300 align:middle line:90% screen, the world is cartoon. 01:27:08.300 --> 01:27:10.290 align:middle line:90% And the other side, it's real. 01:27:10.290 --> 01:27:16.857 align:middle line:84% And so when we were showing this, it goes on for four days, 01:27:16.857 --> 01:27:18.440 align:middle line:84% after the end of the first day, people 01:27:18.440 --> 01:27:19.898 align:middle line:84% started saying, oh, this reminds us 01:27:19.898 --> 01:27:22.820 align:middle line:84% of the A-ha video, the "Take On Me" video. 01:27:22.820 --> 01:27:24.830 align:middle line:84% And I said, yeah, that's really true. 01:27:24.830 --> 01:27:27.110 align:middle line:84% So we got really sick of those comments. 01:27:27.110 --> 01:27:30.200 align:middle line:84% And on our demo, instead of calling it-- 01:27:30.200 --> 01:27:32.480 align:middle line:84% we had some really boring technical name. 01:27:32.480 --> 01:27:35.510 align:middle line:84% We said, this is the A-ha demo, "Take On Me" demo. 01:27:35.510 --> 01:27:40.640 align:middle line:84% And we had it in big letters on a huge screen. 01:27:40.640 --> 01:27:46.633 align:middle line:84% And then, on the second day, the curator of the show came to me. 01:27:46.633 --> 01:27:48.050 align:middle line:84% And she introduced me to this guy. 01:27:48.050 --> 01:27:54.357 align:middle line:84% And she said, this is, I forget his name, David Patterson. 01:27:54.357 --> 01:27:55.940 align:middle line:84% This David Patterson, do you know him? 01:27:55.940 --> 01:27:56.982 align:middle line:90% I said, I don't know him. 01:27:56.982 --> 01:27:59.960 align:middle line:84% He said, he was the music video director of A-ha. 01:27:59.960 --> 01:28:02.420 align:middle line:90% [LAUGHTER] 01:28:02.420 --> 01:28:04.490 align:middle line:84% And so I said, wow, that's great. 01:28:04.490 --> 01:28:05.990 align:middle line:84% We're really inspired by your video. 01:28:05.990 --> 01:28:08.450 align:middle line:90% He looks up on the screen. 01:28:08.450 --> 01:28:10.622 align:middle line:84% And our demo was inside a closed quote. 01:28:10.622 --> 01:28:12.080 align:middle line:84% And they're showing the live output 01:28:12.080 --> 01:28:14.343 align:middle line:90% of that on the big screen. 01:28:14.343 --> 01:28:16.010 align:middle line:84% And he said, what are you showing there? 01:28:16.010 --> 01:28:16.885 align:middle line:90% I said, this is live. 01:28:16.885 --> 01:28:17.870 align:middle line:90% We can go inside. 01:28:17.870 --> 01:28:22.080 align:middle line:84% And this is being computed in real-time. 01:28:22.080 --> 01:28:24.890 align:middle line:84% And he said, that's impossible because when we were doing it 01:28:24.890 --> 01:28:28.700 align:middle line:84% in the mid-80s, every frame took them a whole day-- 01:28:28.700 --> 01:28:30.860 align:middle line:90% [LAUGHTER] 01:28:30.860 --> 01:28:34.820 align:middle line:90% --to rotoscope and so on. 01:28:34.820 --> 01:28:37.850 align:middle line:90% And so we were excited. 01:28:37.850 --> 01:28:38.690 align:middle line:90% Patterson is here. 01:28:38.690 --> 01:28:39.800 align:middle line:90% So he comes inside. 01:28:39.800 --> 01:28:41.725 align:middle line:90% He acts very cool. 01:28:41.725 --> 01:28:42.600 align:middle line:90% He looks right there. 01:28:42.600 --> 01:28:43.790 align:middle line:90% He looks at the images. 01:28:43.790 --> 01:28:45.370 align:middle line:84% And we take-- we get very excited. 01:28:45.370 --> 01:28:47.000 align:middle line:90% We take pictures with him. 01:28:47.000 --> 01:28:48.630 align:middle line:90% He leaves. 01:28:48.630 --> 01:28:53.185 align:middle line:84% But after 1/2 an hour, he brings his son with him. 01:28:53.185 --> 01:28:54.560 align:middle line:84% And he started explaining to him. 01:28:54.560 --> 01:28:56.480 align:middle line:84% His son is two or three years old. 01:28:56.480 --> 01:28:59.510 align:middle line:84% He said, wow, remember in the 1980s, I worked on this video? 01:28:59.510 --> 01:29:01.940 align:middle line:84% These guys are doing it in real-time now. 01:29:01.940 --> 01:29:03.723 align:middle line:90% And we said, welcome. 01:29:03.723 --> 01:29:05.390 align:middle line:84% We're pretty happy you brought your son. 01:29:05.390 --> 01:29:06.650 align:middle line:90% You must be excited. 01:29:06.650 --> 01:29:08.600 align:middle line:84% After a couple of hours, he brings his wife. 01:29:08.600 --> 01:29:11.175 align:middle line:90% [LAUGHTER] 01:29:11.175 --> 01:29:12.800 align:middle line:84% And then, it just goes on all day long. 01:29:12.800 --> 01:29:16.740 align:middle line:84% He just keeps bringing people to show them how this works. 01:29:16.740 --> 01:29:20.000 align:middle line:84% So that was a high-point of that [INAUDIBLE].. 01:29:20.000 --> 01:29:21.350 align:middle line:90% So how does it work? 01:29:21.350 --> 01:29:24.230 align:middle line:90% So you have a camera. 01:29:24.230 --> 01:29:27.890 align:middle line:84% And you have your flashlight, which is shown as P here. 01:29:27.890 --> 01:29:31.490 align:middle line:84% And then, because of the parallax between the lights 01:29:31.490 --> 01:29:34.190 align:middle line:84% and between the lens and the light source, 01:29:34.190 --> 01:29:37.520 align:middle line:84% you're going to cast a shadow of this object. 01:29:37.520 --> 01:29:39.530 align:middle line:84% If the flash is to the left, the shadow 01:29:39.530 --> 01:29:41.910 align:middle line:84% is going to be cast on the right. 01:29:41.910 --> 01:29:44.960 align:middle line:84% And if you just freeze the rig from the light source 01:29:44.960 --> 01:29:47.870 align:middle line:84% to the object to the shadow, it's 01:29:47.870 --> 01:29:49.730 align:middle line:84% going to be projected in the image 01:29:49.730 --> 01:29:52.970 align:middle line:90% as this particular vector. 01:29:52.970 --> 01:29:57.590 align:middle line:84% Now what's interesting is that it turns out 01:29:57.590 --> 01:29:59.135 align:middle line:84% we need at least three light sources. 01:29:59.135 --> 01:30:00.260 align:middle line:90% You don't have to use four. 01:30:00.260 --> 01:30:02.270 align:middle line:84% But you need at least three light sources. 01:30:02.270 --> 01:30:05.240 align:middle line:84% And let's say you have three light sources, P1, P2, P3. 01:30:05.240 --> 01:30:09.410 align:middle line:84% And the image of the light source is denoted by E. 01:30:09.410 --> 01:30:12.290 align:middle line:84% So the shadow, it turns out, lies 01:30:12.290 --> 01:30:14.090 align:middle line:84% along a so-called [INAUDIBLE],, those 01:30:14.090 --> 01:30:17.610 align:middle line:84% of you who are familiar with the [INAUDIBLE] geometry. 01:30:17.610 --> 01:30:24.200 align:middle line:84% And as long as we make sure that for any given 01:30:24.200 --> 01:30:26.000 align:middle line:84% silhouette, any given depth edge, 01:30:26.000 --> 01:30:29.660 align:middle line:84% there is at least one light that's on one side of that edge 01:30:29.660 --> 01:30:31.250 align:middle line:84% and there's at least one light that's 01:30:31.250 --> 01:30:33.800 align:middle line:84% on the other side of that edge, we 01:30:33.800 --> 01:30:36.620 align:middle line:84% can guarantee that at least in one image, 01:30:36.620 --> 01:30:38.270 align:middle line:90% you will see a shadow. 01:30:38.270 --> 01:30:43.400 align:middle line:84% And at least in one image, you will not see a shadow. 01:30:43.400 --> 01:30:45.665 align:middle line:84% And while being able to compare-- 01:30:45.665 --> 01:30:48.338 align:middle line:84% so out of four, in our case, at least one of them 01:30:48.338 --> 01:30:49.130 align:middle line:90% will have a shadow. 01:30:49.130 --> 01:30:52.190 align:middle line:84% And at least one of them will not have a shadow. 01:30:52.190 --> 01:30:56.990 align:middle line:84% So by guaranteeing that, we can analyze the four images 01:30:56.990 --> 01:30:58.950 align:middle line:90% and compute how that works. 01:30:58.950 --> 01:31:03.260 align:middle line:90% So here's a very simple example. 01:31:03.260 --> 01:31:04.650 align:middle line:90% It flashed to the left. 01:31:04.650 --> 01:31:05.900 align:middle line:90% So the shadow is on the right. 01:31:05.900 --> 01:31:07.233 align:middle line:90% Here, the flash is on the right. 01:31:07.233 --> 01:31:08.480 align:middle line:90% So the shadow is on the left. 01:31:08.480 --> 01:31:14.540 align:middle line:84% And all you do is take these two images, find the max operator, 01:31:14.540 --> 01:31:16.610 align:middle line:90% take the max of every pixel. 01:31:16.610 --> 01:31:18.470 align:middle line:84% And this is what you can do with HDR Shop. 01:31:18.470 --> 01:31:19.970 align:middle line:84% I don't know if you can do it with Photoshop. 01:31:19.970 --> 01:31:21.870 align:middle line:84% Can you just take two images in Photoshop 01:31:21.870 --> 01:31:23.910 align:middle line:90% and find the max of the two? 01:31:23.910 --> 01:31:27.090 align:middle line:84% At every pixel, I want to compare the two pixels. 01:31:27.090 --> 01:31:29.485 align:middle line:84% And I want to take the maximum of the two. 01:31:29.485 --> 01:31:30.360 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:31:30.360 --> 01:31:30.870 align:middle line:90% RAMESH RASKAR: You can do that? 01:31:30.870 --> 01:31:32.610 align:middle line:84% AUDIENCE: Well, I learned that there 01:31:32.610 --> 01:31:35.370 align:middle line:84% are filters that you can do that with one layer and the layer 01:31:35.370 --> 01:31:37.330 align:middle line:90% behind it and [INAUDIBLE]. 01:31:37.330 --> 01:31:38.947 align:middle line:84% I don't know if it's exactly that. 01:31:38.947 --> 01:31:39.780 align:middle line:90% RAMESH RASKAR: Yeah. 01:31:39.780 --> 01:31:41.722 align:middle line:90% AUDIENCE: But [INAUDIBLE]. 01:31:41.722 --> 01:31:42.680 align:middle line:90% RAMESH RASKAR: Exactly. 01:31:42.680 --> 01:31:44.760 align:middle line:84% But nobody cares about doing nice, simple 01:31:44.760 --> 01:31:46.930 align:middle line:90% mathematical operations. 01:31:46.930 --> 01:31:49.178 align:middle line:84% So HDR Shop is great for these things. 01:31:49.178 --> 01:31:50.970 align:middle line:84% All these things, you can do it in HDR Shop 01:31:50.970 --> 01:31:53.940 align:middle line:90% in 15 seconds [INAUDIBLE]. 01:31:53.940 --> 01:31:56.100 align:middle line:84% AUDIENCE: Another [INAUDIBLE] introduced it. 01:31:56.100 --> 01:31:57.930 align:middle line:84% But [INAUDIBLE] image processing, 01:31:57.930 --> 01:31:59.499 align:middle line:90% ImageJ is really a nice-- 01:31:59.499 --> 01:32:00.416 align:middle line:90% RAMESH RASKAR: ImageJ? 01:32:00.416 --> 01:32:01.230 align:middle line:90% AUDIENCE: ImageJ. 01:32:01.230 --> 01:32:02.730 align:middle line:84% It's Java-based and all open-source. 01:32:02.730 --> 01:32:05.160 align:middle line:84% So if you want to take a look at the code, it's all there. 01:32:05.160 --> 01:32:05.520 align:middle line:90% RAMESH RASKAR: Excellent. 01:32:05.520 --> 01:32:08.310 align:middle line:84% AUDIENCE: And also, you can script all of these functions 01:32:08.310 --> 01:32:10.320 align:middle line:84% with a lot of images using JavaScript-- 01:32:10.320 --> 01:32:10.950 align:middle line:90% RAMESH RASKAR: Excellent. 01:32:10.950 --> 01:32:11.880 align:middle line:84% AUDIENCE: --which is really useful. 01:32:11.880 --> 01:32:13.790 align:middle line:84% RAMESH RASKAR: Just send a link to the class or to Sam 01:32:13.790 --> 01:32:14.832 align:middle line:90% so it'll go in the notes. 01:32:14.832 --> 01:32:15.457 align:middle line:90% AUDIENCE: Cool. 01:32:15.457 --> 01:32:16.374 align:middle line:90% RAMESH RASKAR: ImageJ. 01:32:16.374 --> 01:32:18.250 align:middle line:84% I know there's OpenCV and all those things. 01:32:18.250 --> 01:32:19.620 align:middle line:90% So I'm sure you'll use them. 01:32:19.620 --> 01:32:25.380 align:middle line:84% In MATLAB, again, it's one-line command, max a, b, a vector, 01:32:25.380 --> 01:32:25.880 align:middle line:90% b vector. 01:32:25.880 --> 01:32:29.190 align:middle line:84% But basically, one-line code to take a max of two images. 01:32:29.190 --> 01:32:33.120 align:middle line:90% In Photoshop, every 30 minutes. 01:32:33.120 --> 01:32:35.280 align:middle line:84% So all you do is take the max of the two 01:32:35.280 --> 01:32:39.090 align:middle line:84% and take the ratio of this image by the max image, which 01:32:39.090 --> 01:32:40.565 align:middle line:90% I call normalized. 01:32:40.565 --> 01:32:41.940 align:middle line:84% And then, we want to divide that. 01:32:41.940 --> 01:32:44.530 align:middle line:84% And if you divide it, all the texture goes away. 01:32:44.530 --> 01:32:45.780 align:middle line:90% And only the shadows are left. 01:32:45.780 --> 01:32:48.930 align:middle line:84% Wherever there was no shadow, you'll get a value of one. 01:32:48.930 --> 01:32:49.890 align:middle line:90% The ratio is one. 01:32:49.890 --> 01:32:51.450 align:middle line:84% Wherever there's a shadow, you'll 01:32:51.450 --> 01:32:53.450 align:middle line:84% get a value that's close to zero. 01:32:53.450 --> 01:32:56.760 align:middle line:84% It's not always zero because there's some other lighting. 01:32:56.760 --> 01:33:01.520 align:middle line:84% And now, all you have to do is if you take one scan line here, 01:33:01.520 --> 01:33:02.520 align:middle line:90% this is how you plot it. 01:33:02.520 --> 01:33:04.260 align:middle line:84% And you can see the shadows very clearly. 01:33:04.260 --> 01:33:05.760 align:middle line:90% And same here. 01:33:05.760 --> 01:33:08.280 align:middle line:84% In the left image, just scan from left to right. 01:33:08.280 --> 01:33:11.145 align:middle line:84% And wherever you see a jump from lit area to an unlit area, 01:33:11.145 --> 01:33:13.110 align:middle line:84% it has to be a [INAUDIBLE] touch. 01:33:13.110 --> 01:33:15.690 align:middle line:84% In the right image, you scan from right to left. 01:33:15.690 --> 01:33:19.110 align:middle line:84% And wherever, again, you see from lit area to unlit area, 01:33:19.110 --> 01:33:20.620 align:middle line:90% that's a [INAUDIBLE] touch. 01:33:20.620 --> 01:33:22.790 align:middle line:84% So in every one of those four images, 01:33:22.790 --> 01:33:24.540 align:middle line:84% you'll be able to find [INAUDIBLE] touches 01:33:24.540 --> 01:33:26.040 align:middle line:90% from different orientations. 01:33:26.040 --> 01:33:30.120 align:middle line:84% And then, we can take a union of all these four images. 01:33:30.120 --> 01:33:34.210 align:middle line:84% And that gives you a full [INAUDIBLE] touch image. 01:33:34.210 --> 01:33:36.810 align:middle line:84% So in this case, for example, all these internal edges 01:33:36.810 --> 01:33:38.100 align:middle line:90% are completely ignored. 01:33:38.100 --> 01:33:41.130 align:middle line:90% And all you see is silhouettes. 01:33:41.130 --> 01:33:44.490 align:middle line:84% And in MATLAB, that's all there is, 15 lines 01:33:44.490 --> 01:33:48.150 align:middle line:90% of code and no parameters. 01:33:48.150 --> 01:33:50.130 align:middle line:90% No tuning required. 01:33:50.130 --> 01:33:53.630 align:middle line:84% There's no constant that's used anyway. 01:33:53.630 --> 01:33:54.990 align:middle line:90% So it's extremely easy to use. 01:33:54.990 --> 01:33:56.090 align:middle line:90% You can implement this. 01:33:56.090 --> 01:33:56.840 align:middle line:90% You can use it. 01:33:56.840 --> 01:33:59.480 align:middle line:84% I'll give it as an option for one of the class 01:33:59.480 --> 01:34:04.040 align:middle line:84% assignments so you have a choice to use this. 01:34:04.040 --> 01:34:06.690 align:middle line:90% 01:34:06.690 --> 01:34:08.720 align:middle line:90% Any questions on this one? 01:34:08.720 --> 01:34:09.220 align:middle line:90% Yes? 01:34:09.220 --> 01:34:11.640 align:middle line:90% AUDIENCE: So you said outline. 01:34:11.640 --> 01:34:13.740 align:middle line:84% The boundary is only a one-pixel line. 01:34:13.740 --> 01:34:17.040 align:middle line:84% But if you actually provide more than one pixel, 01:34:17.040 --> 01:34:19.170 align:middle line:84% depending on how thick the shadow is, 01:34:19.170 --> 01:34:21.180 align:middle line:84% can we get more depth [INAUDIBLE]?? 01:34:21.180 --> 01:34:22.847 align:middle line:84% RAMESH RASKAR: That's a very good point. 01:34:22.847 --> 01:34:26.170 align:middle line:84% So depending on how far things are, so here, for example, 01:34:26.170 --> 01:34:28.915 align:middle line:90% the shadow is wider than here. 01:34:28.915 --> 01:34:30.540 align:middle line:84% And that's because the depth difference 01:34:30.540 --> 01:34:31.448 align:middle line:90% is larger or smaller. 01:34:31.448 --> 01:34:32.490 align:middle line:90% So you are already right. 01:34:32.490 --> 01:34:34.727 align:middle line:84% That thickness of the shadow actually 01:34:34.727 --> 01:34:36.810 align:middle line:84% tells you a little bit about the depth difference, 01:34:36.810 --> 01:34:39.465 align:middle line:84% not the depth itself, but the depth difference. 01:34:39.465 --> 01:34:41.590 align:middle line:84% AUDIENCE: So you get more 3D information out of it. 01:34:41.590 --> 01:34:43.620 align:middle line:84% RAMESH RASKAR: So you could get a little bit of 3D information 01:34:43.620 --> 01:34:44.120 align:middle line:90% as well. 01:34:44.120 --> 01:34:45.230 align:middle line:90% Yeah, that would be nice. 01:34:45.230 --> 01:34:51.840 align:middle line:90% AUDIENCE: [INAUDIBLE] capture? 01:34:51.840 --> 01:34:55.445 align:middle line:84% So [INAUDIBLE] like a machine in the shop. 01:34:55.445 --> 01:34:56.320 align:middle line:90% RAMESH RASKAR: Right. 01:34:56.320 --> 01:34:57.195 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:34:57.195 --> 01:34:57.987 align:middle line:90% RAMESH RASKAR: Yes. 01:34:57.987 --> 01:34:58.870 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:34:58.870 --> 01:35:00.410 align:middle line:84% RAMESH RASKAR: They just use a laser scanner, usually. 01:35:00.410 --> 01:35:00.800 align:middle line:90% AUDIENCE: Yeah. 01:35:00.800 --> 01:35:02.258 align:middle line:84% RAMESH RASKAR: Yeah, unfortunately. 01:35:02.258 --> 01:35:03.405 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:35:03.405 --> 01:35:05.530 align:middle line:84% RAMESH RASKAR: You can just capture the silhouette. 01:35:05.530 --> 01:35:07.840 align:middle line:84% But remember, we are not capturing depth. 01:35:07.840 --> 01:35:10.377 align:middle line:84% We're only capturing depth edges. 01:35:10.377 --> 01:35:10.960 align:middle line:90% AUDIENCE: Yes. 01:35:10.960 --> 01:35:13.585 align:middle line:84% RAMESH RASKAR: It's like saying, I'm not capturing intensities. 01:35:13.585 --> 01:35:15.893 align:middle line:84% I'm just capturing intensity edges. 01:35:15.893 --> 01:35:17.282 align:middle line:90% AUDIENCE: But [INAUDIBLE]. 01:35:17.282 --> 01:35:20.198 align:middle line:90% 01:35:20.198 --> 01:35:22.240 align:middle line:84% RAMESH RASKAR: So this work has really taken off. 01:35:22.240 --> 01:35:24.250 align:middle line:84% There have been lots of papers that are based 01:35:24.250 --> 01:35:25.480 align:middle line:90% on this particular technique. 01:35:25.480 --> 01:35:29.290 align:middle line:84% And so people have tried to do [INAUDIBLE].. 01:35:29.290 --> 01:35:31.600 align:middle line:84% Some people have tried to do some-- 01:35:31.600 --> 01:35:32.290 align:middle line:90% it's still open. 01:35:32.290 --> 01:35:32.962 align:middle line:90% It's very new. 01:35:32.962 --> 01:35:34.420 align:middle line:84% Or it's some three, four years old. 01:35:34.420 --> 01:35:39.310 align:middle line:84% And people are trying to do different color lights. 01:35:39.310 --> 01:35:40.480 align:middle line:90% So there's a lot you can do. 01:35:40.480 --> 01:35:41.485 align:middle line:90% Question up there? 01:35:41.485 --> 01:35:43.277 align:middle line:84% AUDIENCE: Could you actually capture depth? 01:35:43.277 --> 01:35:46.200 align:middle line:84% Or at least the difference between different edges 01:35:46.200 --> 01:35:48.075 align:middle line:84% so you can study the [INAUDIBLE]?? 01:35:48.075 --> 01:35:50.200 align:middle line:84% RAMESH RASKAR: If you can process the shadow width. 01:35:50.200 --> 01:35:52.420 align:middle line:84% But remember, it's not going to be as robust, 01:35:52.420 --> 01:35:56.360 align:middle line:84% because if you look at this plot, 01:35:56.360 --> 01:35:59.570 align:middle line:84% you have to estimate how wide this region is that's shadowed. 01:35:59.570 --> 01:36:02.950 align:middle line:84% So if you can do that fast enough, then yes, 01:36:02.950 --> 01:36:03.770 align:middle line:90% you could do that. 01:36:03.770 --> 01:36:05.830 align:middle line:84% So it's not-- in certain conditions, 01:36:05.830 --> 01:36:07.270 align:middle line:90% maybe it's possible. 01:36:07.270 --> 01:36:11.290 align:middle line:84% For skin, it's very easy because the shadow color is not 01:36:11.290 --> 01:36:12.130 align:middle line:90% going to interfere. 01:36:12.130 --> 01:36:13.630 align:middle line:84% If something was black in the scene, 01:36:13.630 --> 01:36:16.210 align:middle line:84% then that could be detected as a shadow. 01:36:16.210 --> 01:36:19.420 align:middle line:84% But in this matter, even if the object is black, 01:36:19.420 --> 01:36:20.410 align:middle line:90% it doesn't matter. 01:36:20.410 --> 01:36:23.210 align:middle line:84% You'll still get very nice edges. 01:36:23.210 --> 01:36:25.630 align:middle line:84% But if you try to estimate the shadow width, 01:36:25.630 --> 01:36:27.500 align:middle line:84% then you have a little bit of [INAUDIBLE].. 01:36:27.500 --> 01:36:29.167 align:middle line:84% AUDIENCE: Again, all of these techniques 01:36:29.167 --> 01:36:31.870 align:middle line:84% assume that you have one [INAUDIBLE] image, which 01:36:31.870 --> 01:36:34.600 align:middle line:90% could have [INAUDIBLE]. 01:36:34.600 --> 01:36:35.860 align:middle line:90% You can-- then this one-- 01:36:35.860 --> 01:36:38.277 align:middle line:84% RAMESH RASKAR: Hey, now you're thinking like a researcher. 01:36:38.277 --> 01:36:40.870 align:middle line:84% Yeah, now we thought about all the ways 01:36:40.870 --> 01:36:42.845 align:middle line:84% you can change the lighting parameters. 01:36:42.845 --> 01:36:44.470 align:middle line:84% Now you're saying, in addition to that, 01:36:44.470 --> 01:36:46.720 align:middle line:84% can we change camera parameters and do something more? 01:36:46.720 --> 01:36:49.155 align:middle line:84% That's the thinking I want you to have in this class. 01:36:49.155 --> 01:36:49.780 align:middle line:90% AUDIENCE: Yeah. 01:36:49.780 --> 01:36:52.030 align:middle line:84% RAMESH RASKAR: Yeah, always take an idea, x, 01:36:52.030 --> 01:36:55.660 align:middle line:84% and think about how you can do the next x. 01:36:55.660 --> 01:36:58.090 align:middle line:84% And I posted some slides on how to come up 01:36:58.090 --> 01:37:00.760 align:middle line:84% with new ideas on the Stellar webpage. 01:37:00.760 --> 01:37:04.120 align:middle line:84% If you're inspired by an idea x, how do you 01:37:04.120 --> 01:37:05.450 align:middle line:90% come up with the next idea? 01:37:05.450 --> 01:37:08.530 align:middle line:84% And then, there's a systematic process 01:37:08.530 --> 01:37:10.870 align:middle line:84% of how you can come up with great ideas. 01:37:10.870 --> 01:37:13.820 align:middle line:84% And also, the same slides tell you, 01:37:13.820 --> 01:37:15.700 align:middle line:84% if you come up with a great idea, 01:37:15.700 --> 01:37:18.650 align:middle line:84% whether to decide whether to pursue it or not. 01:37:18.650 --> 01:37:21.970 align:middle line:84% So I'm curious about your comments. 01:37:21.970 --> 01:37:23.570 align:middle line:90% It's just a work in progress. 01:37:23.570 --> 01:37:27.340 align:middle line:84% I have been putting it together over the last several months. 01:37:27.340 --> 01:37:31.690 align:middle line:90% 01:37:31.690 --> 01:37:37.400 align:middle line:84% So again, that allows you to do this in real-time. 01:37:37.400 --> 01:37:39.850 align:middle line:84% What are some other things people are doing? 01:37:39.850 --> 01:37:42.760 align:middle line:90% You take a flash photo. 01:37:42.760 --> 01:37:44.350 align:middle line:90% The person is very brightly lit. 01:37:44.350 --> 01:37:46.720 align:middle line:84% The background is not so well-lit. 01:37:46.720 --> 01:37:50.050 align:middle line:90% You take a no-flash photo. 01:37:50.050 --> 01:37:53.244 align:middle line:90% What has changed in the two? 01:37:53.244 --> 01:37:54.119 align:middle line:90% AUDIENCE: [INAUDIBLE] 01:37:54.119 --> 01:37:55.452 align:middle line:90% RAMESH RASKAR: She didn't blink. 01:37:55.452 --> 01:37:57.570 align:middle line:84% Remember, this part of [INAUDIBLE] flash. 01:37:57.570 --> 01:38:00.120 align:middle line:84% But anyway, that's a different point. 01:38:00.120 --> 01:38:02.953 align:middle line:84% What information can you recover from this? 01:38:02.953 --> 01:38:05.370 align:middle line:84% AUDIENCE: You can separate the foreground from background. 01:38:05.370 --> 01:38:07.787 align:middle line:84% RAMESH RASKAR: You can separate foreground from background 01:38:07.787 --> 01:38:10.690 align:middle line:84% because the foreground is well-lit. 01:38:10.690 --> 01:38:13.650 align:middle line:84% But the background doesn't change. 01:38:13.650 --> 01:38:16.870 align:middle line:84% If I just take the ratio of the two images, 01:38:16.870 --> 01:38:21.370 align:middle line:84% it will look something like this if I do one minus the ratio. 01:38:21.370 --> 01:38:24.880 align:middle line:84% So the background hasn't changed. 01:38:24.880 --> 01:38:29.530 align:middle line:84% And the foreground has changed dramatically. 01:38:29.530 --> 01:38:31.900 align:middle line:84% So this particular paper is called flash mapping. 01:38:31.900 --> 01:38:32.860 align:middle line:90% As simple as that. 01:38:32.860 --> 01:38:36.468 align:middle line:84% That was the idea for a SIGGRAPH paper in 2006. 01:38:36.468 --> 01:38:38.510 align:middle line:84% They went a little bit beyond that and just said, 01:38:38.510 --> 01:38:41.890 align:middle line:84% if we just take pure ratios, then you cannot get the strands 01:38:41.890 --> 01:38:44.000 align:middle line:90% of the hair and so on. 01:38:44.000 --> 01:38:47.750 align:middle line:84% So they used some gradient domain techniques, 01:38:47.750 --> 01:38:52.643 align:middle line:84% some graphical techniques to solve this mapping equation of, 01:38:52.643 --> 01:38:54.310 align:middle line:84% what's foreground and what's background? 01:38:54.310 --> 01:38:59.220 align:middle line:84% And they can also detect these very subpixel features, 01:38:59.220 --> 01:39:01.420 align:middle line:90% strands of hair. 01:39:01.420 --> 01:39:03.520 align:middle line:84% And then, of course, you can take that 01:39:03.520 --> 01:39:05.650 align:middle line:90% and replace the background. 01:39:05.650 --> 01:39:06.910 align:middle line:90% So it was mostly well-lit. 01:39:06.910 --> 01:39:08.060 align:middle line:90% It does pretty well here. 01:39:08.060 --> 01:39:09.460 align:middle line:90% There's some small artifacts. 01:39:09.460 --> 01:39:11.950 align:middle line:90% But it looks pretty good. 01:39:11.950 --> 01:39:15.850 align:middle line:84% So just by changing the light, also, this presence and absence 01:39:15.850 --> 01:39:19.450 align:middle line:84% of light, you can create some really interesting [INAUDIBLE] 01:39:19.450 --> 01:39:20.805 align:middle line:90% or interactions. 01:39:20.805 --> 01:39:22.180 align:middle line:84% AUDIENCE: It's the same technique 01:39:22.180 --> 01:39:25.292 align:middle line:84% with a flashing [INAUDIBLE] video. 01:39:25.292 --> 01:39:26.750 align:middle line:84% RAMESH RASKAR: That's a great idea. 01:39:26.750 --> 01:39:29.050 align:middle line:84% So you could do alternate frames. 01:39:29.050 --> 01:39:32.890 align:middle line:84% And you won't-- it won't be disturbing for the user. 01:39:32.890 --> 01:39:38.310 align:middle line:84% And you can just distinguish foreground from background. 01:39:38.310 --> 01:39:39.450 align:middle line:90% So very simple examples. 01:39:39.450 --> 01:39:43.000 align:middle line:84% We'll progressively become more and more complex, I guess, 01:39:43.000 --> 01:39:43.500 align:middle line:90% here. 01:39:43.500 --> 01:39:45.330 align:middle line:90% But let's stop here for today. 01:39:45.330 --> 01:39:49.500 align:middle line:84% And we'll talk about all these other cool things next time. 01:39:49.500 --> 01:39:50.000 align:middle line:90%