WEBVTT 00:00:00.000 --> 00:00:03.864 align:middle line:84% [SQUEAKING] [RUSTLING] [CLICKING] 00:00:03.864 --> 00:00:12.090 align:middle line:90% 00:00:12.090 --> 00:00:14.400 align:middle line:84% RICHARD DE NEUFVILLE: What can we expect to achieve. 00:00:14.400 --> 00:00:17.430 align:middle line:84% So the evaluation has many dimensions. 00:00:17.430 --> 00:00:18.390 align:middle line:90% It is uncertainty. 00:00:18.390 --> 00:00:19.440 align:middle line:90% Best is not defined. 00:00:19.440 --> 00:00:23.580 align:middle line:84% What we can do first is to dominate, to screen out 00:00:23.580 --> 00:00:24.810 align:middle line:90% the dominate solutions. 00:00:24.810 --> 00:00:27.420 align:middle line:84% As we can go through a process and say, 00:00:27.420 --> 00:00:31.050 align:middle line:84% of the 1,000 different solutions or the 10 different solutions, 00:00:31.050 --> 00:00:33.450 align:middle line:84% eight of them, for example, are of good. 00:00:33.450 --> 00:00:36.270 align:middle line:84% And there is a dominant three or four or five 00:00:36.270 --> 00:00:38.790 align:middle line:90% that are the dominant ones. 00:00:38.790 --> 00:00:40.240 align:middle line:90% I'll give it a graph of this. 00:00:40.240 --> 00:00:43.530 align:middle line:84% And then, we want to recognize that if we're 00:00:43.530 --> 00:00:45.030 align:middle line:84% talking about individuals, they must 00:00:45.030 --> 00:00:47.820 align:middle line:84% have a trade-off in some way between risk and reward, 00:00:47.820 --> 00:00:49.620 align:middle line:90% is one way to think about it. 00:00:49.620 --> 00:00:53.340 align:middle line:84% And groups basically have to negotiate in some way 00:00:53.340 --> 00:00:56.730 align:middle line:90% a deal to get things going. 00:00:56.730 --> 00:01:00.090 align:middle line:84% You might say that the factory owners 00:01:00.090 --> 00:01:02.530 align:middle line:90% are the bosses that rule. 00:01:02.530 --> 00:01:05.970 align:middle line:84% But as you know from experience that the workers have 00:01:05.970 --> 00:01:08.400 align:middle line:84% something to say, they may not have that much power. 00:01:08.400 --> 00:01:11.370 align:middle line:84% But if they go on strike, the factory owners just 00:01:11.370 --> 00:01:12.820 align:middle line:90% can't tell them everything. 00:01:12.820 --> 00:01:15.960 align:middle line:84% So there is, in some ways, whether it's explicitly 00:01:15.960 --> 00:01:17.970 align:middle line:84% or implicitly, there is typically 00:01:17.970 --> 00:01:22.710 align:middle line:84% some kind of negotiation between what individuals might prefer 00:01:22.710 --> 00:01:27.140 align:middle line:84% or groups might prefer to what they actually get. 00:01:27.140 --> 00:01:32.660 align:middle line:84% So the first thing to look at is this concept of dominance. 00:01:32.660 --> 00:01:36.950 align:middle line:84% That is, if we look at two axes of benefits, 00:01:36.950 --> 00:01:40.620 align:middle line:84% we can think of a locus or the dominant ones. 00:01:40.620 --> 00:01:49.630 align:middle line:84% There are various solutions that represent an amount of one axis 00:01:49.630 --> 00:01:50.900 align:middle line:90% and one or the other one. 00:01:50.900 --> 00:01:54.130 align:middle line:84% And there are all kinds of other possible solutions interior 00:01:54.130 --> 00:01:58.390 align:middle line:84% to this space so that this point where my arrow is, right 00:01:58.390 --> 00:02:01.510 align:middle line:84% at the point of the arrow, we could agree there are solutions 00:02:01.510 --> 00:02:04.360 align:middle line:90% that are better on all counts. 00:02:04.360 --> 00:02:08.139 align:middle line:84% That is, this particular xy point, 00:02:08.139 --> 00:02:11.170 align:middle line:84% there are points which have the same y, 00:02:11.170 --> 00:02:13.780 align:middle line:84% but better x or the same x and better y. 00:02:13.780 --> 00:02:16.180 align:middle line:84% And so that they are better solutions. 00:02:16.180 --> 00:02:18.310 align:middle line:84% So the ones in here are inferior. 00:02:18.310 --> 00:02:19.850 align:middle line:84% And we don't want to consider them. 00:02:19.850 --> 00:02:22.030 align:middle line:84% So the first thing to do is to try 00:02:22.030 --> 00:02:24.640 align:middle line:90% to focus on the dominated data. 00:02:24.640 --> 00:02:27.310 align:middle line:84% Get out of the dominated alternatives 00:02:27.310 --> 00:02:29.470 align:middle line:84% and look for the dominant ones, the ones that 00:02:29.470 --> 00:02:32.210 align:middle line:90% are on the frontier. 00:02:32.210 --> 00:02:37.280 align:middle line:84% And I would quickly say that expected value by itself 00:02:37.280 --> 00:02:39.680 align:middle line:90% is just one measure. 00:02:39.680 --> 00:02:41.310 align:middle line:90% And we've emphasized that. 00:02:41.310 --> 00:02:46.220 align:middle line:84% So for implicitly emphasized it as a meaningful way 00:02:46.220 --> 00:02:48.530 align:middle line:90% to talk about distributions. 00:02:48.530 --> 00:02:51.770 align:middle line:84% But it's only one of the possible ones. 00:02:51.770 --> 00:02:54.090 align:middle line:90% And we need to go beyond that. 00:02:54.090 --> 00:02:58.380 align:middle line:84% So other dimensions are, for example, 00:02:58.380 --> 00:02:59.960 align:middle line:90% the worst that could happen. 00:02:59.960 --> 00:03:02.690 align:middle line:84% And people are typically risk averse, 00:03:02.690 --> 00:03:09.500 align:middle line:84% don't like it and sensitive loss so that they are looking 00:03:09.500 --> 00:03:11.390 align:middle line:90% at the good value on average. 00:03:11.390 --> 00:03:14.750 align:middle line:84% But maybe consider about the downside. 00:03:14.750 --> 00:03:20.540 align:middle line:84% On the other hand, you may also be considering the upside. 00:03:20.540 --> 00:03:23.570 align:middle line:84% So that you know most startups will fail. 00:03:23.570 --> 00:03:27.170 align:middle line:84% But the reason you're in them is because if you hit it big, 00:03:27.170 --> 00:03:30.320 align:middle line:84% it'd be life-changing, and therefore, 00:03:30.320 --> 00:03:32.960 align:middle line:84% although the expected value for a startup 00:03:32.960 --> 00:03:38.048 align:middle line:84% is not positive, that it might make a difference. 00:03:38.048 --> 00:03:39.090 align:middle line:90% And that's why you do it. 00:03:39.090 --> 00:03:41.330 align:middle line:90% So the downside, the upside. 00:03:41.330 --> 00:03:43.610 align:middle line:84% The capital expenditure may be important. 00:03:43.610 --> 00:03:46.490 align:middle line:84% How much do I have to put in to get this? 00:03:46.490 --> 00:03:49.190 align:middle line:84% You might have some measure of benefit cost. 00:03:49.190 --> 00:03:52.940 align:middle line:84% I'm simply saying that there are many dimensions 00:03:52.940 --> 00:03:54.500 align:middle line:90% besides an expected value. 00:03:54.500 --> 00:03:57.020 align:middle line:90% 00:03:57.020 --> 00:03:59.530 align:middle line:84% One of them is it's a very common one 00:03:59.530 --> 00:04:01.990 align:middle line:84% you should know is this notion of value at risk. 00:04:01.990 --> 00:04:07.390 align:middle line:84% And it reflects in a standard way the lowest 5% 00:04:07.390 --> 00:04:11.230 align:middle line:84% or 10% of a distribution, the tails of distribution, 00:04:11.230 --> 00:04:15.010 align:middle line:84% and is a standard concept in finance that they will often, 00:04:15.010 --> 00:04:17.745 align:middle line:84% banks and others will be interested in saying, well, 00:04:17.745 --> 00:04:19.120 align:middle line:84% what's the worst going to happen? 00:04:19.120 --> 00:04:21.610 align:middle line:84% How probable it is that this company is 00:04:21.610 --> 00:04:23.987 align:middle line:84% going to have a big loss or this borrower 00:04:23.987 --> 00:04:25.570 align:middle line:84% is going to have a big loss and is not 00:04:25.570 --> 00:04:26.890 align:middle line:90% going to be able to repay us. 00:04:26.890 --> 00:04:31.090 align:middle line:84% So they are interested, not only how good it can get, 00:04:31.090 --> 00:04:33.320 align:middle line:84% that's fine, but they want to get their money back. 00:04:33.320 --> 00:04:35.930 align:middle line:84% And they're concerned with what's the downside. 00:04:35.930 --> 00:04:38.440 align:middle line:84% So they might want to have this figure 00:04:38.440 --> 00:04:43.890 align:middle line:84% of what's the value at risk, either at the 5% or 10% level. 00:04:43.890 --> 00:04:46.650 align:middle line:84% Conversely, you can talk about the value in gain. 00:04:46.650 --> 00:04:51.730 align:middle line:84% This is not a well-established notion but it's important. 00:04:51.730 --> 00:04:54.990 align:middle line:84% In fact, my colleagues and I have set this up 00:04:54.990 --> 00:04:57.120 align:middle line:90% as a counterpart. 00:04:57.120 --> 00:05:01.570 align:middle line:84% It's sort of as a way of balancing off the equation, 00:05:01.570 --> 00:05:02.210 align:middle line:90% so to speak. 00:05:02.210 --> 00:05:04.710 align:middle line:84% And, of course, the investors are mostly interested in that. 00:05:04.710 --> 00:05:06.420 align:middle line:90% Why am I going to a startup? 00:05:06.420 --> 00:05:08.655 align:middle line:84% Well, because it might pay off really well. 00:05:08.655 --> 00:05:11.730 align:middle line:90% 00:05:11.730 --> 00:05:14.280 align:middle line:84% Then, the notion that we use a law that you've seen it, 00:05:14.280 --> 00:05:15.900 align:middle line:84% and you've seen it in the garage case, 00:05:15.900 --> 00:05:18.280 align:middle line:84% is this notion of a target curve. 00:05:18.280 --> 00:05:21.150 align:middle line:84% Says the distribution of outcomes. 00:05:21.150 --> 00:05:24.480 align:middle line:84% I particularly like the cumulative distribution, 00:05:24.480 --> 00:05:26.910 align:middle line:84% but some others like a frequency distribution. 00:05:26.910 --> 00:05:28.860 align:middle line:90% But it's the distribution. 00:05:28.860 --> 00:05:33.460 align:middle line:84% And it goes from the worst case to the best case. 00:05:33.460 --> 00:05:37.320 align:middle line:84% And you can see, read from it the value at risk 00:05:37.320 --> 00:05:38.590 align:middle line:90% and the value in gain. 00:05:38.590 --> 00:05:41.640 align:middle line:84% So if we looked at these curves, which 00:05:41.640 --> 00:05:44.970 align:middle line:84% were done in conjunction with some studies 00:05:44.970 --> 00:05:51.150 align:middle line:84% from BP by Jijun Lin when he did his PhD with us, that here 00:05:51.150 --> 00:05:54.840 align:middle line:84% at the cumulative probability and the benefits 00:05:54.840 --> 00:05:58.710 align:middle line:84% of this project and billions of dollars, as it was, 00:05:58.710 --> 00:06:01.140 align:middle line:84% it was fun to play with that kind of money, 00:06:01.140 --> 00:06:09.720 align:middle line:84% they value at risk at 10% level was a net PV of about a million 00:06:09.720 --> 00:06:13.950 align:middle line:84% for the original design, a million-plus 00:06:13.950 --> 00:06:18.220 align:middle line:90% for that 5% level it was a loss. 00:06:18.220 --> 00:06:26.750 align:middle line:84% And so that you can read off those extremes fairly easily. 00:06:26.750 --> 00:06:28.540 align:middle line:90% Now, small question. 00:06:28.540 --> 00:06:33.010 align:middle line:84% If the target curve is always to the right of another, 00:06:33.010 --> 00:06:34.960 align:middle line:90% does it dominate? 00:06:34.960 --> 00:06:40.990 align:middle line:84% That is, if you have this black line, which 00:06:40.990 --> 00:06:43.870 align:middle line:84% represents one design, it's always to the right-- 00:06:43.870 --> 00:06:48.220 align:middle line:84% it is in the positive direction to the green line-- 00:06:48.220 --> 00:06:53.230 align:middle line:84% does that mean that a design represented by the black line 00:06:53.230 --> 00:06:55.810 align:middle line:84% will always be better than the design 00:06:55.810 --> 00:06:58.180 align:middle line:90% represented by the green line? 00:06:58.180 --> 00:07:01.570 align:middle line:84% Now, the obvious reaction is, well, yeah. 00:07:01.570 --> 00:07:04.450 align:middle line:84% It's always to the right so how could it not be? 00:07:04.450 --> 00:07:07.300 align:middle line:84% The thing is that the situation that 00:07:07.300 --> 00:07:10.360 align:middle line:84% may be bad for this design, that is, 00:07:10.360 --> 00:07:13.540 align:middle line:84% that occur for these low returns here, 00:07:13.540 --> 00:07:16.280 align:middle line:84% may be exactly what's good for the other one. 00:07:16.280 --> 00:07:21.910 align:middle line:84% So that although overall the distribution for the green line 00:07:21.910 --> 00:07:24.460 align:middle line:84% is inferior to that for the black line, 00:07:24.460 --> 00:07:29.410 align:middle line:84% they are cases where the bad performance of this design 00:07:29.410 --> 00:07:31.960 align:middle line:84% are the situation where there's a good performance 00:07:31.960 --> 00:07:33.070 align:middle line:90% for another line. 00:07:33.070 --> 00:07:38.100 align:middle line:84% You can imagine it easily that you're having a gold mine, 00:07:38.100 --> 00:07:40.850 align:middle line:84% and you're digging in one direction or another. 00:07:40.850 --> 00:07:44.950 align:middle line:84% And it could happen that the choice that 00:07:44.950 --> 00:07:49.840 align:middle line:84% was great for one case was the bad one for the other case. 00:07:49.840 --> 00:07:51.400 align:middle line:90% So it's possible. 00:07:51.400 --> 00:07:55.080 align:middle line:90% Just a subtlety to alert you to. 00:07:55.080 --> 00:08:00.000 align:middle line:90%