WEBVTT

00:00:00.000 --> 00:00:01.992 align:middle line:90%
[SQUEAKING]

00:00:01.992 --> 00:00:03.984 align:middle line:90%
[RUSTLING]

00:00:03.984 --> 00:00:12.287 align:middle line:90%
[CLICKING]

00:00:12.287 --> 00:00:13.870 align:middle line:84%
RICHARD DE NEUFVILLE:
So there's still

00:00:13.870 --> 00:00:17.470 align:middle line:84%
the question-- we've
got an upper bound,

00:00:17.470 --> 00:00:21.200 align:middle line:84%
but we need to know
the value to its cost.

00:00:21.200 --> 00:00:22.990 align:middle line:84%
So is its expected
value information

00:00:22.990 --> 00:00:26.340 align:middle line:84%
from a test sufficiently
greater than its cost?

00:00:26.340 --> 00:00:36.300 align:middle line:84%
And so we've organized it,
we've done this analysis,

00:00:36.300 --> 00:00:38.890 align:middle line:90%
and is it worthwhile?

00:00:38.890 --> 00:00:42.000 align:middle line:84%
So clearly, if the
cost of the test

00:00:42.000 --> 00:00:45.870 align:middle line:84%
is greater than the expected
value of perfect information,

00:00:45.870 --> 00:00:47.040 align:middle line:90%
you can reject the test.

00:00:47.040 --> 00:00:48.880 align:middle line:90%
So that's a good thing.

00:00:48.880 --> 00:00:51.220 align:middle line:84%
The pragmatic rule
of sum that I've

00:00:51.220 --> 00:00:55.390 align:middle line:84%
seen by comparing some
calculations with the results

00:00:55.390 --> 00:01:00.250 align:middle line:84%
is that if the cost
is greater than 50%

00:01:00.250 --> 00:01:02.650 align:middle line:84%
of the expected value
of perfect information,

00:01:02.650 --> 00:01:04.870 align:middle line:90%
it's not worthwhile, either.

00:01:04.870 --> 00:01:06.460 align:middle line:90%
Can I document this?

00:01:06.460 --> 00:01:08.110 align:middle line:90%
No.

00:01:08.110 --> 00:01:12.580 align:middle line:84%
This is my take on it,
but it's not improbable

00:01:12.580 --> 00:01:18.160 align:middle line:84%
insofar as the real tests
are not close to perfect.

00:01:18.160 --> 00:01:19.150 align:middle line:90%
We know that.

00:01:19.150 --> 00:01:23.887 align:middle line:84%
So we can then think
about, all right, maybe we

00:01:23.887 --> 00:01:24.970 align:middle line:90%
ought to reject it anyway.

00:01:24.970 --> 00:01:28.420 align:middle line:84%
So we create, if
you want, an area

00:01:28.420 --> 00:01:30.410 align:middle line:90%
where at least it's plausible.

00:01:30.410 --> 00:01:35.780 align:middle line:84%
Now if you're in that range,
and it's a big decision--

00:01:35.780 --> 00:01:39.850 align:middle line:84%
so if I was consulting to
you, for example, Robert,

00:01:39.850 --> 00:01:41.740 align:middle line:84%
I might say, well,
look-- calculate this.

00:01:41.740 --> 00:01:43.180 align:middle line:90%
It's really easy to do.

00:01:43.180 --> 00:01:45.790 align:middle line:84%
See if you're in
a plausible range.

00:01:45.790 --> 00:01:48.460 align:middle line:84%
If you're in a
plausible range, then it

00:01:48.460 --> 00:01:50.380 align:middle line:84%
might be worthwhile
to go through a more

00:01:50.380 --> 00:01:54.510 align:middle line:84%
detailed analysis to
see if it's worthwhile.

00:01:54.510 --> 00:01:58.490 align:middle line:90%
But otherwise, don't bother.

00:01:58.490 --> 00:02:03.010 align:middle line:84%
So I think that you
create a plausible range.

00:02:03.010 --> 00:02:05.410 align:middle line:84%
Now, you might say,
well, that's too tough.

00:02:05.410 --> 00:02:07.630 align:middle line:84%
Maybe it should
be less than 75%,

00:02:07.630 --> 00:02:09.250 align:middle line:90%
or maybe it's not tough enough.

00:02:09.250 --> 00:02:12.460 align:middle line:84%
But the point is, you
can use this device

00:02:12.460 --> 00:02:17.890 align:middle line:84%
to show whether the proposal
passes what some people call

00:02:17.890 --> 00:02:18.700 align:middle line:90%
the "giggle test."

00:02:18.700 --> 00:02:22.330 align:middle line:84%
Is it in the right range,
or it's just too far out?

00:02:22.330 --> 00:02:24.910 align:middle line:90%


00:02:24.910 --> 00:02:27.040 align:middle line:84%
It'd be too close to
being perfect information

00:02:27.040 --> 00:02:29.960 align:middle line:84%
to be credible or it's
beyond perfect information.

00:02:29.960 --> 00:02:33.460 align:middle line:84%
So you establish a
range of possibilities

00:02:33.460 --> 00:02:35.110 align:middle line:90%
for which you have discussion.

00:02:35.110 --> 00:02:38.420 align:middle line:84%
And you could then augment
that with more detail.

00:02:38.420 --> 00:02:43.720 align:middle line:84%
But at this moment, it gives
you a quick and dirty way

00:02:43.720 --> 00:02:46.030 align:middle line:84%
of assessing whether
or not you should even

00:02:46.030 --> 00:02:48.850 align:middle line:90%
consider the argument.

00:02:48.850 --> 00:02:53.350 align:middle line:84%
Defines the ballpark you can be
in, and the area we can be in,

00:02:53.350 --> 00:02:55.970 align:middle line:90%
and go with that.

00:02:55.970 --> 00:03:00.070 align:middle line:90%
That's what I'm talking about.

00:03:00.070 --> 00:03:08.070 align:middle line:84%
Now, let me zoom out of this and
think about the whole approach.

00:03:08.070 --> 00:03:13.190 align:middle line:84%
So the decision to
collect information

00:03:13.190 --> 00:03:16.550 align:middle line:84%
is basically a decision
to-- implies the decision

00:03:16.550 --> 00:03:20.280 align:middle line:84%
to insert flexibility into
your development strategy.

00:03:20.280 --> 00:03:22.745 align:middle line:84%
In other words,
the moment you say,

00:03:22.745 --> 00:03:25.370 align:middle line:90%
hmm, we don't really know this.

00:03:25.370 --> 00:03:27.260 align:middle line:84%
We've recognized
this uncertainty,

00:03:27.260 --> 00:03:28.760 align:middle line:84%
but we don't have
to live with them.

00:03:28.760 --> 00:03:30.620 align:middle line:84%
We should explore
them a bit, figure out

00:03:30.620 --> 00:03:31.940 align:middle line:90%
what might be happening here.

00:03:31.940 --> 00:03:34.020 align:middle line:90%
Should we have this test or not?

00:03:34.020 --> 00:03:37.760 align:middle line:84%
And the rationale for that is,
well, if we find out things

00:03:37.760 --> 00:03:40.430 align:middle line:84%
are going to be different
than we imagined,

00:03:40.430 --> 00:03:42.980 align:middle line:90%
we should change our results.

00:03:42.980 --> 00:03:45.530 align:middle line:84%
So the value of
the information is

00:03:45.530 --> 00:03:49.860 align:middle line:84%
a way of valuing the flexibility
that you might come up with.

00:03:49.860 --> 00:03:54.050 align:middle line:84%
So the practical
lesson, in a way,

00:03:54.050 --> 00:03:57.310 align:middle line:84%
is we need to
recognize uncertainty.

00:03:57.310 --> 00:03:59.140 align:middle line:84%
That was the first
message of this course,

00:03:59.140 --> 00:04:03.340 align:middle line:84%
because the uncertainty shows
that the value of a project

00:04:03.340 --> 00:04:07.900 align:middle line:84%
is different than it would be
under the deterministic fairy

00:04:07.900 --> 00:04:10.810 align:middle line:84%
tale analysis, so you
should recognize it.

00:04:10.810 --> 00:04:14.350 align:middle line:84%
You should recognize it because
it will give you some other

00:04:14.350 --> 00:04:18.670 align:middle line:84%
possibilities, and that
the better design might be

00:04:18.670 --> 00:04:22.089 align:middle line:84%
different from-- not just
less or different from--

00:04:22.089 --> 00:04:24.310 align:middle line:84%
the calculations of
deterministic numbers.

00:04:24.310 --> 00:04:27.380 align:middle line:84%
It might be a different
result entirely.

00:04:27.380 --> 00:04:30.080 align:middle line:84%
But you should then go
further to say, well,

00:04:30.080 --> 00:04:33.830 align:middle line:84%
having recognized the
uncertainty in general,

00:04:33.830 --> 00:04:38.390 align:middle line:84%
let's think about some ways
to build into our design

00:04:38.390 --> 00:04:42.650 align:middle line:84%
and management process some
way of gathering information--

00:04:42.650 --> 00:04:47.960 align:middle line:84%
that can be expensive, or not
so expensive, or whatever--

00:04:47.960 --> 00:04:52.070 align:middle line:84%
to see what we should do
and how we should do it.

00:04:52.070 --> 00:04:55.040 align:middle line:84%
And use that as part
of your strategy

00:04:55.040 --> 00:04:58.280 align:middle line:84%
to have a flexible strategy
to respond to the situations

00:04:58.280 --> 00:05:01.240 align:middle line:90%
as they develop.

00:05:01.240 --> 00:05:06.060 align:middle line:84%
So the overall takeaway,
now, is that the information

00:05:06.060 --> 00:05:12.510 align:middle line:84%
can increase the value
significantly of a project.

00:05:12.510 --> 00:05:14.820 align:middle line:84%
And you need to estimate
the expected value

00:05:14.820 --> 00:05:18.690 align:middle line:84%
of this information in terms
of expected improved value

00:05:18.690 --> 00:05:22.280 align:middle line:90%
of your optimal strategies.

00:05:22.280 --> 00:05:24.560 align:middle line:84%
The expected value of
perfect information

00:05:24.560 --> 00:05:27.770 align:middle line:84%
is a way to establish
an upper bound on this.

00:05:27.770 --> 00:05:33.170 align:middle line:84%
It is attractive because it's
really very simple to do.

00:05:33.170 --> 00:05:40.610 align:middle line:84%
And it tells you whether
you're in the right area,

00:05:40.610 --> 00:05:42.860 align:middle line:84%
the right ballpark, whether
it passes the giggle

00:05:42.860 --> 00:05:47.670 align:middle line:84%
test, other shorthand
ways to describe it.

00:05:47.670 --> 00:05:52.070 align:middle line:84%
So part of the
information collection

00:05:52.070 --> 00:05:57.110 align:middle line:84%
is one way to begin to insert
the flexibility in a system

00:05:57.110 --> 00:05:59.520 align:middle line:90%
design and development process.

00:05:59.520 --> 00:06:01.840 align:middle line:90%
So that's the main takeaway.

00:06:01.840 --> 00:06:07.000 align:middle line:90%