WEBVTT

00:00:01.680 --> 00:00:04.080
The following content is
provided under a Creative

00:00:04.080 --> 00:00:05.620
Commons license.

00:00:05.620 --> 00:00:07.920
Your support will help
MIT OpenCourseWare

00:00:07.920 --> 00:00:12.280
continue to offer high-quality
educational resources for free.

00:00:12.280 --> 00:00:14.910
To make a donation or
view additional materials

00:00:14.910 --> 00:00:18.870
from hundreds of MIT courses,
visit MIT OpenCourseWare

00:00:18.870 --> 00:00:21.814
at ocw.mit.edu.

00:00:21.814 --> 00:00:23.480
PATRICK HENRY WINSTON:
Oh, how to start.

00:00:23.480 --> 00:00:27.520
I have been on the Navy Science
Board for a couple of decades.

00:00:27.520 --> 00:00:30.180
And so as a consequence,
I've had an opportunity

00:00:30.180 --> 00:00:35.920
to spend two weeks for a
couple of decades in San Diego.

00:00:35.920 --> 00:00:38.230
And the first thing I do
when I get to San Diego

00:00:38.230 --> 00:00:43.030
is I go to the zoo, and
I look at the orangutans

00:00:43.030 --> 00:00:49.560
and ask myself, how come I'm
out here and they're in there?

00:00:49.560 --> 00:00:54.390
How come you're not all covered
with orange hair instead

00:00:54.390 --> 00:00:56.640
of hardly any hair at all?

00:00:59.800 --> 00:01:05.680
Well, my answer to that is that
we can tell stories and they

00:01:05.680 --> 00:01:07.180
can't.

00:01:07.180 --> 00:01:08.720
So this is the
center of my talk.

00:01:13.700 --> 00:01:20.450
That's what I have come to
believe after a long time.

00:01:20.450 --> 00:01:22.780
So I'm going to be
talking about stories

00:01:22.780 --> 00:01:25.960
and to give you a
preview of where

00:01:25.960 --> 00:01:28.990
I'm going to go with this,
I like to just show you

00:01:28.990 --> 00:01:32.190
some of the questions
that I propose to address

00:01:32.190 --> 00:01:34.630
in the course of the next hour.

00:01:34.630 --> 00:01:36.920
I want to start talking
about this by way of history.

00:01:36.920 --> 00:01:38.420
I've been in
artificial intelligence

00:01:38.420 --> 00:01:41.650
almost since the
beginning, and so

00:01:41.650 --> 00:01:43.220
I was a student
of Marvin Minsky,

00:01:43.220 --> 00:01:46.540
and it's interesting that the
field started a long time ago--

00:01:46.540 --> 00:01:51.850
55 years ago-- with perhaps
the most important paper

00:01:51.850 --> 00:01:56.680
in the field, Steps Toward
Artificial Intelligence.

00:01:56.680 --> 00:02:01.510
It was about that time that
the first intelligent program

00:02:01.510 --> 00:02:02.440
was written.

00:02:02.440 --> 00:02:04.690
It was a program that
could do calculus problems

00:02:04.690 --> 00:02:08.530
as well as an MIT freshman--
a very good MIT freshman.

00:02:08.530 --> 00:02:11.470
That was done in 1961.

00:02:11.470 --> 00:02:16.600
And those programs led to a half
century of enormous progress

00:02:16.600 --> 00:02:18.310
in the field.

00:02:18.310 --> 00:02:21.490
All sorts of ideas and
subfields like machine learning

00:02:21.490 --> 00:02:22.240
were spawned.

00:02:22.240 --> 00:02:24.390
Useful applications.

00:02:24.390 --> 00:02:26.350
But as you know, there's
been an explosion

00:02:26.350 --> 00:02:28.370
in these useful applications
in recent years.

00:02:28.370 --> 00:02:29.827
I've stolen this
slide from Tomaso

00:02:29.827 --> 00:02:31.910
but I added a couple of
things that I particularly

00:02:31.910 --> 00:02:35.710
think are of special
importance like Echo.

00:02:35.710 --> 00:02:38.860
You all know about Amazon
Echo, of course, right?

00:02:38.860 --> 00:02:40.060
I swear, it astonishes me.

00:02:40.060 --> 00:02:41.950
Many people don't know
about Amazon Echo.

00:02:41.950 --> 00:02:45.640
It's Siri in a beer can.

00:02:45.640 --> 00:02:48.130
It's a Siri that you can
talk to across the room

00:02:48.130 --> 00:02:50.680
and say how long should
I boil a hard boiled egg

00:02:50.680 --> 00:02:52.390
or I've fallen down.

00:02:52.390 --> 00:02:54.430
Please call for an ambulance.

00:02:54.430 --> 00:02:55.870
So it's a wonderful thing.

00:02:55.870 --> 00:02:57.820
I think most people
don't know about it

00:02:57.820 --> 00:02:59.352
because of privacy concerns.

00:02:59.352 --> 00:03:01.810
Having something listening to
you all the time in your home

00:03:01.810 --> 00:03:04.152
may not be the most
comfortable kind of thing.

00:03:04.152 --> 00:03:05.110
But anyhow, it's there.

00:03:07.950 --> 00:03:10.390
So this has caused quite a
lot of interest in the field

00:03:10.390 --> 00:03:11.380
lately.

00:03:11.380 --> 00:03:15.467
Boris has just talked
about Siri and Jeopardy,

00:03:15.467 --> 00:03:17.800
but maybe the thing that's
astonished the world the most

00:03:17.800 --> 00:03:20.290
is this captioning program
I think you've probably seen

00:03:20.290 --> 00:03:24.670
before the last week or two.

00:03:24.670 --> 00:03:27.370
And man, I don't know
what to think of this.

00:03:27.370 --> 00:03:29.260
I don't know why it's
created such a stir,

00:03:29.260 --> 00:03:32.500
except that that
caption makes it

00:03:32.500 --> 00:03:36.300
look like that system knows a
lot that it doesn't actually

00:03:36.300 --> 00:03:37.720
know.

00:03:37.720 --> 00:03:41.260
First of all, it's
trained on still photos.

00:03:41.260 --> 00:03:43.960
So it doesn't know about motion.

00:03:43.960 --> 00:03:46.150
It doesn't know about play.

00:03:46.150 --> 00:03:48.460
It doesn't know what
it means to be young.

00:03:48.460 --> 00:03:50.560
It would probably
say the same thing

00:03:50.560 --> 00:03:54.520
if we replaced those
faces with geriatrics.

00:03:54.520 --> 00:03:59.440
Yet when we see that, we
presume that it knows a lot.

00:03:59.440 --> 00:04:01.840
It's sort of
parasitic semantics.

00:04:01.840 --> 00:04:04.210
The intelligence is coming
from our interpretation

00:04:04.210 --> 00:04:07.364
of the sentence, not from
it producing the sentence.

00:04:07.364 --> 00:04:09.280
So yeah, it's a great
engineering achievement,

00:04:09.280 --> 00:04:12.830
but it's not as
smart as it looks.

00:04:12.830 --> 00:04:14.470
And of course, as
you know, there's

00:04:14.470 --> 00:04:18.940
been a whole industry
of fooling papers.

00:04:18.940 --> 00:04:22.690
Here's a fooling example.

00:04:22.690 --> 00:04:26.800
One of those is considered
by a standard deep neural net

00:04:26.800 --> 00:04:28.810
to be a school bus,
and the other is not

00:04:28.810 --> 00:04:30.160
considered to be a school bus.

00:04:30.160 --> 00:04:33.940
And just a few pixels
have been changed.

00:04:33.940 --> 00:04:35.210
Imperceptible to us.

00:04:35.210 --> 00:04:39.809
It still looks like a school bus
but not to the deep neural net.

00:04:39.809 --> 00:04:42.100
And of course, these things
are considered school buses

00:04:42.100 --> 00:04:44.837
in another fooling paper.

00:04:44.837 --> 00:04:47.170
Why in the world would those
be considered school buses?

00:04:47.170 --> 00:04:50.410
Presumably because of
some sharing of texture.

00:04:54.360 --> 00:04:56.580
Well, anyhow, people
have gotten all excited

00:04:56.580 --> 00:05:00.850
and especially Elon Musk
has gotten all excited.

00:05:00.850 --> 00:05:03.540
We are summoning the demon.

00:05:03.540 --> 00:05:07.230
So Musk said that in
an off-the-cuff answer

00:05:07.230 --> 00:05:14.410
to a question at the, let's
see, an anniversary of the MIT

00:05:14.410 --> 00:05:16.190
AeroAstro Department.

00:05:16.190 --> 00:05:18.130
Someone asked him if he
was interested in AI,

00:05:18.130 --> 00:05:22.150
and that's his
off-the-cuff response.

00:05:22.150 --> 00:05:24.700
So it's interesting
that those of us

00:05:24.700 --> 00:05:27.520
who've been around for a while
find this beyond interesting.

00:05:27.520 --> 00:05:32.110
It's curious, because a
long time ago philosophers

00:05:32.110 --> 00:05:36.774
like Hubert Dreyfus were saying
that it was not possible.

00:05:36.774 --> 00:05:39.190
And so now we've shifted from
not possible to the scariest

00:05:39.190 --> 00:05:41.820
thing imaginable.

00:05:41.820 --> 00:05:44.722
We can't stop ourselves
from chuckling.

00:05:49.150 --> 00:05:54.679
Well, Dreyfus must
know that he's wrong,

00:05:54.679 --> 00:05:55.720
but maybe he's right too.

00:05:55.720 --> 00:05:58.220
What we really had to wait for
for all of these achievements

00:05:58.220 --> 00:06:04.104
was massive amounts
of computing.

00:06:09.050 --> 00:06:11.410
So I'd like to go a little
further back in history

00:06:11.410 --> 00:06:14.410
while I'm talking about
history, and the next thing back

00:06:14.410 --> 00:06:21.730
there is 65 years ago Turing's
paper on machine intelligence.

00:06:21.730 --> 00:06:25.060
It's interesting that that paper
is widely assumed to be about

00:06:25.060 --> 00:06:28.120
the Turing test and it isn't.

00:06:28.120 --> 00:06:31.060
If you look at the actual
paper content, what you find

00:06:31.060 --> 00:06:34.180
is that only a couple of pages
were devoted to the test.

00:06:34.180 --> 00:06:36.160
Most of it was
devoted to discussion

00:06:36.160 --> 00:06:39.250
of arguments for and
against the possibility

00:06:39.250 --> 00:06:41.320
of artificial intelligence.

00:06:41.320 --> 00:06:44.444
I've read that paper 20
times, because I prescribe it

00:06:44.444 --> 00:06:46.360
in my course, so I have
to read it every year.

00:06:46.360 --> 00:06:49.174
And every time I read it,
I become more convinced

00:06:49.174 --> 00:06:50.590
that what Turing
was talking about

00:06:50.590 --> 00:06:54.880
is arguments against
AI, not about the tests.

00:06:54.880 --> 00:06:56.620
So why the test?

00:06:56.620 --> 00:06:58.630
Well, he's a mathematician
and a philosopher.

00:06:58.630 --> 00:07:00.490
And no mathematician
or philosopher

00:07:00.490 --> 00:07:03.380
would write a paper without
defining their terms.

00:07:03.380 --> 00:07:06.730
So he squeezed into
some kind of definition.

00:07:06.730 --> 00:07:09.481
Counterarguments took
up a lot of space,

00:07:09.481 --> 00:07:10.480
but they didn't have to.

00:07:10.480 --> 00:07:13.150
If Turing had taken
Marvin Minsky's course

00:07:13.150 --> 00:07:14.920
in the last couple
of years, he wouldn't

00:07:14.920 --> 00:07:19.450
have bothered with that test,
because Minsky introduced

00:07:19.450 --> 00:07:22.210
the notion of a suitcase word.

00:07:22.210 --> 00:07:24.459
That's a word--
he likes that term

00:07:24.459 --> 00:07:26.500
because what he means by
that is that the word is

00:07:26.500 --> 00:07:30.670
so big, like a big suitcase,
you can stuff anything into it.

00:07:30.670 --> 00:07:34.780
So for me, this has been a great
thought because, if you ask me,

00:07:34.780 --> 00:07:37.850
is Watson intelligent?

00:07:37.850 --> 00:07:40.120
Is the Jeopardy-playing
system intelligent?

00:07:40.120 --> 00:07:42.910
My immediate response is, sure.

00:07:42.910 --> 00:07:44.800
It has a kind of intelligence.

00:07:44.800 --> 00:07:47.470
As Boris points out, it's not
every kind of intelligence,

00:07:47.470 --> 00:07:49.471
and it doesn't think
like we do, and there

00:07:49.471 --> 00:07:51.220
are some kinds of
thinking that it doesn't

00:07:51.220 --> 00:07:53.660
do that we do quite handily.

00:07:53.660 --> 00:07:55.950
But it's silly to argue about
whether it's intelligent

00:07:55.950 --> 00:07:56.450
or not.

00:07:56.450 --> 00:08:00.073
It has aspects, some
kinds of intelligence.

00:08:04.150 --> 00:08:09.280
So what Turing really
did was establish

00:08:09.280 --> 00:08:13.604
that this is something that
serious people can think about

00:08:13.604 --> 00:08:15.520
and suggests that there's
no reason to believe

00:08:15.520 --> 00:08:16.728
that it won't happen someday.

00:08:20.071 --> 00:08:23.870
And he centered that whole paper
on these kind of arguments,

00:08:23.870 --> 00:08:25.530
to the arguments against AI.

00:08:28.740 --> 00:08:30.630
It's fun to talk about those.

00:08:30.630 --> 00:08:32.490
Each of them deserves attention.

00:08:32.490 --> 00:08:36.600
I'll just say a word or two
about number four there,

00:08:36.600 --> 00:08:39.630
Lady Lovelace's objection.

00:08:39.630 --> 00:08:44.119
She was, as you all know,
the sponsor and programmer

00:08:44.119 --> 00:08:46.410
of Charles Babbage when he
attempted to make a computer

00:08:46.410 --> 00:08:48.200
mechanically.

00:08:48.200 --> 00:08:51.054
And what she said,
she was obviously

00:08:51.054 --> 00:08:53.470
pestered with the same kind
of things that everybody in AI

00:08:53.470 --> 00:08:55.000
is always pestered with.

00:08:55.000 --> 00:08:58.620
And at one point, she was
reported to have said--

00:08:58.620 --> 00:09:00.807
let me put it in my patois.

00:09:00.807 --> 00:09:01.890
Don't worry about a thing.

00:09:01.890 --> 00:09:03.848
They can only do what
they're programmed to do.

00:09:06.410 --> 00:09:08.300
And of course, what
she should have said

00:09:08.300 --> 00:09:11.900
is that they can only do what
they've been programmed to do

00:09:11.900 --> 00:09:14.720
and what we've taught them
to do and what they've

00:09:14.720 --> 00:09:17.474
learned how to do on their own.

00:09:17.474 --> 00:09:19.640
But maybe that wouldn't
have had the soothing effect

00:09:19.640 --> 00:09:22.528
she was looking for.

00:09:22.528 --> 00:09:27.390
In any event, this is when
people started thinking about

00:09:27.390 --> 00:09:30.590
whether computers could think.

00:09:30.590 --> 00:09:33.860
But it's not the first time
people thought about thinking,

00:09:33.860 --> 00:09:39.960
that we have to go back 2,400
years or so to get to that.

00:09:39.960 --> 00:09:44.390
And when we do, we think about
Plato's most famous work,

00:09:44.390 --> 00:09:47.720
The Republic, which
was clearly a metaphor

00:09:47.720 --> 00:09:52.330
to what goes on in our brains
and our minds and our thinking.

00:09:52.330 --> 00:09:54.380
He couched it in
terms of a metaphor

00:09:54.380 --> 00:09:57.029
with how a state is
organized with philosopher

00:09:57.029 --> 00:09:58.820
kings and merchants
and soldiers and stuff.

00:09:58.820 --> 00:10:00.444
But he was clearly
talking about a kind

00:10:00.444 --> 00:10:04.010
of theory of brain-mind
thinking that

00:10:04.010 --> 00:10:06.170
suggested there are
agents in there that are

00:10:06.170 --> 00:10:08.780
kind of all working together.

00:10:08.780 --> 00:10:12.110
But it's important
to note, I think,

00:10:12.110 --> 00:10:17.630
that The Republic is a good
translation of the Latin de re

00:10:17.630 --> 00:10:20.120
publica, which is
a bad translation

00:10:20.120 --> 00:10:23.150
of the Greek politeia.

00:10:23.150 --> 00:10:27.080
And politeia, interestingly,
is a Greek word

00:10:27.080 --> 00:10:31.040
that my Greek friends
tell me is untranslatable.

00:10:31.040 --> 00:10:34.970
But it means something
like a society or community

00:10:34.970 --> 00:10:36.920
or something like that.

00:10:36.920 --> 00:10:39.620
And the book was about the mind.

00:10:39.620 --> 00:10:43.040
So Plato-- it could
have been translated

00:10:43.040 --> 00:10:47.230
as the society of
mind, in which case

00:10:47.230 --> 00:10:49.040
it would have anticipated
Marvin Minsky's

00:10:49.040 --> 00:10:53.390
book with the same
title by 2,400 years.

00:10:53.390 --> 00:10:57.480
Well, maybe that was
not the first time

00:10:57.480 --> 00:10:59.420
that humans thought
about thinking,

00:10:59.420 --> 00:11:03.800
but it was an early landmark.

00:11:03.800 --> 00:11:06.440
And now it's, I think,
useful to go back to when

00:11:06.440 --> 00:11:08.380
humans started thinking.

00:11:08.380 --> 00:11:12.620
And that takes us back
about 50,000 years--

00:11:12.620 --> 00:11:14.450
not millions of years--

00:11:14.450 --> 00:11:17.480
a few tens of
thousands of years.

00:11:17.480 --> 00:11:20.030
So it probably happened
in southern Africa.

00:11:20.030 --> 00:11:22.250
It was probably 60,000
or 70,000 years ago

00:11:22.250 --> 00:11:24.860
that it started happening.

00:11:24.860 --> 00:11:27.729
It probably happened in
the neck-down population,

00:11:27.729 --> 00:11:29.270
because if the
population is too big,

00:11:29.270 --> 00:11:32.920
an innovation can't take hold.

00:11:32.920 --> 00:11:36.560
It was about that
time that people--

00:11:36.560 --> 00:11:40.610
us, we-- started drilling
holes in sea shells,

00:11:40.610 --> 00:11:43.154
presumably for making jewelry.

00:11:43.154 --> 00:11:44.570
And then it wasn't
long after that

00:11:44.570 --> 00:11:49.130
that we departed from those
Neanderthal guys in a big way.

00:11:49.130 --> 00:11:51.220
And we started painting
caves like the ones

00:11:51.220 --> 00:11:56.980
at Lascaux, carving figurines
like the one at Brassempouy.

00:11:56.980 --> 00:12:01.790
And I think the most important
question we can ask in AI

00:12:01.790 --> 00:12:04.970
is what makes us different
from that Neanderthal who

00:12:04.970 --> 00:12:07.382
couldn't do these things?

00:12:07.382 --> 00:12:09.340
See, that's not the
question that Turing asked.

00:12:09.340 --> 00:12:11.050
The question Turing
asked was how can we

00:12:11.050 --> 00:12:13.367
make a computer reason?

00:12:13.367 --> 00:12:14.950
Because as a
mathematician, he thought

00:12:14.950 --> 00:12:18.660
that was a kind of supreme
capability of human thinking.

00:12:18.660 --> 00:12:22.330
And so for 20, 30
years, AI people

00:12:22.330 --> 00:12:26.542
focused on reasoning
as the center of AI.

00:12:26.542 --> 00:12:28.250
And what they should
have been asking is,

00:12:28.250 --> 00:12:31.870
what makes us different from
the Neanderthals and chimpanzees

00:12:31.870 --> 00:12:34.180
and other species?

00:12:34.180 --> 00:12:38.120
It creates a different
research agenda.

00:12:38.120 --> 00:12:41.890
Well, I'm much influenced by
the paleoanthropologist, Ian

00:12:41.890 --> 00:12:44.320
Tattersall, who writes
extensively about this

00:12:44.320 --> 00:12:46.870
and says that it
didn't evolve, it

00:12:46.870 --> 00:12:50.770
was more of a discovery
than an evolution.

00:12:50.770 --> 00:12:53.770
Our brains came to be
what they are for reasons

00:12:53.770 --> 00:12:56.990
other than human intelligence.

00:12:56.990 --> 00:13:00.370
So he thinks of it
as a minor change

00:13:00.370 --> 00:13:04.260
or even a discovery of
something we didn't know we had.

00:13:04.260 --> 00:13:10.540
In any event, he talks
about becoming symbolic.

00:13:10.540 --> 00:13:12.290
But of course, as a
paleoanthropologist

00:13:12.290 --> 00:13:14.680
and not a computationalist,
he doesn't

00:13:14.680 --> 00:13:18.520
have the vocabulary
for talking about that

00:13:18.520 --> 00:13:21.670
in computational terms.

00:13:21.670 --> 00:13:25.480
So you have to go to
someone like Noam Chomsky

00:13:25.480 --> 00:13:28.360
to get a more computational
perspective on this.

00:13:28.360 --> 00:13:30.570
And what Chomsky says is that--

00:13:30.570 --> 00:13:33.270
who is also, by the way,
a fan of Tattersall--

00:13:33.270 --> 00:13:35.560
what Chomsky says is
that what happened

00:13:35.560 --> 00:13:38.980
is that we acquired the
ability to take two concepts

00:13:38.980 --> 00:13:41.107
and put them together
and make a third concept

00:13:41.107 --> 00:13:42.190
and do that without limit.

00:13:45.480 --> 00:13:47.410
An AI person would say,
oh, Chomsky's talking

00:13:47.410 --> 00:13:48.346
about semantic nets.

00:13:48.346 --> 00:13:49.720
A linguist would
say he's talking

00:13:49.720 --> 00:13:50.710
about the merge operation.

00:13:50.710 --> 00:13:51.710
But it's the same thing.

00:13:56.060 --> 00:13:59.750
As an aside, I'll tell you that
a very important book will come

00:13:59.750 --> 00:14:02.960
out in January, I think,
by Berwick and Chomsky

00:14:02.960 --> 00:14:05.670
and addresses two questions--

00:14:05.670 --> 00:14:08.840
why do we have any language and
why do we have more than one?

00:14:08.840 --> 00:14:11.030
You know, when you think
about it, it's weird.

00:14:11.030 --> 00:14:13.520
Why should we have a language
and now that we have one,

00:14:13.520 --> 00:14:15.740
why should we all
have different ones?

00:14:15.740 --> 00:14:20.180
And their answer is roughly
that this innovation

00:14:20.180 --> 00:14:22.170
made language possible.

00:14:22.170 --> 00:14:26.240
But once you've
got the competence,

00:14:26.240 --> 00:14:30.380
it can manifest itself in
many engineering solutions.

00:14:30.380 --> 00:14:34.190
They also talk a
lot about how we're

00:14:34.190 --> 00:14:36.470
different from other
species, and they

00:14:36.470 --> 00:14:38.900
like to talk about the
fact that we can think

00:14:38.900 --> 00:14:41.480
about stuff that isn't there.

00:14:41.480 --> 00:14:43.610
So we can think about
apples even when

00:14:43.610 --> 00:14:46.460
we're not looking at an apple.

00:14:46.460 --> 00:14:49.406
But back to the main line,
when Spelke talked to you,

00:14:49.406 --> 00:14:50.780
she didn't talk
to you about what

00:14:50.780 --> 00:14:53.270
I consider to be the
greatest experiment

00:14:53.270 --> 00:14:55.640
in developmental
psychology ever,

00:14:55.640 --> 00:14:58.340
even though it wasn't
necessarily-- well,

00:14:58.340 --> 00:14:59.240
I confused myself.

00:14:59.240 --> 00:15:01.090
Let me tell you
about the experiment.

00:15:01.090 --> 00:15:04.430
Spelke doesn't do
rats, but other people

00:15:04.430 --> 00:15:06.980
do rats with the
following observation--

00:15:06.980 --> 00:15:10.385
take a rectangular
room and there

00:15:10.385 --> 00:15:14.120
are hiding places in all four
corners that are identical.

00:15:14.120 --> 00:15:16.010
While the rat is
watching you put food

00:15:16.010 --> 00:15:20.790
in one of these places,
a box cloth over it,

00:15:20.790 --> 00:15:24.560
and then you disorient the
rat by spinning it around.

00:15:24.560 --> 00:15:27.170
And you watch what the rat does.

00:15:27.170 --> 00:15:28.730
And rats are pretty smart.

00:15:28.730 --> 00:15:29.730
They do the right thing.

00:15:29.730 --> 00:15:33.530
Those opposite corners are
the right answer, right?

00:15:33.530 --> 00:15:35.120
Because the room is
rectangular, those

00:15:35.120 --> 00:15:40.100
are the two possible places
that the food could be.

00:15:40.100 --> 00:15:43.700
So then you can repeat this
experiment with a small child,

00:15:43.700 --> 00:15:48.170
you get the same answer, or with
a intelligent adult like me,

00:15:48.170 --> 00:15:51.920
and you get the same answer
because it's the right answer.

00:15:51.920 --> 00:15:57.680
But now the next thing is
you paint one wall blue

00:15:57.680 --> 00:16:00.390
and repeat the experiment.

00:16:00.390 --> 00:16:03.320
What do you think the rat does?

00:16:03.320 --> 00:16:06.090
Both ways.

00:16:06.090 --> 00:16:12.100
You repeat the experiment
with a small child, both ways.

00:16:12.100 --> 00:16:15.320
Repeat the experiment with me.

00:16:15.320 --> 00:16:17.770
Finally we get it right.

00:16:17.770 --> 00:16:21.431
What's the difference?

00:16:21.431 --> 00:16:22.430
And when does it happen?

00:16:22.430 --> 00:16:26.270
When does this small
child become an adult?

00:16:26.270 --> 00:16:28.480
After elaborate and
careful experiments

00:16:28.480 --> 00:16:31.810
of the kind that
Spelke is noted for,

00:16:31.810 --> 00:16:34.930
she has determined that the
onset of this capability

00:16:34.930 --> 00:16:37.570
arises when the child
starts using the words left

00:16:37.570 --> 00:16:41.299
and right in their own
descriptions of the world.

00:16:41.299 --> 00:16:43.090
They understand left
and right before that,

00:16:43.090 --> 00:16:46.940
but this is when they
start using those words.

00:16:46.940 --> 00:16:49.880
That's when it happens.

00:16:49.880 --> 00:16:53.130
Now we introduce the
notion of verbal shadowing.

00:16:53.130 --> 00:16:57.260
So I read to you the Declaration
of Independence or something

00:16:57.260 --> 00:17:00.599
else and as I say it,
you say it back to me.

00:17:00.599 --> 00:17:02.390
It's sort of like
simultaneous translation,

00:17:02.390 --> 00:17:05.329
only it's English to English.

00:17:05.329 --> 00:17:08.390
And now you take an adult
human, and even while they're

00:17:08.390 --> 00:17:12.349
walking into the room, they're
doing this verbal shadowing.

00:17:12.349 --> 00:17:16.190
And what happens in
that circumstance?

00:17:16.190 --> 00:17:19.910
That reduces the adult
to the level of a rat.

00:17:19.910 --> 00:17:21.329
They can't do it.

00:17:21.329 --> 00:17:23.329
And you say, well, didn't
you see the blue wall?

00:17:23.329 --> 00:17:28.770
And they'll say, yeah, I saw the
blue wall but couldn't use it.

00:17:28.770 --> 00:17:32.880
So Spelke's
interpretation of this

00:17:32.880 --> 00:17:36.550
is that the words have
jammed the processor.

00:17:36.550 --> 00:17:39.660
That's why we don't use our
laptops in class, right,

00:17:39.660 --> 00:17:42.390
because we only have
one language processor,

00:17:42.390 --> 00:17:43.920
and it can be jammed.

00:17:43.920 --> 00:17:44.870
It's jammed by email.

00:17:44.870 --> 00:17:48.170
I'm jammed by used car
salesmen talking fast.

00:17:48.170 --> 00:17:49.450
It's easy to jam it.

00:17:49.450 --> 00:17:51.634
And when we jam it,
it can't do what

00:17:51.634 --> 00:17:52.800
you would think it could do.

00:17:59.380 --> 00:18:01.180
So Spelke has an
interpretation to this

00:18:01.180 --> 00:18:04.090
that says what we humans have
is combinators, the ability

00:18:04.090 --> 00:18:07.324
to take formation of different
kinds and put it together.

00:18:07.324 --> 00:18:08.740
I have a different
interpretation,

00:18:08.740 --> 00:18:11.980
which I'll tell you
about at the end

00:18:11.980 --> 00:18:15.259
if you ask me why I
think Spelke has it--

00:18:15.259 --> 00:18:17.300
why I have a different
interpretation from Spelke

00:18:17.300 --> 00:18:21.560
for this experiment.

00:18:21.560 --> 00:18:23.560
And then what we've got
is we've got the ability

00:18:23.560 --> 00:18:25.750
to build descriptions that
seem to be the defining

00:18:25.750 --> 00:18:28.000
characteristic of
human intelligence.

00:18:28.000 --> 00:18:30.564
We've got it in the
case at Lascaux.

00:18:30.564 --> 00:18:32.230
We've got it in the
thoughts of Chomsky.

00:18:32.230 --> 00:18:34.330
We've got it in these
experiments of Spelke.

00:18:34.330 --> 00:18:35.290
We've got descriptions.

00:18:35.290 --> 00:18:39.640
And so those are the influences
that led me to this thing

00:18:39.640 --> 00:18:42.364
I call the strong
story hypothesis.

00:18:42.364 --> 00:18:44.530
And when you think about
it, almost all of education

00:18:44.530 --> 00:18:47.110
is about stories.

00:18:47.110 --> 00:18:49.780
You know, you start with
fairy tales that keep you

00:18:49.780 --> 00:18:51.700
from running away in the mall.

00:18:51.700 --> 00:18:55.287
You'll be eaten by big
bad wolf if you do.

00:18:55.287 --> 00:18:57.370
And you end up with all
these professional schools

00:18:57.370 --> 00:19:00.470
that people go to--
law, business, medicine,

00:19:00.470 --> 00:19:02.920
and even engineering.

00:19:02.920 --> 00:19:05.200
You might say, well,
engineering, that's not

00:19:05.200 --> 00:19:06.670
really-- is that case studies?

00:19:06.670 --> 00:19:09.610
And the answer is, if
you talk to somebody

00:19:09.610 --> 00:19:12.550
that knows what they're
doing, what they do

00:19:12.550 --> 00:19:16.030
is very often telling a story.

00:19:16.030 --> 00:19:19.060
My friend, Gerry Sussman,
a computer scientist

00:19:19.060 --> 00:19:23.830
whose work I use, is fond
of teaching circuit theory

00:19:23.830 --> 00:19:26.110
as a hobby.

00:19:26.110 --> 00:19:28.670
And when you hear him
talk about this circuit,

00:19:28.670 --> 00:19:31.210
he talks about a signal
coming in from the left

00:19:31.210 --> 00:19:33.775
and migrating through
that capacitor

00:19:33.775 --> 00:19:37.950
and going into the
base of a transistor

00:19:37.950 --> 00:19:40.990
and causing a voltage drop
across the emitter, which

00:19:40.990 --> 00:19:44.040
creates a current that
flows into the collector,

00:19:44.040 --> 00:19:45.580
and that causes--

00:19:45.580 --> 00:19:48.910
he's just basically telling the
story of how that signal flows

00:19:48.910 --> 00:19:49.860
through the network.

00:19:49.860 --> 00:19:51.028
It's storytelling.

00:19:54.310 --> 00:19:56.100
So if you believe
that, then these

00:19:56.100 --> 00:20:01.140
are the steps that were
prescribed by Marr and Tomaso

00:20:01.140 --> 00:20:08.314
as well in the early days
of their presence at MIT.

00:20:08.314 --> 00:20:10.980
These are the things you need to
do if you believe that and want

00:20:10.980 --> 00:20:12.021
to do something about it.

00:20:15.200 --> 00:20:18.870
And these steps were
articulated at the time,

00:20:18.870 --> 00:20:20.940
in part because people in
artificial intelligence

00:20:20.940 --> 00:20:25.110
were, in Marr's words,
too mechanistic.

00:20:25.110 --> 00:20:27.360
I talked about this
on the first day,

00:20:27.360 --> 00:20:30.420
that people would fall in love
with a particular mechanism--

00:20:30.420 --> 00:20:34.080
a hammer-- and try to use
it for everything instead

00:20:34.080 --> 00:20:35.580
of understanding
what the problem is

00:20:35.580 --> 00:20:39.210
before you select the
tools to bring to bear

00:20:39.210 --> 00:20:41.040
on producing a solution.

00:20:43.590 --> 00:20:46.971
And so being an engineer,
one of the steps here

00:20:46.971 --> 00:20:48.720
that I'm particularly
fond of, once you've

00:20:48.720 --> 00:20:52.680
got the articulated
behavior 100%,

00:20:52.680 --> 00:20:54.591
eventually you have
to build something.

00:20:54.591 --> 00:20:57.090
Because as an engineer, I think
I don't really understand it

00:20:57.090 --> 00:20:59.532
unless I can build it.

00:20:59.532 --> 00:21:00.990
And then building
it, things emerge

00:21:00.990 --> 00:21:02.615
that I wouldn't have
thought about if I

00:21:02.615 --> 00:21:04.930
hadn't tried to build it.

00:21:04.930 --> 00:21:06.240
Well, anyhow, let's see.

00:21:06.240 --> 00:21:08.310
Step one, characterize
the behavior.

00:21:08.310 --> 00:21:10.734
The behavior has to
story understanding.

00:21:10.734 --> 00:21:12.150
So I'm going to
need some stories.

00:21:12.150 --> 00:21:16.980
And so I tend to work
with short summaries

00:21:16.980 --> 00:21:21.640
of Shakespearean plays,
medical cases, cyber

00:21:21.640 --> 00:21:28.660
warfare, classical social
studies, and psychology.

00:21:31.240 --> 00:21:34.950
And these stories are
written by us so as

00:21:34.950 --> 00:21:36.600
to get through Boris's parser.

00:21:39.260 --> 00:21:43.220
So they are carefully prepared.

00:21:43.220 --> 00:21:45.380
But they're human readable.

00:21:45.380 --> 00:21:47.310
We're not encoding
this stuff up.

00:21:47.310 --> 00:21:49.060
This is the sort of
thing you could read,

00:21:49.060 --> 00:21:52.200
and if you read that you say,
yeah, this is kind of funny,

00:21:52.200 --> 00:21:54.250
but you can understand it.

00:21:54.250 --> 00:21:56.910
Summary of Macbeth.

00:21:56.910 --> 00:22:00.840
Here is a little fragment of
it, and it is easier to read.

00:22:03.650 --> 00:22:04.820
So what do we want to do?

00:22:04.820 --> 00:22:07.270
What can we bring to bear
on understanding the story?

00:22:07.270 --> 00:22:09.310
If you read that
story, you'd see--

00:22:09.310 --> 00:22:14.390
I could ask you a question,
is Duncan dead at the end?

00:22:14.390 --> 00:22:15.640
And how would you know?

00:22:15.640 --> 00:22:17.890
It doesn't say he's
dead at the end.

00:22:17.890 --> 00:22:21.640
He was murdered, but--

00:22:21.640 --> 00:22:25.100
I could ask you is this
a story about revenge?

00:22:25.100 --> 00:22:27.427
The word revenge
is never mentioned,

00:22:27.427 --> 00:22:29.260
and you have to think
about it a little bit.

00:22:29.260 --> 00:22:30.910
But you'd probably
conclude in the end

00:22:30.910 --> 00:22:33.390
that it's about revenge.

00:22:33.390 --> 00:22:35.620
So now we ask ourselves
what kinds of knowledge

00:22:35.620 --> 00:22:41.380
is required to know that
Duncan is dead at the end

00:22:41.380 --> 00:22:44.410
and that it's about revenge?

00:22:44.410 --> 00:22:47.870
Well, first of all, we
need some common sense.

00:22:47.870 --> 00:22:50.710
And what we've found,
somewhat to our surprise,

00:22:50.710 --> 00:22:52.810
is that much of that can
be expressed in terms

00:22:52.810 --> 00:22:56.550
of simple if-then rules.

00:22:56.550 --> 00:23:04.600
And these seven rule types
arose because people building

00:23:04.600 --> 00:23:08.500
software to understand stories
found that they were necessary.

00:23:08.500 --> 00:23:11.170
We knew we needed
the first kind.

00:23:13.690 --> 00:23:16.210
If you kill someone,
then they are dead.

00:23:16.210 --> 00:23:19.970
Every other one here arose
because we reached an impasse

00:23:19.970 --> 00:23:23.860
in our construction of our
story understanding system.

00:23:23.860 --> 00:23:25.360
So the may rules.

00:23:25.360 --> 00:23:29.470
If I anger you, you may kill me.

00:23:29.470 --> 00:23:32.680
Thank god we don't always
kill people who anger us.

00:23:32.680 --> 00:23:37.840
But we humans always are
searching for explanations.

00:23:37.840 --> 00:23:40.510
So if you kill me, and I've
previously angered you,

00:23:40.510 --> 00:23:43.660
and you can't think of any other
reason for why the killing took

00:23:43.660 --> 00:23:47.140
place, then the
anger is supposed.

00:23:47.140 --> 00:23:51.850
So that's the explanation
for rule type number two.

00:23:51.850 --> 00:23:56.160
Sometimes we use abduction.

00:23:56.160 --> 00:23:58.660
You might have a firm belief
that anybody who kills somebody

00:23:58.660 --> 00:23:59.290
is crazy.

00:23:59.290 --> 00:24:01.210
That's abduction.

00:24:01.210 --> 00:24:03.940
You're presuming the
antecedent from the presence

00:24:03.940 --> 00:24:05.098
of a consequent.

00:24:10.580 --> 00:24:13.340
So those are kinds of rules
that work in the background

00:24:13.340 --> 00:24:14.720
to deal with the story.

00:24:14.720 --> 00:24:16.610
And of course, there
are things that

00:24:16.610 --> 00:24:18.789
are explicit in the story too.

00:24:18.789 --> 00:24:20.330
Here are some examples
of things that

00:24:20.330 --> 00:24:23.830
are-- of causal relations that
are explicit in the story.

00:24:23.830 --> 00:24:26.460
The first kind says that this
happens because that happened.

00:24:26.460 --> 00:24:28.640
A close, tight
causal connection.

00:24:28.640 --> 00:24:31.225
Second kind, we know
there's a causal connection,

00:24:31.225 --> 00:24:32.225
but it might be lengthy.

00:24:35.550 --> 00:24:37.560
The third kind,
the strangely kind,

00:24:37.560 --> 00:24:40.140
arose when one of
my students was

00:24:40.140 --> 00:24:45.390
working on Crow creation myths.

00:24:45.390 --> 00:24:48.440
He happened to be a
Crow Indian and so

00:24:48.440 --> 00:24:51.060
a natural interest
in that mythology.

00:24:51.060 --> 00:24:54.810
And what he noted was
that in Crow mythology,

00:24:54.810 --> 00:24:58.080
you're often told that
something is connected causally

00:24:58.080 --> 00:25:02.190
and also told that you'll
never understand it.

00:25:02.190 --> 00:25:04.860
Old Man Coyote
reached into the lake

00:25:04.860 --> 00:25:07.236
and pulled up a handful
of mud and made the world,

00:25:07.236 --> 00:25:08.610
and you will never
understand how

00:25:08.610 --> 00:25:13.840
that happened, is a kind of
typical expression in Crow

00:25:13.840 --> 00:25:15.210
creation mythology.

00:25:15.210 --> 00:25:18.150
So all of these arose because
we were trying to understand

00:25:18.150 --> 00:25:20.890
particular kinds of stories.

00:25:20.890 --> 00:25:21.390
OK.

00:25:21.390 --> 00:25:23.220
So that's all background.

00:25:23.220 --> 00:25:27.710
Here it is in operation.

00:25:27.710 --> 00:25:30.420
And that's about the
speed that it goes.

00:25:30.420 --> 00:25:32.310
That is reading the story--

00:25:32.310 --> 00:25:33.900
the summary of
Macbeth that you saw

00:25:33.900 --> 00:25:37.050
on the screen a few moments ago.

00:25:37.050 --> 00:25:39.230
But of course, it's
invisible at that scale.

00:25:39.230 --> 00:25:42.730
So let me blow a piece of it up.

00:25:42.730 --> 00:25:47.360
There you see the piece that
says, oh, Macbeth murdered

00:25:47.360 --> 00:25:47.960
Duncan.

00:25:47.960 --> 00:25:49.180
Duncan becomes dead.

00:25:49.180 --> 00:25:55.250
So the yellow parts are inserted
by background knowledge.

00:25:55.250 --> 00:25:58.850
The white parts are
explicit in the story.

00:26:02.890 --> 00:26:04.150
So you may have also noted--

00:26:04.150 --> 00:26:05.608
no, you wouldn't
have noted-- well,

00:26:05.608 --> 00:26:07.210
my drawing attention to it.

00:26:07.210 --> 00:26:09.760
We have not only the concept
that the yellow parts--

00:26:14.410 --> 00:26:17.760
yes, the yellow parts
there are conclusions.

00:26:17.760 --> 00:26:19.522
The white parts are explicit.

00:26:19.522 --> 00:26:20.980
And what you can
see, incidentally,

00:26:20.980 --> 00:26:23.140
is that-- just from
the colors-- that much

00:26:23.140 --> 00:26:24.580
of the understanding
of the story

00:26:24.580 --> 00:26:28.487
is inferred from what is there.

00:26:28.487 --> 00:26:30.070
It's a funny kind
of way of saying it,

00:26:30.070 --> 00:26:32.740
but you've seen
in computer vision

00:26:32.740 --> 00:26:34.290
or you will see
in computer vision

00:26:34.290 --> 00:26:39.006
that what you think you
see is half hallucination,

00:26:39.006 --> 00:26:40.630
and what you think
you see in the story

00:26:40.630 --> 00:26:43.252
is also half hallucinated.

00:26:43.252 --> 00:26:44.710
It seems that the
authors just tell

00:26:44.710 --> 00:26:47.850
us enough to keep us on track.

00:26:47.850 --> 00:26:50.860
In any event, we have
not only the yellow parts

00:26:50.860 --> 00:26:52.900
that are inferred,
but we also have

00:26:52.900 --> 00:26:55.974
the observation that one piece
may be connected to another.

00:26:55.974 --> 00:26:57.640
And that can only be
determined by doing

00:26:57.640 --> 00:27:01.100
a search through that
so-called elaboration graph.

00:27:01.100 --> 00:27:05.200
So here are the same sorts of
things that you can search for.

00:27:05.200 --> 00:27:07.630
There's a definition
of a Pyrrhic victory.

00:27:07.630 --> 00:27:11.310
You do something or rather you
want something at least you

00:27:11.310 --> 00:27:13.780
becoming happy, but
ultimately the same wanting

00:27:13.780 --> 00:27:17.440
leads to disaster.

00:27:17.440 --> 00:27:18.110
So there it is.

00:27:18.110 --> 00:27:20.530
That green thing down there is
reflected in the green elements

00:27:20.530 --> 00:27:22.570
up there that are picked
out of the entire graph

00:27:22.570 --> 00:27:23.653
because they're connected.

00:27:23.653 --> 00:27:26.680
And I'll show you how they're
connected in this one.

00:27:26.680 --> 00:27:29.170
So this is the Pyrrhic
victory concept

00:27:29.170 --> 00:27:32.560
that has been extracted from
the story by a search program.

00:27:32.560 --> 00:27:35.560
So we start with Macbeth
wanting to be king.

00:27:35.560 --> 00:27:37.240
He murders Duncan
because of that.

00:27:37.240 --> 00:27:39.250
He becomes happy,
because he eventually

00:27:39.250 --> 00:27:43.130
ends up being king himself,
but downstream he's harmed.

00:27:43.130 --> 00:27:44.902
So it's a kind of
Pyrrhic victory.

00:27:44.902 --> 00:27:46.360
And now you say to
me, well, that's

00:27:46.360 --> 00:27:47.901
not my definition
of Pyrrhic victory,

00:27:47.901 --> 00:27:51.830
and that's OK, because
we all have nuanced

00:27:51.830 --> 00:27:53.960
differences in our concepts.

00:27:53.960 --> 00:27:56.380
So this is just
one computer's idea

00:27:56.380 --> 00:27:59.974
of what a Pyrrhic victory is.

00:27:59.974 --> 00:28:01.390
Here are the kinds
of things we've

00:28:01.390 --> 00:28:05.120
been able to do as a
consequence of, to our surprise,

00:28:05.120 --> 00:28:08.471
just having a suite of rules
and a suite of concepts.

00:28:08.471 --> 00:28:10.720
And what I'm going to spend
the next few minutes doing

00:28:10.720 --> 00:28:13.480
is just taking you quickly
through a few examples

00:28:13.480 --> 00:28:15.070
of these kinds of things.

00:28:19.800 --> 00:28:22.980
This is reading Macbeth from
two different cultural points

00:28:22.980 --> 00:28:26.940
of view, an Asian point of
view and a US point of view.

00:28:26.940 --> 00:28:28.680
There were some
fabulous experiments

00:28:28.680 --> 00:28:33.570
conducted in the
'90s in a high school

00:28:33.570 --> 00:28:38.490
in the outskirts of Beijing and
in a high school in Wisconsin.

00:28:38.490 --> 00:28:42.390
And these experiments involved
having the students read

00:28:42.390 --> 00:28:45.240
stories about
violence and observing

00:28:45.240 --> 00:28:48.480
the reaction of the
students to those stories.

00:28:48.480 --> 00:28:53.520
And what they found was that
at a statistically significant

00:28:53.520 --> 00:28:57.870
level, not this or that, but
a statistically significant

00:28:57.870 --> 00:29:00.780
level, the Asian students
outside of Beijing

00:29:00.780 --> 00:29:03.920
attributed violence
to situations.

00:29:03.920 --> 00:29:08.190
And they would ask, what made
that person want to do that?

00:29:08.190 --> 00:29:12.450
Whereas the kids in Wisconsin
had a greater tendency to say,

00:29:12.450 --> 00:29:15.946
that person must be
completely crazy.

00:29:15.946 --> 00:29:17.820
So one attributed to
the situation, the other

00:29:17.820 --> 00:29:24.790
was dispositional, to
use the technical term.

00:29:24.790 --> 00:29:28.080
So here we see in one
reading of Macbeth--

00:29:28.080 --> 00:29:29.600
let me show it blown up.

00:29:29.600 --> 00:29:34.350
In the top version,
Macduff kills Macbeth

00:29:34.350 --> 00:29:38.620
because there's a revenge
situation that forces it.

00:29:38.620 --> 00:29:42.090
And the other interpretation
is because Macduff is crazy.

00:29:44.970 --> 00:29:48.490
So another kind of
similar pairing--

00:29:48.490 --> 00:29:48.990
oh, wow.

00:29:48.990 --> 00:29:50.190
That was fast.

00:29:50.190 --> 00:29:54.650
This is a story about the
Estonian Russian cyber

00:29:54.650 --> 00:29:56.179
war of 2007.

00:29:56.179 --> 00:29:58.720
Could you-- you probably didn't
hear the rules of engagement,

00:29:58.720 --> 00:30:00.600
but I don't talk to
the back of laptops.

00:30:00.600 --> 00:30:02.670
So if you'd put that
away, I'd appreciate it.

00:30:02.670 --> 00:30:12.360
So in 2007, the Estonians
moved a war memorial

00:30:12.360 --> 00:30:15.060
from the Soviet era out
of the center of town

00:30:15.060 --> 00:30:18.060
to a cemetery in the outskirts.

00:30:18.060 --> 00:30:23.700
And about 30% of the Estonian
population is Russian,

00:30:23.700 --> 00:30:28.362
and they were irritated by this.

00:30:28.362 --> 00:30:32.340
And it had never been
proven, but the next day

00:30:32.340 --> 00:30:35.790
the Estonian National
Network went down,

00:30:35.790 --> 00:30:38.930
and government
websites were defaced.

00:30:38.930 --> 00:30:41.490
And this was hurtful, because
the Estonians pride themselves

00:30:41.490 --> 00:30:44.220
in being very
technically advanced.

00:30:44.220 --> 00:30:48.330
In Estonia, it's a
right to be educated

00:30:48.330 --> 00:30:51.540
on how to use the internet.

00:30:51.540 --> 00:30:53.820
They have a national ID card.

00:30:53.820 --> 00:30:56.530
They have a law that says if
anybody looks at your data,

00:30:56.530 --> 00:30:57.960
they've got to explain why.

00:30:57.960 --> 00:31:01.690
They're a very technically
sophisticated country.

00:31:01.690 --> 00:31:06.570
And so what's the interpretation
of this attack, which

00:31:06.570 --> 00:31:09.630
was presumed to be done
by either ethnic Russians

00:31:09.630 --> 00:31:12.410
in Estonia or by
people from Russia?

00:31:12.410 --> 00:31:16.860
Well, was it an
aggressive revenge or

00:31:16.860 --> 00:31:20.490
was it teaching the
Estonians a lesson?

00:31:20.490 --> 00:31:23.460
It depends on what?

00:31:23.460 --> 00:31:25.855
It depends on whose
side you're on.

00:31:25.855 --> 00:31:26.980
That's the only difference.

00:31:26.980 --> 00:31:29.040
And that's what produced the
difference in interpretation

00:31:29.040 --> 00:31:30.810
on those two sides--
one being aggressive

00:31:30.810 --> 00:31:33.420
revenge and the
other being teaching

00:31:33.420 --> 00:31:34.450
the Estonians a lesson.

00:31:34.450 --> 00:31:38.340
By the way, I was in
Estonia in January.

00:31:38.340 --> 00:31:39.910
That's the statue
that wasn't there.

00:31:43.270 --> 00:31:44.870
Give you another example.

00:31:44.870 --> 00:31:48.230
I'm just trying to show you
some of the breadth of our story

00:31:48.230 --> 00:31:50.390
understanding activity.

00:31:50.390 --> 00:31:53.890
So the next example comes
about because shortly

00:31:53.890 --> 00:31:57.370
after the terrorist attack
on the World Trade Center,

00:31:57.370 --> 00:32:00.880
there was a strong
interest in bringing

00:32:00.880 --> 00:32:03.220
political science and
artificial intelligence

00:32:03.220 --> 00:32:08.830
together to make it
possible to understand how

00:32:08.830 --> 00:32:11.950
other people think when
they're not necessarily crazy,

00:32:11.950 --> 00:32:14.770
they've just got
different backgrounds.

00:32:14.770 --> 00:32:17.130
The thesis is that we are
the stories in our culture.

00:32:20.290 --> 00:32:24.360
So I was at a meeting
in Washington,

00:32:24.360 --> 00:32:26.520
and the only thing I
remember from that meeting

00:32:26.520 --> 00:32:31.530
is one of the participants
drew a parallel between the Tet

00:32:31.530 --> 00:32:36.000
Offensive in Vietnam and
the Arab-Israeli war that

00:32:36.000 --> 00:32:41.010
took place about six
or seven years later.

00:32:41.010 --> 00:32:42.160
And here's the story.

00:32:51.900 --> 00:32:52.619
OK.

00:32:52.619 --> 00:32:54.410
And here's what happened
seven years later.

00:33:06.590 --> 00:33:10.140
What do you suppose
happened next?

00:33:10.140 --> 00:33:12.970
And of course, the
answer is quite clear.

00:33:12.970 --> 00:33:19.270
And when we feel like talking
about the long-range eventual

00:33:19.270 --> 00:33:21.416
practical uses of the
stuff we're talking about,

00:33:21.416 --> 00:33:22.790
this is the kind
of thing we say.

00:33:22.790 --> 00:33:26.504
What we want to do is we want
to build for political analysts

00:33:26.504 --> 00:33:28.170
tools that would be
as important to them

00:33:28.170 --> 00:33:31.230
as spreadsheets are to a
financial analyst, tools that

00:33:31.230 --> 00:33:35.890
can enable to predict
or expect or understand

00:33:35.890 --> 00:33:40.190
unintended consequence of
actions you might perform.

00:33:40.190 --> 00:33:44.210
So this a gap and
alignment problem.

00:33:44.210 --> 00:33:45.760
And here is one case
in which we have

00:33:45.760 --> 00:33:48.450
departed from modeling humans.

00:33:48.450 --> 00:33:51.580
And we did it because
one of our students

00:33:51.580 --> 00:33:56.230
was a refugee from
bioengineering.

00:33:56.230 --> 00:34:01.120
And he knew a lot about aligning
proteins, sequences, and DNA

00:34:01.120 --> 00:34:02.560
sequences.

00:34:02.560 --> 00:34:06.260
And so he brought to our group
the Needleman-Wunsch algorithm

00:34:06.260 --> 00:34:07.620
for doing alignment.

00:34:07.620 --> 00:34:12.170
And we used that to
align those stories.

00:34:12.170 --> 00:34:15.531
So there they are
with the gaps in them.

00:34:15.531 --> 00:34:17.989
We took those two stories I
showed you on a previous slide.

00:34:17.989 --> 00:34:20.699
We put a couple of gaps in.

00:34:20.699 --> 00:34:23.139
The Needleman-Wunsch
algorithm aligned them,

00:34:23.139 --> 00:34:25.659
and then we were able to fill
in the gaps using one to fill

00:34:25.659 --> 00:34:26.794
in the gap in the other.

00:34:26.794 --> 00:34:28.210
And since you can't
see it, here's

00:34:28.210 --> 00:34:29.580
what it would have filled in.

00:34:34.110 --> 00:34:38.780
So that's an example
of how we can

00:34:38.780 --> 00:34:41.989
use precedence to think
about what happens next

00:34:41.989 --> 00:34:46.790
or what happened in the missing
piece or what led to this.

00:34:46.790 --> 00:34:50.330
It's a kind of
analogical reasoning.

00:34:50.330 --> 00:34:55.550
My next example is putting
the system in teaching mode.

00:34:55.550 --> 00:34:56.510
We have a system.

00:34:56.510 --> 00:34:57.320
We have a student.

00:34:57.320 --> 00:34:58.750
We want to teach the
student something.

00:34:58.750 --> 00:35:01.125
Maybe the student is from Mars
and doesn't know anything.

00:35:01.125 --> 00:35:05.090
So this is an example of
how the Genesis system can

00:35:05.090 --> 00:35:09.590
watch another version of
itself, not understand the story

00:35:09.590 --> 00:35:12.741
and supply the
missing knowledge.

00:35:12.741 --> 00:35:14.240
So this is a hint
at how it might be

00:35:14.240 --> 00:35:15.531
used in an educational context.

00:35:20.400 --> 00:35:23.406
And once you can have a
model of the listener,

00:35:23.406 --> 00:35:24.780
then you can also
think about how

00:35:24.780 --> 00:35:27.990
you can shape the story so
as to make some aspect of it

00:35:27.990 --> 00:35:31.500
more or less believable.

00:35:31.500 --> 00:35:33.000
So you notice I'm
carefully avoiding

00:35:33.000 --> 00:35:35.475
the word propaganda, which
puts a pejorative spin on it.

00:35:35.475 --> 00:35:37.600
But if you're just trying
to teach somebody values,

00:35:37.600 --> 00:35:39.910
that's another way
of thinking about it.

00:35:39.910 --> 00:35:42.660
So this is the Hansel
and Gretel story.

00:35:42.660 --> 00:35:45.750
And the system has been ordered
to make the woodcutter be

00:35:45.750 --> 00:35:50.930
likeable, because he does some
good things and bad things.

00:35:50.930 --> 00:35:52.680
So when we do that,
you'll note that there

00:35:52.680 --> 00:35:56.600
are some things that are
struck out, the stuff in red,

00:35:56.600 --> 00:36:00.120
and some things that are
marked in green for emphasis.

00:36:00.120 --> 00:36:02.880
And let me blow those
up so you can see them.

00:36:06.460 --> 00:36:08.980
So the stuff that the
woodcutter does that's good

00:36:08.980 --> 00:36:10.990
are highlighted and
bolded, and the things

00:36:10.990 --> 00:36:12.906
that we don't want to
say about the woodcutter

00:36:12.906 --> 00:36:16.610
because it makes him look
bad, we strike those out.

00:36:16.610 --> 00:36:18.110
And of course,
another way of making

00:36:18.110 --> 00:36:22.690
somebody-- what's another way
of making somebody look good?

00:36:22.690 --> 00:36:24.760
Make everybody else
look bad, right?

00:36:24.760 --> 00:36:27.280
So we can flip a
switch and have him

00:36:27.280 --> 00:36:29.160
make comments about
the witch too so

00:36:29.160 --> 00:36:35.335
that the woodcutter looks even
better because the bad behavior

00:36:35.335 --> 00:36:38.450
of the witch is highlighted.

00:36:38.450 --> 00:36:40.407
So these are just some
examples of the kinds

00:36:40.407 --> 00:36:41.240
of things we can do.

00:36:41.240 --> 00:36:42.430
Here's another one.

00:36:42.430 --> 00:36:45.720
This is that Macbeth
story played out in--

00:36:45.720 --> 00:36:49.250
it's about 180 80
or 100 sentences,

00:36:49.250 --> 00:36:52.450
and we can summarize it.

00:36:52.450 --> 00:36:56.050
So we can use our
understanding of the story

00:36:56.050 --> 00:37:00.560
to trim away all the stuff that
is not particularly important.

00:37:00.560 --> 00:37:02.882
So what is not
particularly important?

00:37:02.882 --> 00:37:04.840
Anything that's not
connected to something else

00:37:04.840 --> 00:37:08.141
is not particularly important.

00:37:08.141 --> 00:37:09.140
Think about it this way.

00:37:09.140 --> 00:37:11.220
The only reason
you read a story--

00:37:11.220 --> 00:37:14.110
if it's not just for fun-- the
only reason you read a story--

00:37:14.110 --> 00:37:18.010
a case study-- is because you
think it'll be useful later.

00:37:18.010 --> 00:37:21.420
And it's only useful later
if it exerts constraint.

00:37:21.420 --> 00:37:24.430
And it only exerts constraint
if there are connections--

00:37:24.430 --> 00:37:26.234
causal connections in this case.

00:37:26.234 --> 00:37:28.150
So we take all the stuff
that's not connected,

00:37:28.150 --> 00:37:29.904
and we get rid of it.

00:37:29.904 --> 00:37:31.570
Then we get rid of
anything that doesn't

00:37:31.570 --> 00:37:33.760
lead to a central concept.

00:37:33.760 --> 00:37:36.756
So in this case, we say that the
thing is about Pyrrhic victory.

00:37:36.756 --> 00:37:37.880
That's the central concept.

00:37:37.880 --> 00:37:42.340
We get rid of everything that
doesn't bear on that concept

00:37:42.340 --> 00:37:43.900
pattern instantiation.

00:37:43.900 --> 00:37:46.450
And then we can squeeze
this thing down to about 20%

00:37:46.450 --> 00:37:47.587
of its original size.

00:37:47.587 --> 00:37:50.170
And now I come to the thing that
we were talking about before,

00:37:50.170 --> 00:37:53.530
and that is how do you
find the right precedent?

00:37:53.530 --> 00:37:56.620
Well, if you do a Google
search, it's mostly--

00:37:56.620 --> 00:37:58.880
well, they're getting more
and more sophisticated,

00:37:58.880 --> 00:38:02.067
but most searches
are mostly keywords.

00:38:02.067 --> 00:38:04.150
But now we've got something
better than key words.

00:38:04.150 --> 00:38:05.415
We've got concepts.

00:38:05.415 --> 00:38:06.790
So what I'm going
to show you now

00:38:06.790 --> 00:38:11.980
is a portrayal of an
information retrieval test case

00:38:11.980 --> 00:38:15.580
that we did with 14 or
15 conflict stories.

00:38:15.580 --> 00:38:18.509
We're interested in how
close they were together.

00:38:18.509 --> 00:38:20.050
Because the closer
they are together,

00:38:20.050 --> 00:38:24.080
the more one is likely to be a
useful precedent for another.

00:38:24.080 --> 00:38:27.550
So in one of these
matrices, what you see

00:38:27.550 --> 00:38:30.470
is how close they are when
viewed from the point of view

00:38:30.470 --> 00:38:32.290
of key words.

00:38:32.290 --> 00:38:41.130
That's the one on the bottom.

00:38:41.130 --> 00:38:43.560
The one on the top is how
close they are with respect

00:38:43.560 --> 00:38:46.170
to the concepts that
they contain, you know,

00:38:46.170 --> 00:38:48.900
the words like revenge,
attack, and not

00:38:48.900 --> 00:38:51.390
present in the story as
words, but are present there

00:38:51.390 --> 00:38:53.110
anyway as concepts.

00:38:53.110 --> 00:38:55.590
And the only point
of this pairing

00:38:55.590 --> 00:38:58.124
is to show that the
consideration of similarity

00:38:58.124 --> 00:38:59.790
is different depending
on whether you're

00:38:59.790 --> 00:39:06.540
thinking in terms of concepts
or thinking in terms of words.

00:39:06.540 --> 00:39:08.400
So here's a story.

00:39:12.250 --> 00:39:14.490
A young man went to
work for a company.

00:39:14.490 --> 00:39:16.980
His boss was pretty mean.

00:39:16.980 --> 00:39:20.640
Wouldn't let him
go to conferences.

00:39:20.640 --> 00:39:23.270
One day somebody
else in the company

00:39:23.270 --> 00:39:25.950
arranged for him to go
to a conference anyhow.

00:39:25.950 --> 00:39:27.970
Provided transportation.

00:39:27.970 --> 00:39:31.459
He went to the conference, and
he met some interesting people.

00:39:31.459 --> 00:39:33.000
But unfortunately,
circumstances were

00:39:33.000 --> 00:39:36.744
that he had to leave early
to catch a flight back home.

00:39:36.744 --> 00:39:38.910
And then some of the people
he met at the conference

00:39:38.910 --> 00:39:40.920
started looking for
him because he was so--

00:39:40.920 --> 00:39:43.980
so what story am I telling?

00:39:43.980 --> 00:39:47.950
It's pretty obviously a
Cinderella story, right?

00:39:47.950 --> 00:39:50.115
But there's no pumpkin.

00:39:50.115 --> 00:39:51.240
There's no fairy godmother.

00:39:51.240 --> 00:39:54.120
It's just that even
though the agents are

00:39:54.120 --> 00:39:56.670
very different in terms
of their descriptions,

00:39:56.670 --> 00:40:01.610
the relationships between
them are pretty much the same.

00:40:01.610 --> 00:40:04.950
So over the years, what we've
done quite without intending

00:40:04.950 --> 00:40:07.110
it or expecting
it or realizing it

00:40:07.110 --> 00:40:12.200
is that we have duplicated in
Genesis the kinds of thinking

00:40:12.200 --> 00:40:15.880
that Marvin Minsky talks a lot
about in his most recent book,

00:40:15.880 --> 00:40:17.850
The Emotion Machine.

00:40:17.850 --> 00:40:21.144
He likes to talk in
terms of multiplicities.

00:40:21.144 --> 00:40:22.560
We have multiple
ways of thinking.

00:40:22.560 --> 00:40:26.130
We have multiple
representations.

00:40:26.130 --> 00:40:29.760
And those kinds of reasoning
occur on multiple levels,

00:40:29.760 --> 00:40:32.250
from instinctive
reactions at the bottom

00:40:32.250 --> 00:40:34.380
to self-conscious
reflection on the top.

00:40:38.690 --> 00:40:40.700
So quite without
our intending it,

00:40:40.700 --> 00:40:44.400
when we thought about
it one day by accident,

00:40:44.400 --> 00:40:50.450
we had this epiphany that
we've been working to implement

00:40:50.450 --> 00:40:54.160
much of what is in that book.

00:40:54.160 --> 00:40:57.000
So so far, and I'm going to
depart from story understanding

00:40:57.000 --> 00:40:59.840
a little bit to talk to you
about some other hypotheses

00:40:59.840 --> 00:41:01.130
of mine.

00:41:01.130 --> 00:41:03.410
So far, there are two--
the strong story hypothesis

00:41:03.410 --> 00:41:06.770
and then there's this
inner language hypothesis

00:41:06.770 --> 00:41:09.650
that Chomsky likes
to talk a lot about.

00:41:09.650 --> 00:41:12.920
We have an inner language,
and our inner language

00:41:12.920 --> 00:41:15.620
came before our outer language.

00:41:15.620 --> 00:41:17.840
And this is what makes
it possible to think.

00:41:17.840 --> 00:41:20.410
So those are two
hypotheses-- inner language

00:41:20.410 --> 00:41:22.250
and strong story.

00:41:22.250 --> 00:41:27.320
Here's another
one-- it's important

00:41:27.320 --> 00:41:31.400
that we're social animals,
and it's actually important

00:41:31.400 --> 00:41:33.450
that we talk to each other.

00:41:33.450 --> 00:41:38.180
Once Danny Hillis, a famous
guy, a graduate of ours,

00:41:38.180 --> 00:41:41.300
came into my office
and said, have you ever

00:41:41.300 --> 00:41:42.980
had the experience of--

00:41:42.980 --> 00:41:44.630
well, you often
talked to Marvin.

00:41:44.630 --> 00:41:45.984
Yeah, I do.

00:41:45.984 --> 00:41:47.900
And have you ever had
the experience, he said,

00:41:47.900 --> 00:41:50.480
of having Marvin guess?

00:41:50.480 --> 00:41:52.440
He has a very short
attention span, Marvin,

00:41:52.440 --> 00:41:56.540
and he'll often guess your
idea before you've fully

00:41:56.540 --> 00:41:57.500
explained it?

00:41:57.500 --> 00:41:58.100
Yes, I said.

00:41:58.100 --> 00:42:00.000
It happens all the time.

00:42:00.000 --> 00:42:02.870
Isn't it the case, Danny said,
that the idea that he guesses

00:42:02.870 --> 00:42:04.610
you have is better
than the idea you're

00:42:04.610 --> 00:42:07.520
actually trying
to tell him about?

00:42:07.520 --> 00:42:08.120
Yes.

00:42:08.120 --> 00:42:11.120
And then he pointed out, well,
maybe when we talk to ourself,

00:42:11.120 --> 00:42:13.440
it's doing the
same kind of thing.

00:42:13.440 --> 00:42:16.460
It's accessing ideas and
putting them together

00:42:16.460 --> 00:42:21.470
in ways that wouldn't be
possible if we weren't talking.

00:42:21.470 --> 00:42:24.440
So it often happens
that ideas come

00:42:24.440 --> 00:42:26.240
about when we talk
to each other,

00:42:26.240 --> 00:42:31.640
because it forces the rendering
of our thoughts and language.

00:42:31.640 --> 00:42:34.400
And if we don't have a friend
or don't happen to have anybody

00:42:34.400 --> 00:42:35.450
around we can talk to--

00:42:35.450 --> 00:42:38.500
I feel like I talk to
myself all the time.

00:42:38.500 --> 00:42:41.750
And maybe that's an
important consequence

00:42:41.750 --> 00:42:44.930
or important aspect
of our intelligence

00:42:44.930 --> 00:42:47.320
is conversation that we
carry on with ourself.

00:42:52.430 --> 00:42:56.060
Be careful doing this out loud.

00:42:56.060 --> 00:42:58.940
Some people will think
you've got a screw loose.

00:42:58.940 --> 00:43:01.190
But let me let me show
you an experiment I

00:43:01.190 --> 00:43:05.510
consider to be extraordinarily
interesting along these lines.

00:43:05.510 --> 00:43:08.391
It was done by a friend of mine
at the University Pittsburgh

00:43:08.391 --> 00:43:08.890
Michelin.

00:43:11.640 --> 00:43:14.600
Mickey, as she is called,
was working with students

00:43:14.600 --> 00:43:15.839
on physics problems.

00:43:15.839 --> 00:43:17.380
You've all done this
kind of problem,

00:43:17.380 --> 00:43:22.040
and it's about pulleys and
weights and forces and stuff.

00:43:22.040 --> 00:43:25.100
And so these students
were learning the subject,

00:43:25.100 --> 00:43:28.250
and so she gave them a quiz,
and she had them talk out loud

00:43:28.250 --> 00:43:30.970
as they were
working on the quiz.

00:43:30.970 --> 00:43:37.780
And she kept track
of how many things

00:43:37.780 --> 00:43:39.990
the best students
said to themselves

00:43:39.990 --> 00:43:43.726
and how many things the worst
students said to themselves.

00:43:43.726 --> 00:43:45.100
So in this particular
experiment,

00:43:45.100 --> 00:43:46.266
there weren't many students.

00:43:46.266 --> 00:43:50.170
I think eight-- four good
ones and four bad ones.

00:43:50.170 --> 00:43:52.960
And the good ones scored
twice as high as the bad ones,

00:43:52.960 --> 00:43:54.490
and here's the data.

00:43:57.770 --> 00:44:03.020
The better students said about
3 and 1/2 times more stuff

00:44:03.020 --> 00:44:06.080
to themselves than
the other ones.

00:44:06.080 --> 00:44:07.580
So unfortunately,
this is backwards.

00:44:07.580 --> 00:44:09.288
We don't know if we
took the bad students

00:44:09.288 --> 00:44:13.679
and encouraged them to talk
more, if they'd become smarter.

00:44:13.679 --> 00:44:14.720
So we're not saying that.

00:44:14.720 --> 00:44:16.160
But it is interesting
observation

00:44:16.160 --> 00:44:18.050
that the ones who talked
to themselves more

00:44:18.050 --> 00:44:19.275
were actually better at it.

00:44:19.275 --> 00:44:20.900
And what they were
saying was a mixture

00:44:20.900 --> 00:44:23.900
of problem-solving things
and physics things.

00:44:23.900 --> 00:44:28.590
Like I'm stuck or maybe
I should try that again

00:44:28.590 --> 00:44:31.280
or physics things like I think
I have to do a force diagram.

00:44:31.280 --> 00:44:34.500
A mixture of those
kinds of things.

00:44:34.500 --> 00:44:41.660
So talking seems to surface
a kind of capability

00:44:41.660 --> 00:44:43.400
that not every animal has.

00:44:45.737 --> 00:44:46.820
And then there's this one.

00:44:51.440 --> 00:44:54.630
So it isn't just that we
have a perceptual apparatus,

00:44:54.630 --> 00:44:59.300
it's that we can direct it
to do stuff in our behalf.

00:44:59.300 --> 00:45:03.170
That's what I think
is part of the magic.

00:45:03.170 --> 00:45:05.360
So my standard examples--

00:45:05.360 --> 00:45:06.870
John kissed Mary.

00:45:06.870 --> 00:45:07.910
Did John touch Mary?

00:45:11.630 --> 00:45:13.440
Everybody knows that
the answer is yes.

00:45:13.440 --> 00:45:14.231
How do you know it?

00:45:14.231 --> 00:45:17.164
Because you imagine
it and you see it.

00:45:17.164 --> 00:45:18.830
So there's a lot of
talk in AI about how

00:45:18.830 --> 00:45:21.080
you gather
common-sense knowledge

00:45:21.080 --> 00:45:23.774
and how you can only know
a limited number of facts.

00:45:23.774 --> 00:45:25.190
I think it's all
screwy, because I

00:45:25.190 --> 00:45:26.990
think a lot of our
common sense comes just

00:45:26.990 --> 00:45:34.550
in time by the engagement
of our perceptual apparatus.

00:45:34.550 --> 00:45:36.550
So there's John
kissing Mary, and now I

00:45:36.550 --> 00:45:39.080
want to give you another puzzle.

00:45:39.080 --> 00:45:43.450
And the next puzzle is how
many countries in Africa

00:45:43.450 --> 00:45:44.630
does the equator go through?

00:45:48.100 --> 00:45:48.850
Does anybody know?

00:45:51.360 --> 00:45:56.430
I've asked students who
come to MIT from Africa,

00:45:56.430 --> 00:45:58.350
and they don't know.

00:45:58.350 --> 00:45:59.850
And some of them
come from countries

00:45:59.850 --> 00:46:02.270
that are on the equator
and they don't know.

00:46:05.620 --> 00:46:07.300
But now you know.

00:46:07.300 --> 00:46:08.380
And what's happened?

00:46:11.690 --> 00:46:14.502
Your eyes scan across that
red line, and you count.

00:46:14.502 --> 00:46:17.630
Shimon Ullman would call
it a visual routine.

00:46:17.630 --> 00:46:20.490
So you're forming-- you're
creating a little program.

00:46:20.490 --> 00:46:23.050
Your language
system is demanding

00:46:23.050 --> 00:46:26.170
that your visual system
run a little program that

00:46:26.170 --> 00:46:28.340
scans across and counts.

00:46:28.340 --> 00:46:31.020
And your vision system
reports back the answer.

00:46:31.020 --> 00:46:32.660
And that I think is a miracle.

00:46:35.320 --> 00:46:36.790
So one more example.

00:46:36.790 --> 00:46:37.900
It's a little grizzly.

00:46:37.900 --> 00:46:40.200
I hope you don't mind.

00:46:40.200 --> 00:46:45.200
So a couple of years ago,
I installed a table saw.

00:46:45.200 --> 00:46:46.135
I like to--

00:46:46.135 --> 00:46:47.050
I'm an engineer.

00:46:47.050 --> 00:46:48.360
I like to build stuff.

00:46:48.360 --> 00:46:50.080
I like to make stuff.

00:46:50.080 --> 00:46:52.840
And I had a friend of mine
who's a cabinetmaker--

00:46:52.840 --> 00:46:56.020
a good cabinetmaker-- helped
me to install the saw.

00:46:56.020 --> 00:46:59.560
And he said, you must
never wear gloves

00:46:59.560 --> 00:47:03.280
when you operate this tool.

00:47:03.280 --> 00:47:04.550
And I said well--

00:47:04.550 --> 00:47:08.140
and before I got the first
word out, I knew why.

00:47:08.140 --> 00:47:10.360
No one had ever told me.

00:47:10.360 --> 00:47:12.670
I had never witnessed
an experience that would

00:47:12.670 --> 00:47:15.010
suggest that should be a rule.

00:47:15.010 --> 00:47:16.870
But I imagined it.

00:47:16.870 --> 00:47:19.430
Can you imagine it?

00:47:19.430 --> 00:47:21.410
What I need you to
imagine is that you're

00:47:21.410 --> 00:47:26.010
wearing the kind of
fluffy cotton gloves.

00:47:26.010 --> 00:47:27.760
Got it now?

00:47:27.760 --> 00:47:32.720
And the fluffy cotton glove
gets caught in the blade.

00:47:32.720 --> 00:47:34.440
And now you know why
you would never--

00:47:34.440 --> 00:47:36.360
I don't think any of you
would ever use gloves

00:47:36.360 --> 00:47:38.734
when you operate a table saw
now, because you can imagine

00:47:38.734 --> 00:47:42.474
the grisly result. So it's not
just our perceptual apparatus.

00:47:42.474 --> 00:47:44.640
It's our ability to deploy
our perceptual apparatus,

00:47:44.640 --> 00:47:48.990
and our imagination, I
think, is a great miracle.

00:47:48.990 --> 00:47:51.317
That vision is still hard,
as everyone in vision

00:47:51.317 --> 00:47:51.900
will tell you.

00:47:51.900 --> 00:47:54.420
Some years ago, I was involved
in a DARPA program that

00:47:54.420 --> 00:48:01.050
had as its objective recognizing
48 activities, 47 of which

00:48:01.050 --> 00:48:03.960
can be performed by humans.

00:48:03.960 --> 00:48:05.160
One of them is fly.

00:48:05.160 --> 00:48:08.370
So that doesn't count, I guess.

00:48:08.370 --> 00:48:12.781
After a couple of years into the
program, they retrenched to 17.

00:48:12.781 --> 00:48:14.280
At the end of the
program, they said

00:48:14.280 --> 00:48:18.420
if you could do six
reliably, you'll be a hero.

00:48:18.420 --> 00:48:21.540
And my team caught
everyone's attention

00:48:21.540 --> 00:48:24.070
by saying we wouldn't
recognize any actions that

00:48:24.070 --> 00:48:26.670
would distract people.

00:48:26.670 --> 00:48:29.160
So vision is very hard.

00:48:29.160 --> 00:48:31.860
And then stories do, of
course, come together

00:48:31.860 --> 00:48:34.266
with perception, right?

00:48:34.266 --> 00:48:35.640
At some point,
you've doubtlessly

00:48:35.640 --> 00:48:37.740
in the course of the last
two weeks seen this example.

00:48:37.740 --> 00:48:38.406
What am I doing?

00:48:41.380 --> 00:48:42.430
What am I doing?

00:48:42.430 --> 00:48:43.810
AUDIENCE: Drinking.

00:48:43.810 --> 00:48:47.460
PATRICK HENRY WINSTON: And
then there's Ullman's cat.

00:48:47.460 --> 00:48:49.830
What's it doing?

00:48:49.830 --> 00:48:57.900
So my interpretation of this
is that that cat and I are--

00:48:57.900 --> 00:49:00.510
it's the same story.

00:49:00.510 --> 00:49:02.200
You can imagine
that there's thirst,

00:49:02.200 --> 00:49:05.640
that there are activities
that lead to water or liquid

00:49:05.640 --> 00:49:06.700
passing into the mouth.

00:49:06.700 --> 00:49:10.530
So we give them the same label,
even though visually they're

00:49:10.530 --> 00:49:12.650
as different as
anything could be.

00:49:12.650 --> 00:49:15.300
You would never get a
deep neural net program

00:49:15.300 --> 00:49:17.700
that's been trained
on me to recognize

00:49:17.700 --> 00:49:19.710
that that's a cat drinking.

00:49:19.710 --> 00:49:22.120
There's visually
nothing similar at all.

00:49:24.760 --> 00:49:26.760
All right.

00:49:26.760 --> 00:49:29.710
But we might ask this question--

00:49:29.710 --> 00:49:31.150
can we have an
intelligent machine

00:49:31.150 --> 00:49:33.430
without a perceptual system?

00:49:33.430 --> 00:49:37.464
You know, that Genesis system
with Macbeth and all that?

00:49:37.464 --> 00:49:39.130
Is it really intelligent
when it doesn't

00:49:39.130 --> 00:49:41.110
have any perceptual
apparatus at all?

00:49:41.110 --> 00:49:42.150
It can't see anything.

00:49:42.150 --> 00:49:44.191
It doesn't know what it
feels like to be stabbed.

00:49:47.300 --> 00:49:50.070
I think it's an interesting
philosophical question.

00:49:50.070 --> 00:49:54.230
And I'm a little agnostic
on this right now,

00:49:54.230 --> 00:50:00.560
a little more agnostic than I
was half a year ago, because I

00:50:00.560 --> 00:50:03.290
went back to the republic.

00:50:03.290 --> 00:50:07.750
You remember that metaphor
of the cave and the republic?

00:50:07.750 --> 00:50:09.320
There's a metaphor of the cave.

00:50:09.320 --> 00:50:13.130
You have some prisoners, and
they're chained in this cave,

00:50:13.130 --> 00:50:14.720
and there's a fire somewhere.

00:50:14.720 --> 00:50:19.442
And all they can see is their
own shadows against the wall.

00:50:19.442 --> 00:50:20.900
That's all they've
got for reality.

00:50:23.490 --> 00:50:28.250
And so their reality
is extremely limited.

00:50:28.250 --> 00:50:31.650
So they're intelligent
but they're limited.

00:50:31.650 --> 00:50:34.866
So I think, generalizing
that metaphor,

00:50:34.866 --> 00:50:36.740
I think a machine without
a perceptual system

00:50:36.740 --> 00:50:40.440
has extraordinarily
limited reality.

00:50:40.440 --> 00:50:42.920
And maybe we need a
little bit of perception

00:50:42.920 --> 00:50:44.410
to have any kind--

00:50:44.410 --> 00:50:46.999
to have what we would be
comfortable to calling

00:50:46.999 --> 00:50:47.540
intelligence.

00:50:47.540 --> 00:50:48.530
But we don't need much.

00:50:48.530 --> 00:50:51.620
And another way of thinking
about it is, sort of the fact

00:50:51.620 --> 00:50:56.210
that our own reality is limited.

00:50:56.210 --> 00:51:00.380
If you compare our visual
system to that of a bee,

00:51:00.380 --> 00:51:04.220
they have a much broader
spectrum of wavelengths

00:51:04.220 --> 00:51:08.170
that they can make use of,
because they don't have--

00:51:08.170 --> 00:51:09.760
do you know why?

00:51:09.760 --> 00:51:13.740
We are limited because we
have water in our eyeballs.

00:51:13.740 --> 00:51:15.920
And so some of that stuff
in the far ultraviolet

00:51:15.920 --> 00:51:17.420
can't get through.

00:51:17.420 --> 00:51:20.120
But bees can see it.

00:51:20.120 --> 00:51:23.704
And then that's comparing
us humans with bees.

00:51:23.704 --> 00:51:25.620
How about comparing one
human against another?

00:51:31.690 --> 00:51:33.585
At this distance, I
can hardly see it.

00:51:33.585 --> 00:51:34.210
Can you see it?

00:51:34.210 --> 00:51:35.020
AUDIENCE: Boat.

00:51:35.020 --> 00:51:36.478
PATRICK HENRY
WINSTON: It's a boat,

00:51:36.478 --> 00:51:40.540
but some people can't see it,
because they're colorblind.

00:51:40.540 --> 00:51:45.340
So for them, there's a
slight impairment of reality,

00:51:45.340 --> 00:51:47.680
just like a computer
without a perceptual system

00:51:47.680 --> 00:51:49.315
would have a big
impairment of reality.

00:51:52.570 --> 00:51:57.430
So it's been said many times
that what we're doing here

00:51:57.430 --> 00:52:00.025
is we're trying to understand
the science side of things.

00:52:00.025 --> 00:52:02.400
And we think that that will
lead to engineering advances.

00:52:02.400 --> 00:52:08.410
And people often
ask this, and those

00:52:08.410 --> 00:52:13.950
who don't believe that human
intelligence is relevant

00:52:13.950 --> 00:52:18.142
will say, well, airplanes
don't fly like birds.

00:52:18.142 --> 00:52:19.600
What do you think
of that argument?

00:52:22.300 --> 00:52:27.040
I think it has a gigantic
hole, and it turns out

00:52:27.040 --> 00:52:31.180
that the Wright brothers were
extremely interested in birds,

00:52:31.180 --> 00:52:34.600
because they knew that
birds could tell them

00:52:34.600 --> 00:52:38.300
something about the
secrets of aerodynamics.

00:52:38.300 --> 00:52:43.065
And all flying machines have
to deal with the secrets

00:52:43.065 --> 00:52:44.065
of that kind of physics.

00:52:46.730 --> 00:52:48.370
So we study humans,
not because we're

00:52:48.370 --> 00:52:50.200
going to build a machine
that has neurons in it,

00:52:50.200 --> 00:52:51.574
but because we
want to understand

00:52:51.574 --> 00:52:55.330
the computational imperatives
that human intelligence can

00:52:55.330 --> 00:52:57.280
shed light on.

00:52:57.280 --> 00:52:59.890
That's why we think it has
engineering value, even

00:52:59.890 --> 00:53:03.610
though we won't in
any likely future

00:53:03.610 --> 00:53:11.170
be building computers with
synapses of the kind we have.

00:53:11.170 --> 00:53:17.969
Well, now we come to the dangers
that we started out with.

00:53:17.969 --> 00:53:20.260
What do you think we should--
suppose that machines can

00:53:20.260 --> 00:53:21.060
become--

00:53:21.060 --> 00:53:23.949
they do become really smart,
and we've got machine learning.

00:53:23.949 --> 00:53:24.490
What is that?

00:53:24.490 --> 00:53:26.290
That's modern statistics.

00:53:26.290 --> 00:53:29.439
And of course, it's useful.

00:53:29.439 --> 00:53:31.480
What if they became really
smart in the same ways

00:53:31.480 --> 00:53:32.188
that we're smart?

00:53:32.188 --> 00:53:34.378
What would we want to
do to protect ourselves?

00:53:37.190 --> 00:53:41.270
Well, for this, I'd like
to introduce the subject

00:53:41.270 --> 00:53:44.510
by asking you to read
the following story.

00:53:44.510 --> 00:53:50.431
This was part of that Morrison
paying a suite of experiments.

00:53:59.114 --> 00:54:00.780
I'm sorry these are
so full of violence.

00:54:00.780 --> 00:54:02.770
This happens to be
what they worked with.

00:54:09.730 --> 00:54:16.290
So after the students read
this story, they were asked

00:54:16.290 --> 00:54:23.010
did Lu kill Sean because
America is individualistic?

00:54:23.010 --> 00:54:25.750
And the Asian students would
have a tendency to say yes.

00:54:25.750 --> 00:54:28.760
So how can we model
that in our system?

00:54:28.760 --> 00:54:33.190
Well, to start out with, this
is the original interpretation

00:54:33.190 --> 00:54:35.350
of the story as told.

00:54:35.350 --> 00:54:38.170
And if you look at
where that arrow is,

00:54:38.170 --> 00:54:42.460
you see that that's where Lu
kills Sean, and just in back

00:54:42.460 --> 00:54:44.800
of it is the means.

00:54:44.800 --> 00:54:46.082
He shot him with a gun.

00:54:46.082 --> 00:54:47.540
But it's not
connected to anything.

00:54:50.320 --> 00:54:52.470
And so the instantaneous
response is,

00:54:52.470 --> 00:54:54.930
we don't know why
Lu killed Sean.

00:54:54.930 --> 00:54:57.602
But what Genesis does
at this point is,

00:54:57.602 --> 00:54:59.310
when asked the question,
did Lu kill Sean

00:54:59.310 --> 00:55:01.440
because America is
individualistic?

00:55:01.440 --> 00:55:04.470
It goes into its
own memory and said

00:55:04.470 --> 00:55:08.340
I am modeling an Asian reader.

00:55:08.340 --> 00:55:11.610
I believe that America
is individualistic.

00:55:11.610 --> 00:55:14.430
I will insert that
into the story.

00:55:14.430 --> 00:55:18.480
I will examine the
consequences of that insertion,

00:55:18.480 --> 00:55:20.955
and then see what happens.

00:55:20.955 --> 00:55:23.485
And this is what happens.

00:55:26.005 --> 00:55:28.580
The question is asked and
inserts into the story--

00:55:28.580 --> 00:55:30.640
boom, boom, boom.

00:55:30.640 --> 00:55:35.120
And now Lu kills Sean is
connected all the way back

00:55:35.120 --> 00:55:38.060
to America is individualistic.

00:55:38.060 --> 00:55:40.940
And so the machine can
say yes, but that's not

00:55:40.940 --> 00:55:43.239
the interesting part.

00:55:43.239 --> 00:55:44.780
Now, this is what
it says, but that's

00:55:44.780 --> 00:55:47.000
not the interesting part.

00:55:47.000 --> 00:55:48.640
The interesting part is this--

00:55:52.900 --> 00:55:57.400
it describes to
itself what it's doing

00:55:57.400 --> 00:56:00.070
in its own language,
which it treats

00:56:00.070 --> 00:56:04.040
as its story of
its own behavior.

00:56:04.040 --> 00:56:07.420
So it now has the
capacity to introspect

00:56:07.420 --> 00:56:09.164
into what it itself is doing.

00:56:13.250 --> 00:56:16.010
I think that's pretty cool.

00:56:16.010 --> 00:56:16.770
It's a kind of--

00:56:16.770 --> 00:56:17.270
OK.

00:56:17.270 --> 00:56:19.478
It's a suitcase word--
self-awareness, consciousness,

00:56:19.478 --> 00:56:23.270
a big suitcase word, but you
can say that this system is

00:56:23.270 --> 00:56:24.969
aware of its own behavior.

00:56:24.969 --> 00:56:26.510
By the way, this is
one of the things

00:56:26.510 --> 00:56:29.630
that Turing addressed
in his original paper.

00:56:29.630 --> 00:56:34.400
One of the arguments
against AI was--

00:56:34.400 --> 00:56:36.290
I forgot what Turing called it--

00:56:36.290 --> 00:56:37.790
the disabilities argument.

00:56:37.790 --> 00:56:40.640
And people were saying
computers can never

00:56:40.640 --> 00:56:42.590
do these kinds of
things, one of which

00:56:42.590 --> 00:56:44.360
is be the subject
of its own thought.

00:56:44.360 --> 00:56:47.992
But Genesis is now reading
the story of its own behavior

00:56:47.992 --> 00:56:49.700
and being the subject
of its own thought.

00:56:52.331 --> 00:56:52.830
OK.

00:56:52.830 --> 00:56:55.050
So what if they
really become smart?

00:56:55.050 --> 00:56:57.390
Now, I will become a little
bit on the whimsical side.

00:56:57.390 --> 00:56:58.680
Suppose they really get smart.

00:56:58.680 --> 00:57:01.800
What will we want to do?

00:57:01.800 --> 00:57:03.750
Maybe we ought to simulate
these suckers first

00:57:03.750 --> 00:57:05.375
before we turn them
loose in the world.

00:57:05.375 --> 00:57:07.110
Do you agree with that?

00:57:07.110 --> 00:57:11.070
After all, simulation is
now a well-developed art.

00:57:11.070 --> 00:57:13.600
So we can take these machines--
maybe there will be robots.

00:57:13.600 --> 00:57:15.016
Maybe there will
just be programs,

00:57:15.016 --> 00:57:17.640
and we can do elaborate
simulations to make sure

00:57:17.640 --> 00:57:21.320
that they're not dangerous.

00:57:21.320 --> 00:57:24.560
And we would want to do that in
as natural a world as possible.

00:57:28.690 --> 00:57:32.620
And we'd want to do these
experiments for a long time

00:57:32.620 --> 00:57:36.280
before we turned them loose
to see what kinds of behaviors

00:57:36.280 --> 00:57:39.958
were to be expected.

00:57:39.958 --> 00:57:41.541
And you see where
I'm going with this?

00:57:44.630 --> 00:57:45.862
Maybe we're at it.

00:57:49.930 --> 00:57:52.330
I think it's a pretty
interesting possibility.

00:57:52.330 --> 00:57:56.400
I'm not sure any of you are it,
but I know that I might be it.

00:57:56.400 --> 00:57:59.350
This is a great simulation
to see if we're dangerous.

00:57:59.350 --> 00:58:01.720
And I must say, if
we are a simulation

00:58:01.720 --> 00:58:05.800
to see if we're dangerous,
it's not going very well.

00:58:05.800 --> 00:58:07.192
Key questions revisited.

00:58:07.192 --> 00:58:08.650
Why has AI made so
little progress?

00:58:08.650 --> 00:58:11.650
Because for too many years, it
was about reasoning instead of

00:58:11.650 --> 00:58:13.712
about what's different.

00:58:13.712 --> 00:58:14.920
How can we make progress now?

00:58:14.920 --> 00:58:17.320
By focusing on what
it is that makes

00:58:17.320 --> 00:58:19.960
human intelligence unique.

00:58:19.960 --> 00:58:21.460
How can a computer
be really smart

00:58:21.460 --> 00:58:23.270
without a perceptual
system or can it be?

00:58:23.270 --> 00:58:27.490
And I think yes, but I'm
a little bit agnostic.

00:58:27.490 --> 00:58:28.630
Should engineers care?

00:58:28.630 --> 00:58:31.299
Absolutely, because
it's not the hardware

00:58:31.299 --> 00:58:32.590
that we're trying to replicate.

00:58:32.590 --> 00:58:36.171
It's the understanding of the
computational imperatives.

00:58:36.171 --> 00:58:38.420
What are the dangers and
what should we do about them?

00:58:38.420 --> 00:58:42.250
We need to make any system
that we depend on capable

00:58:42.250 --> 00:58:44.360
of explaining itself.

00:58:44.360 --> 00:58:45.970
It needs to have
a kind of ability

00:58:45.970 --> 00:58:49.600
to explain what it's
doing in our terms.

00:58:49.600 --> 00:58:50.260
No.

00:58:50.260 --> 00:58:51.801
Don't just tell me
it's a school bus.

00:58:51.801 --> 00:58:54.280
Tell me why you think
it's a school bus.

00:58:54.280 --> 00:58:55.210
You did this thing.

00:58:55.210 --> 00:58:57.970
You better be able to
defend yourself in something

00:58:57.970 --> 00:58:59.410
analogous to a court of law.

00:58:59.410 --> 00:59:02.900
These are the things
we need to do.

00:59:02.900 --> 00:59:06.042
And finally, my final
slide is this one.

00:59:06.042 --> 00:59:07.750
This is just a summary
of the things I've

00:59:07.750 --> 00:59:11.430
tried to cover this morning.

00:59:11.430 --> 00:59:17.430
And one last thing, I think it
was Cato the Elder who said,

00:59:17.430 --> 00:59:20.830
carthago delenda
est. Every speech

00:59:20.830 --> 00:59:24.880
to the Roman Senate ended with
Carthage must be destroyed.

00:59:24.880 --> 00:59:26.690
He could be talking
about the sewer system,

00:59:26.690 --> 00:59:31.240
and the final words would be
Carthage must be destroyed.

00:59:31.240 --> 00:59:37.090
Well, here is my analog to
Carthage must be destroyed.

00:59:37.090 --> 00:59:39.130
I think this is so,
because I think--

00:59:39.130 --> 00:59:43.600
well, many people consider
artificial intelligence

00:59:43.600 --> 00:59:44.950
products to be dangerous.

00:59:44.950 --> 00:59:48.010
I think understanding
our own intelligence is

00:59:48.010 --> 00:59:50.470
essential to the
survival of the species.

00:59:50.470 --> 00:59:53.640
We really do need to
understand ourselves better.