1
00:00:01,680 --> 00:00:04,080
The following content is
provided under a Creative

2
00:00:04,080 --> 00:00:05,620
Commons license.

3
00:00:05,620 --> 00:00:07,920
Your support will help
MIT OpenCourseWare

4
00:00:07,920 --> 00:00:12,280
continue to offer high-quality
educational resources for free.

5
00:00:12,280 --> 00:00:14,910
To make a donation or
view additional materials

6
00:00:14,910 --> 00:00:18,870
from hundreds of MIT courses,
visit MIT OpenCourseWare

7
00:00:18,870 --> 00:00:21,814
at ocw.mit.edu.

8
00:00:21,814 --> 00:00:23,480
PATRICK HENRY WINSTON:
Oh, how to start.

9
00:00:23,480 --> 00:00:27,520
I have been on the Navy Science
Board for a couple of decades.

10
00:00:27,520 --> 00:00:30,180
And so as a consequence,
I've had an opportunity

11
00:00:30,180 --> 00:00:35,920
to spend two weeks for a
couple of decades in San Diego.

12
00:00:35,920 --> 00:00:38,230
And the first thing I do
when I get to San Diego

13
00:00:38,230 --> 00:00:43,030
is I go to the zoo, and
I look at the orangutans

14
00:00:43,030 --> 00:00:49,560
and ask myself, how come I'm
out here and they're in there?

15
00:00:49,560 --> 00:00:54,390
How come you're not all covered
with orange hair instead

16
00:00:54,390 --> 00:00:56,640
of hardly any hair at all?

17
00:00:59,800 --> 00:01:05,680
Well, my answer to that is that
we can tell stories and they

18
00:01:05,680 --> 00:01:07,180
can't.

19
00:01:07,180 --> 00:01:08,720
So this is the
center of my talk.

20
00:01:13,700 --> 00:01:20,450
That's what I have come to
believe after a long time.

21
00:01:20,450 --> 00:01:22,780
So I'm going to be
talking about stories

22
00:01:22,780 --> 00:01:25,960
and to give you a
preview of where

23
00:01:25,960 --> 00:01:28,990
I'm going to go with this,
I like to just show you

24
00:01:28,990 --> 00:01:32,190
some of the questions
that I propose to address

25
00:01:32,190 --> 00:01:34,630
in the course of the next hour.

26
00:01:34,630 --> 00:01:36,920
I want to start talking
about this by way of history.

27
00:01:36,920 --> 00:01:38,420
I've been in
artificial intelligence

28
00:01:38,420 --> 00:01:41,650
almost since the
beginning, and so

29
00:01:41,650 --> 00:01:43,220
I was a student
of Marvin Minsky,

30
00:01:43,220 --> 00:01:46,540
and it's interesting that the
field started a long time ago--

31
00:01:46,540 --> 00:01:51,850
55 years ago-- with perhaps
the most important paper

32
00:01:51,850 --> 00:01:56,680
in the field, Steps Toward
Artificial Intelligence.

33
00:01:56,680 --> 00:02:01,510
It was about that time that
the first intelligent program

34
00:02:01,510 --> 00:02:02,440
was written.

35
00:02:02,440 --> 00:02:04,690
It was a program that
could do calculus problems

36
00:02:04,690 --> 00:02:08,530
as well as an MIT freshman--
a very good MIT freshman.

37
00:02:08,530 --> 00:02:11,470
That was done in 1961.

38
00:02:11,470 --> 00:02:16,600
And those programs led to a half
century of enormous progress

39
00:02:16,600 --> 00:02:18,310
in the field.

40
00:02:18,310 --> 00:02:21,490
All sorts of ideas and
subfields like machine learning

41
00:02:21,490 --> 00:02:22,240
were spawned.

42
00:02:22,240 --> 00:02:24,390
Useful applications.

43
00:02:24,390 --> 00:02:26,350
But as you know, there's
been an explosion

44
00:02:26,350 --> 00:02:28,370
in these useful applications
in recent years.

45
00:02:28,370 --> 00:02:29,827
I've stolen this
slide from Tomaso

46
00:02:29,827 --> 00:02:31,910
but I added a couple of
things that I particularly

47
00:02:31,910 --> 00:02:35,710
think are of special
importance like Echo.

48
00:02:35,710 --> 00:02:38,860
You all know about Amazon
Echo, of course, right?

49
00:02:38,860 --> 00:02:40,060
I swear, it astonishes me.

50
00:02:40,060 --> 00:02:41,950
Many people don't know
about Amazon Echo.

51
00:02:41,950 --> 00:02:45,640
It's Siri in a beer can.

52
00:02:45,640 --> 00:02:48,130
It's a Siri that you can
talk to across the room

53
00:02:48,130 --> 00:02:50,680
and say how long should
I boil a hard boiled egg

54
00:02:50,680 --> 00:02:52,390
or I've fallen down.

55
00:02:52,390 --> 00:02:54,430
Please call for an ambulance.

56
00:02:54,430 --> 00:02:55,870
So it's a wonderful thing.

57
00:02:55,870 --> 00:02:57,820
I think most people
don't know about it

58
00:02:57,820 --> 00:02:59,352
because of privacy concerns.

59
00:02:59,352 --> 00:03:01,810
Having something listening to
you all the time in your home

60
00:03:01,810 --> 00:03:04,152
may not be the most
comfortable kind of thing.

61
00:03:04,152 --> 00:03:05,110
But anyhow, it's there.

62
00:03:07,950 --> 00:03:10,390
So this has caused quite a
lot of interest in the field

63
00:03:10,390 --> 00:03:11,380
lately.

64
00:03:11,380 --> 00:03:15,467
Boris has just talked
about Siri and Jeopardy,

65
00:03:15,467 --> 00:03:17,800
but maybe the thing that's
astonished the world the most

66
00:03:17,800 --> 00:03:20,290
is this captioning program
I think you've probably seen

67
00:03:20,290 --> 00:03:24,670
before the last week or two.

68
00:03:24,670 --> 00:03:27,370
And man, I don't know
what to think of this.

69
00:03:27,370 --> 00:03:29,260
I don't know why it's
created such a stir,

70
00:03:29,260 --> 00:03:32,500
except that that
caption makes it

71
00:03:32,500 --> 00:03:36,300
look like that system knows a
lot that it doesn't actually

72
00:03:36,300 --> 00:03:37,720
know.

73
00:03:37,720 --> 00:03:41,260
First of all, it's
trained on still photos.

74
00:03:41,260 --> 00:03:43,960
So it doesn't know about motion.

75
00:03:43,960 --> 00:03:46,150
It doesn't know about play.

76
00:03:46,150 --> 00:03:48,460
It doesn't know what
it means to be young.

77
00:03:48,460 --> 00:03:50,560
It would probably
say the same thing

78
00:03:50,560 --> 00:03:54,520
if we replaced those
faces with geriatrics.

79
00:03:54,520 --> 00:03:59,440
Yet when we see that, we
presume that it knows a lot.

80
00:03:59,440 --> 00:04:01,840
It's sort of
parasitic semantics.

81
00:04:01,840 --> 00:04:04,210
The intelligence is coming
from our interpretation

82
00:04:04,210 --> 00:04:07,364
of the sentence, not from
it producing the sentence.

83
00:04:07,364 --> 00:04:09,280
So yeah, it's a great
engineering achievement,

84
00:04:09,280 --> 00:04:12,830
but it's not as
smart as it looks.

85
00:04:12,830 --> 00:04:14,470
And of course, as
you know, there's

86
00:04:14,470 --> 00:04:18,940
been a whole industry
of fooling papers.

87
00:04:18,940 --> 00:04:22,690
Here's a fooling example.

88
00:04:22,690 --> 00:04:26,800
One of those is considered
by a standard deep neural net

89
00:04:26,800 --> 00:04:28,810
to be a school bus,
and the other is not

90
00:04:28,810 --> 00:04:30,160
considered to be a school bus.

91
00:04:30,160 --> 00:04:33,940
And just a few pixels
have been changed.

92
00:04:33,940 --> 00:04:35,210
Imperceptible to us.

93
00:04:35,210 --> 00:04:39,809
It still looks like a school bus
but not to the deep neural net.

94
00:04:39,809 --> 00:04:42,100
And of course, these things
are considered school buses

95
00:04:42,100 --> 00:04:44,837
in another fooling paper.

96
00:04:44,837 --> 00:04:47,170
Why in the world would those
be considered school buses?

97
00:04:47,170 --> 00:04:50,410
Presumably because of
some sharing of texture.

98
00:04:54,360 --> 00:04:56,580
Well, anyhow, people
have gotten all excited

99
00:04:56,580 --> 00:05:00,850
and especially Elon Musk
has gotten all excited.

100
00:05:00,850 --> 00:05:03,540
We are summoning the demon.

101
00:05:03,540 --> 00:05:07,230
So Musk said that in
an off-the-cuff answer

102
00:05:07,230 --> 00:05:14,410
to a question at the, let's
see, an anniversary of the MIT

103
00:05:14,410 --> 00:05:16,190
AeroAstro Department.

104
00:05:16,190 --> 00:05:18,130
Someone asked him if he
was interested in AI,

105
00:05:18,130 --> 00:05:22,150
and that's his
off-the-cuff response.

106
00:05:22,150 --> 00:05:24,700
So it's interesting
that those of us

107
00:05:24,700 --> 00:05:27,520
who've been around for a while
find this beyond interesting.

108
00:05:27,520 --> 00:05:32,110
It's curious, because a
long time ago philosophers

109
00:05:32,110 --> 00:05:36,774
like Hubert Dreyfus were saying
that it was not possible.

110
00:05:36,774 --> 00:05:39,190
And so now we've shifted from
not possible to the scariest

111
00:05:39,190 --> 00:05:41,820
thing imaginable.

112
00:05:41,820 --> 00:05:44,722
We can't stop ourselves
from chuckling.

113
00:05:49,150 --> 00:05:54,679
Well, Dreyfus must
know that he's wrong,

114
00:05:54,679 --> 00:05:55,720
but maybe he's right too.

115
00:05:55,720 --> 00:05:58,220
What we really had to wait for
for all of these achievements

116
00:05:58,220 --> 00:06:04,104
was massive amounts
of computing.

117
00:06:09,050 --> 00:06:11,410
So I'd like to go a little
further back in history

118
00:06:11,410 --> 00:06:14,410
while I'm talking about
history, and the next thing back

119
00:06:14,410 --> 00:06:21,730
there is 65 years ago Turing's
paper on machine intelligence.

120
00:06:21,730 --> 00:06:25,060
It's interesting that that paper
is widely assumed to be about

121
00:06:25,060 --> 00:06:28,120
the Turing test and it isn't.

122
00:06:28,120 --> 00:06:31,060
If you look at the actual
paper content, what you find

123
00:06:31,060 --> 00:06:34,180
is that only a couple of pages
were devoted to the test.

124
00:06:34,180 --> 00:06:36,160
Most of it was
devoted to discussion

125
00:06:36,160 --> 00:06:39,250
of arguments for and
against the possibility

126
00:06:39,250 --> 00:06:41,320
of artificial intelligence.

127
00:06:41,320 --> 00:06:44,444
I've read that paper 20
times, because I prescribe it

128
00:06:44,444 --> 00:06:46,360
in my course, so I have
to read it every year.

129
00:06:46,360 --> 00:06:49,174
And every time I read it,
I become more convinced

130
00:06:49,174 --> 00:06:50,590
that what Turing
was talking about

131
00:06:50,590 --> 00:06:54,880
is arguments against
AI, not about the tests.

132
00:06:54,880 --> 00:06:56,620
So why the test?

133
00:06:56,620 --> 00:06:58,630
Well, he's a mathematician
and a philosopher.

134
00:06:58,630 --> 00:07:00,490
And no mathematician
or philosopher

135
00:07:00,490 --> 00:07:03,380
would write a paper without
defining their terms.

136
00:07:03,380 --> 00:07:06,730
So he squeezed into
some kind of definition.

137
00:07:06,730 --> 00:07:09,481
Counterarguments took
up a lot of space,

138
00:07:09,481 --> 00:07:10,480
but they didn't have to.

139
00:07:10,480 --> 00:07:13,150
If Turing had taken
Marvin Minsky's course

140
00:07:13,150 --> 00:07:14,920
in the last couple
of years, he wouldn't

141
00:07:14,920 --> 00:07:19,450
have bothered with that test,
because Minsky introduced

142
00:07:19,450 --> 00:07:22,210
the notion of a suitcase word.

143
00:07:22,210 --> 00:07:24,459
That's a word--
he likes that term

144
00:07:24,459 --> 00:07:26,500
because what he means by
that is that the word is

145
00:07:26,500 --> 00:07:30,670
so big, like a big suitcase,
you can stuff anything into it.

146
00:07:30,670 --> 00:07:34,780
So for me, this has been a great
thought because, if you ask me,

147
00:07:34,780 --> 00:07:37,850
is Watson intelligent?

148
00:07:37,850 --> 00:07:40,120
Is the Jeopardy-playing
system intelligent?

149
00:07:40,120 --> 00:07:42,910
My immediate response is, sure.

150
00:07:42,910 --> 00:07:44,800
It has a kind of intelligence.

151
00:07:44,800 --> 00:07:47,470
As Boris points out, it's not
every kind of intelligence,

152
00:07:47,470 --> 00:07:49,471
and it doesn't think
like we do, and there

153
00:07:49,471 --> 00:07:51,220
are some kinds of
thinking that it doesn't

154
00:07:51,220 --> 00:07:53,660
do that we do quite handily.

155
00:07:53,660 --> 00:07:55,950
But it's silly to argue about
whether it's intelligent

156
00:07:55,950 --> 00:07:56,450
or not.

157
00:07:56,450 --> 00:08:00,073
It has aspects, some
kinds of intelligence.

158
00:08:04,150 --> 00:08:09,280
So what Turing really
did was establish

159
00:08:09,280 --> 00:08:13,604
that this is something that
serious people can think about

160
00:08:13,604 --> 00:08:15,520
and suggests that there's
no reason to believe

161
00:08:15,520 --> 00:08:16,728
that it won't happen someday.

162
00:08:20,071 --> 00:08:23,870
And he centered that whole paper
on these kind of arguments,

163
00:08:23,870 --> 00:08:25,530
to the arguments against AI.

164
00:08:28,740 --> 00:08:30,630
It's fun to talk about those.

165
00:08:30,630 --> 00:08:32,490
Each of them deserves attention.

166
00:08:32,490 --> 00:08:36,600
I'll just say a word or two
about number four there,

167
00:08:36,600 --> 00:08:39,630
Lady Lovelace's objection.

168
00:08:39,630 --> 00:08:44,119
She was, as you all know,
the sponsor and programmer

169
00:08:44,119 --> 00:08:46,410
of Charles Babbage when he
attempted to make a computer

170
00:08:46,410 --> 00:08:48,200
mechanically.

171
00:08:48,200 --> 00:08:51,054
And what she said,
she was obviously

172
00:08:51,054 --> 00:08:53,470
pestered with the same kind
of things that everybody in AI

173
00:08:53,470 --> 00:08:55,000
is always pestered with.

174
00:08:55,000 --> 00:08:58,620
And at one point, she was
reported to have said--

175
00:08:58,620 --> 00:09:00,807
let me put it in my patois.

176
00:09:00,807 --> 00:09:01,890
Don't worry about a thing.

177
00:09:01,890 --> 00:09:03,848
They can only do what
they're programmed to do.

178
00:09:06,410 --> 00:09:08,300
And of course, what
she should have said

179
00:09:08,300 --> 00:09:11,900
is that they can only do what
they've been programmed to do

180
00:09:11,900 --> 00:09:14,720
and what we've taught them
to do and what they've

181
00:09:14,720 --> 00:09:17,474
learned how to do on their own.

182
00:09:17,474 --> 00:09:19,640
But maybe that wouldn't
have had the soothing effect

183
00:09:19,640 --> 00:09:22,528
she was looking for.

184
00:09:22,528 --> 00:09:27,390
In any event, this is when
people started thinking about

185
00:09:27,390 --> 00:09:30,590
whether computers could think.

186
00:09:30,590 --> 00:09:33,860
But it's not the first time
people thought about thinking,

187
00:09:33,860 --> 00:09:39,960
that we have to go back 2,400
years or so to get to that.

188
00:09:39,960 --> 00:09:44,390
And when we do, we think about
Plato's most famous work,

189
00:09:44,390 --> 00:09:47,720
The Republic, which
was clearly a metaphor

190
00:09:47,720 --> 00:09:52,330
to what goes on in our brains
and our minds and our thinking.

191
00:09:52,330 --> 00:09:54,380
He couched it in
terms of a metaphor

192
00:09:54,380 --> 00:09:57,029
with how a state is
organized with philosopher

193
00:09:57,029 --> 00:09:58,820
kings and merchants
and soldiers and stuff.

194
00:09:58,820 --> 00:10:00,444
But he was clearly
talking about a kind

195
00:10:00,444 --> 00:10:04,010
of theory of brain-mind
thinking that

196
00:10:04,010 --> 00:10:06,170
suggested there are
agents in there that are

197
00:10:06,170 --> 00:10:08,780
kind of all working together.

198
00:10:08,780 --> 00:10:12,110
But it's important
to note, I think,

199
00:10:12,110 --> 00:10:17,630
that The Republic is a good
translation of the Latin de re

200
00:10:17,630 --> 00:10:20,120
publica, which is
a bad translation

201
00:10:20,120 --> 00:10:23,150
of the Greek politeia.

202
00:10:23,150 --> 00:10:27,080
And politeia, interestingly,
is a Greek word

203
00:10:27,080 --> 00:10:31,040
that my Greek friends
tell me is untranslatable.

204
00:10:31,040 --> 00:10:34,970
But it means something
like a society or community

205
00:10:34,970 --> 00:10:36,920
or something like that.

206
00:10:36,920 --> 00:10:39,620
And the book was about the mind.

207
00:10:39,620 --> 00:10:43,040
So Plato-- it could
have been translated

208
00:10:43,040 --> 00:10:47,230
as the society of
mind, in which case

209
00:10:47,230 --> 00:10:49,040
it would have anticipated
Marvin Minsky's

210
00:10:49,040 --> 00:10:53,390
book with the same
title by 2,400 years.

211
00:10:53,390 --> 00:10:57,480
Well, maybe that was
not the first time

212
00:10:57,480 --> 00:10:59,420
that humans thought
about thinking,

213
00:10:59,420 --> 00:11:03,800
but it was an early landmark.

214
00:11:03,800 --> 00:11:06,440
And now it's, I think,
useful to go back to when

215
00:11:06,440 --> 00:11:08,380
humans started thinking.

216
00:11:08,380 --> 00:11:12,620
And that takes us back
about 50,000 years--

217
00:11:12,620 --> 00:11:14,450
not millions of years--

218
00:11:14,450 --> 00:11:17,480
a few tens of
thousands of years.

219
00:11:17,480 --> 00:11:20,030
So it probably happened
in southern Africa.

220
00:11:20,030 --> 00:11:22,250
It was probably 60,000
or 70,000 years ago

221
00:11:22,250 --> 00:11:24,860
that it started happening.

222
00:11:24,860 --> 00:11:27,729
It probably happened in
the neck-down population,

223
00:11:27,729 --> 00:11:29,270
because if the
population is too big,

224
00:11:29,270 --> 00:11:32,920
an innovation can't take hold.

225
00:11:32,920 --> 00:11:36,560
It was about that
time that people--

226
00:11:36,560 --> 00:11:40,610
us, we-- started drilling
holes in sea shells,

227
00:11:40,610 --> 00:11:43,154
presumably for making jewelry.

228
00:11:43,154 --> 00:11:44,570
And then it wasn't
long after that

229
00:11:44,570 --> 00:11:49,130
that we departed from those
Neanderthal guys in a big way.

230
00:11:49,130 --> 00:11:51,220
And we started painting
caves like the ones

231
00:11:51,220 --> 00:11:56,980
at Lascaux, carving figurines
like the one at Brassempouy.

232
00:11:56,980 --> 00:12:01,790
And I think the most important
question we can ask in AI

233
00:12:01,790 --> 00:12:04,970
is what makes us different
from that Neanderthal who

234
00:12:04,970 --> 00:12:07,382
couldn't do these things?

235
00:12:07,382 --> 00:12:09,340
See, that's not the
question that Turing asked.

236
00:12:09,340 --> 00:12:11,050
The question Turing
asked was how can we

237
00:12:11,050 --> 00:12:13,367
make a computer reason?

238
00:12:13,367 --> 00:12:14,950
Because as a
mathematician, he thought

239
00:12:14,950 --> 00:12:18,660
that was a kind of supreme
capability of human thinking.

240
00:12:18,660 --> 00:12:22,330
And so for 20, 30
years, AI people

241
00:12:22,330 --> 00:12:26,542
focused on reasoning
as the center of AI.

242
00:12:26,542 --> 00:12:28,250
And what they should
have been asking is,

243
00:12:28,250 --> 00:12:31,870
what makes us different from
the Neanderthals and chimpanzees

244
00:12:31,870 --> 00:12:34,180
and other species?

245
00:12:34,180 --> 00:12:38,120
It creates a different
research agenda.

246
00:12:38,120 --> 00:12:41,890
Well, I'm much influenced by
the paleoanthropologist, Ian

247
00:12:41,890 --> 00:12:44,320
Tattersall, who writes
extensively about this

248
00:12:44,320 --> 00:12:46,870
and says that it
didn't evolve, it

249
00:12:46,870 --> 00:12:50,770
was more of a discovery
than an evolution.

250
00:12:50,770 --> 00:12:53,770
Our brains came to be
what they are for reasons

251
00:12:53,770 --> 00:12:56,990
other than human intelligence.

252
00:12:56,990 --> 00:13:00,370
So he thinks of it
as a minor change

253
00:13:00,370 --> 00:13:04,260
or even a discovery of
something we didn't know we had.

254
00:13:04,260 --> 00:13:10,540
In any event, he talks
about becoming symbolic.

255
00:13:10,540 --> 00:13:12,290
But of course, as a
paleoanthropologist

256
00:13:12,290 --> 00:13:14,680
and not a computationalist,
he doesn't

257
00:13:14,680 --> 00:13:18,520
have the vocabulary
for talking about that

258
00:13:18,520 --> 00:13:21,670
in computational terms.

259
00:13:21,670 --> 00:13:25,480
So you have to go to
someone like Noam Chomsky

260
00:13:25,480 --> 00:13:28,360
to get a more computational
perspective on this.

261
00:13:28,360 --> 00:13:30,570
And what Chomsky says is that--

262
00:13:30,570 --> 00:13:33,270
who is also, by the way,
a fan of Tattersall--

263
00:13:33,270 --> 00:13:35,560
what Chomsky says is
that what happened

264
00:13:35,560 --> 00:13:38,980
is that we acquired the
ability to take two concepts

265
00:13:38,980 --> 00:13:41,107
and put them together
and make a third concept

266
00:13:41,107 --> 00:13:42,190
and do that without limit.

267
00:13:45,480 --> 00:13:47,410
An AI person would say,
oh, Chomsky's talking

268
00:13:47,410 --> 00:13:48,346
about semantic nets.

269
00:13:48,346 --> 00:13:49,720
A linguist would
say he's talking

270
00:13:49,720 --> 00:13:50,710
about the merge operation.

271
00:13:50,710 --> 00:13:51,710
But it's the same thing.

272
00:13:56,060 --> 00:13:59,750
As an aside, I'll tell you that
a very important book will come

273
00:13:59,750 --> 00:14:02,960
out in January, I think,
by Berwick and Chomsky

274
00:14:02,960 --> 00:14:05,670
and addresses two questions--

275
00:14:05,670 --> 00:14:08,840
why do we have any language and
why do we have more than one?

276
00:14:08,840 --> 00:14:11,030
You know, when you think
about it, it's weird.

277
00:14:11,030 --> 00:14:13,520
Why should we have a language
and now that we have one,

278
00:14:13,520 --> 00:14:15,740
why should we all
have different ones?

279
00:14:15,740 --> 00:14:20,180
And their answer is roughly
that this innovation

280
00:14:20,180 --> 00:14:22,170
made language possible.

281
00:14:22,170 --> 00:14:26,240
But once you've
got the competence,

282
00:14:26,240 --> 00:14:30,380
it can manifest itself in
many engineering solutions.

283
00:14:30,380 --> 00:14:34,190
They also talk a
lot about how we're

284
00:14:34,190 --> 00:14:36,470
different from other
species, and they

285
00:14:36,470 --> 00:14:38,900
like to talk about the
fact that we can think

286
00:14:38,900 --> 00:14:41,480
about stuff that isn't there.

287
00:14:41,480 --> 00:14:43,610
So we can think about
apples even when

288
00:14:43,610 --> 00:14:46,460
we're not looking at an apple.

289
00:14:46,460 --> 00:14:49,406
But back to the main line,
when Spelke talked to you,

290
00:14:49,406 --> 00:14:50,780
she didn't talk
to you about what

291
00:14:50,780 --> 00:14:53,270
I consider to be the
greatest experiment

292
00:14:53,270 --> 00:14:55,640
in developmental
psychology ever,

293
00:14:55,640 --> 00:14:58,340
even though it wasn't
necessarily-- well,

294
00:14:58,340 --> 00:14:59,240
I confused myself.

295
00:14:59,240 --> 00:15:01,090
Let me tell you
about the experiment.

296
00:15:01,090 --> 00:15:04,430
Spelke doesn't do
rats, but other people

297
00:15:04,430 --> 00:15:06,980
do rats with the
following observation--

298
00:15:06,980 --> 00:15:10,385
take a rectangular
room and there

299
00:15:10,385 --> 00:15:14,120
are hiding places in all four
corners that are identical.

300
00:15:14,120 --> 00:15:16,010
While the rat is
watching you put food

301
00:15:16,010 --> 00:15:20,790
in one of these places,
a box cloth over it,

302
00:15:20,790 --> 00:15:24,560
and then you disorient the
rat by spinning it around.

303
00:15:24,560 --> 00:15:27,170
And you watch what the rat does.

304
00:15:27,170 --> 00:15:28,730
And rats are pretty smart.

305
00:15:28,730 --> 00:15:29,730
They do the right thing.

306
00:15:29,730 --> 00:15:33,530
Those opposite corners are
the right answer, right?

307
00:15:33,530 --> 00:15:35,120
Because the room is
rectangular, those

308
00:15:35,120 --> 00:15:40,100
are the two possible places
that the food could be.

309
00:15:40,100 --> 00:15:43,700
So then you can repeat this
experiment with a small child,

310
00:15:43,700 --> 00:15:48,170
you get the same answer, or with
a intelligent adult like me,

311
00:15:48,170 --> 00:15:51,920
and you get the same answer
because it's the right answer.

312
00:15:51,920 --> 00:15:57,680
But now the next thing is
you paint one wall blue

313
00:15:57,680 --> 00:16:00,390
and repeat the experiment.

314
00:16:00,390 --> 00:16:03,320
What do you think the rat does?

315
00:16:03,320 --> 00:16:06,090
Both ways.

316
00:16:06,090 --> 00:16:12,100
You repeat the experiment
with a small child, both ways.

317
00:16:12,100 --> 00:16:15,320
Repeat the experiment with me.

318
00:16:15,320 --> 00:16:17,770
Finally we get it right.

319
00:16:17,770 --> 00:16:21,431
What's the difference?

320
00:16:21,431 --> 00:16:22,430
And when does it happen?

321
00:16:22,430 --> 00:16:26,270
When does this small
child become an adult?

322
00:16:26,270 --> 00:16:28,480
After elaborate and
careful experiments

323
00:16:28,480 --> 00:16:31,810
of the kind that
Spelke is noted for,

324
00:16:31,810 --> 00:16:34,930
she has determined that the
onset of this capability

325
00:16:34,930 --> 00:16:37,570
arises when the child
starts using the words left

326
00:16:37,570 --> 00:16:41,299
and right in their own
descriptions of the world.

327
00:16:41,299 --> 00:16:43,090
They understand left
and right before that,

328
00:16:43,090 --> 00:16:46,940
but this is when they
start using those words.

329
00:16:46,940 --> 00:16:49,880
That's when it happens.

330
00:16:49,880 --> 00:16:53,130
Now we introduce the
notion of verbal shadowing.

331
00:16:53,130 --> 00:16:57,260
So I read to you the Declaration
of Independence or something

332
00:16:57,260 --> 00:17:00,599
else and as I say it,
you say it back to me.

333
00:17:00,599 --> 00:17:02,390
It's sort of like
simultaneous translation,

334
00:17:02,390 --> 00:17:05,329
only it's English to English.

335
00:17:05,329 --> 00:17:08,390
And now you take an adult
human, and even while they're

336
00:17:08,390 --> 00:17:12,349
walking into the room, they're
doing this verbal shadowing.

337
00:17:12,349 --> 00:17:16,190
And what happens in
that circumstance?

338
00:17:16,190 --> 00:17:19,910
That reduces the adult
to the level of a rat.

339
00:17:19,910 --> 00:17:21,329
They can't do it.

340
00:17:21,329 --> 00:17:23,329
And you say, well, didn't
you see the blue wall?

341
00:17:23,329 --> 00:17:28,770
And they'll say, yeah, I saw the
blue wall but couldn't use it.

342
00:17:28,770 --> 00:17:32,880
So Spelke's
interpretation of this

343
00:17:32,880 --> 00:17:36,550
is that the words have
jammed the processor.

344
00:17:36,550 --> 00:17:39,660
That's why we don't use our
laptops in class, right,

345
00:17:39,660 --> 00:17:42,390
because we only have
one language processor,

346
00:17:42,390 --> 00:17:43,920
and it can be jammed.

347
00:17:43,920 --> 00:17:44,870
It's jammed by email.

348
00:17:44,870 --> 00:17:48,170
I'm jammed by used car
salesmen talking fast.

349
00:17:48,170 --> 00:17:49,450
It's easy to jam it.

350
00:17:49,450 --> 00:17:51,634
And when we jam it,
it can't do what

351
00:17:51,634 --> 00:17:52,800
you would think it could do.

352
00:17:59,380 --> 00:18:01,180
So Spelke has an
interpretation to this

353
00:18:01,180 --> 00:18:04,090
that says what we humans have
is combinators, the ability

354
00:18:04,090 --> 00:18:07,324
to take formation of different
kinds and put it together.

355
00:18:07,324 --> 00:18:08,740
I have a different
interpretation,

356
00:18:08,740 --> 00:18:11,980
which I'll tell you
about at the end

357
00:18:11,980 --> 00:18:15,259
if you ask me why I
think Spelke has it--

358
00:18:15,259 --> 00:18:17,300
why I have a different
interpretation from Spelke

359
00:18:17,300 --> 00:18:21,560
for this experiment.

360
00:18:21,560 --> 00:18:23,560
And then what we've got
is we've got the ability

361
00:18:23,560 --> 00:18:25,750
to build descriptions that
seem to be the defining

362
00:18:25,750 --> 00:18:28,000
characteristic of
human intelligence.

363
00:18:28,000 --> 00:18:30,564
We've got it in the
case at Lascaux.

364
00:18:30,564 --> 00:18:32,230
We've got it in the
thoughts of Chomsky.

365
00:18:32,230 --> 00:18:34,330
We've got it in these
experiments of Spelke.

366
00:18:34,330 --> 00:18:35,290
We've got descriptions.

367
00:18:35,290 --> 00:18:39,640
And so those are the influences
that led me to this thing

368
00:18:39,640 --> 00:18:42,364
I call the strong
story hypothesis.

369
00:18:42,364 --> 00:18:44,530
And when you think about
it, almost all of education

370
00:18:44,530 --> 00:18:47,110
is about stories.

371
00:18:47,110 --> 00:18:49,780
You know, you start with
fairy tales that keep you

372
00:18:49,780 --> 00:18:51,700
from running away in the mall.

373
00:18:51,700 --> 00:18:55,287
You'll be eaten by big
bad wolf if you do.

374
00:18:55,287 --> 00:18:57,370
And you end up with all
these professional schools

375
00:18:57,370 --> 00:19:00,470
that people go to--
law, business, medicine,

376
00:19:00,470 --> 00:19:02,920
and even engineering.

377
00:19:02,920 --> 00:19:05,200
You might say, well,
engineering, that's not

378
00:19:05,200 --> 00:19:06,670
really-- is that case studies?

379
00:19:06,670 --> 00:19:09,610
And the answer is, if
you talk to somebody

380
00:19:09,610 --> 00:19:12,550
that knows what they're
doing, what they do

381
00:19:12,550 --> 00:19:16,030
is very often telling a story.

382
00:19:16,030 --> 00:19:19,060
My friend, Gerry Sussman,
a computer scientist

383
00:19:19,060 --> 00:19:23,830
whose work I use, is fond
of teaching circuit theory

384
00:19:23,830 --> 00:19:26,110
as a hobby.

385
00:19:26,110 --> 00:19:28,670
And when you hear him
talk about this circuit,

386
00:19:28,670 --> 00:19:31,210
he talks about a signal
coming in from the left

387
00:19:31,210 --> 00:19:33,775
and migrating through
that capacitor

388
00:19:33,775 --> 00:19:37,950
and going into the
base of a transistor

389
00:19:37,950 --> 00:19:40,990
and causing a voltage drop
across the emitter, which

390
00:19:40,990 --> 00:19:44,040
creates a current that
flows into the collector,

391
00:19:44,040 --> 00:19:45,580
and that causes--

392
00:19:45,580 --> 00:19:48,910
he's just basically telling the
story of how that signal flows

393
00:19:48,910 --> 00:19:49,860
through the network.

394
00:19:49,860 --> 00:19:51,028
It's storytelling.

395
00:19:54,310 --> 00:19:56,100
So if you believe
that, then these

396
00:19:56,100 --> 00:20:01,140
are the steps that were
prescribed by Marr and Tomaso

397
00:20:01,140 --> 00:20:08,314
as well in the early days
of their presence at MIT.

398
00:20:08,314 --> 00:20:10,980
These are the things you need to
do if you believe that and want

399
00:20:10,980 --> 00:20:12,021
to do something about it.

400
00:20:15,200 --> 00:20:18,870
And these steps were
articulated at the time,

401
00:20:18,870 --> 00:20:20,940
in part because people in
artificial intelligence

402
00:20:20,940 --> 00:20:25,110
were, in Marr's words,
too mechanistic.

403
00:20:25,110 --> 00:20:27,360
I talked about this
on the first day,

404
00:20:27,360 --> 00:20:30,420
that people would fall in love
with a particular mechanism--

405
00:20:30,420 --> 00:20:34,080
a hammer-- and try to use
it for everything instead

406
00:20:34,080 --> 00:20:35,580
of understanding
what the problem is

407
00:20:35,580 --> 00:20:39,210
before you select the
tools to bring to bear

408
00:20:39,210 --> 00:20:41,040
on producing a solution.

409
00:20:43,590 --> 00:20:46,971
And so being an engineer,
one of the steps here

410
00:20:46,971 --> 00:20:48,720
that I'm particularly
fond of, once you've

411
00:20:48,720 --> 00:20:52,680
got the articulated
behavior 100%,

412
00:20:52,680 --> 00:20:54,591
eventually you have
to build something.

413
00:20:54,591 --> 00:20:57,090
Because as an engineer, I think
I don't really understand it

414
00:20:57,090 --> 00:20:59,532
unless I can build it.

415
00:20:59,532 --> 00:21:00,990
And then building
it, things emerge

416
00:21:00,990 --> 00:21:02,615
that I wouldn't have
thought about if I

417
00:21:02,615 --> 00:21:04,930
hadn't tried to build it.

418
00:21:04,930 --> 00:21:06,240
Well, anyhow, let's see.

419
00:21:06,240 --> 00:21:08,310
Step one, characterize
the behavior.

420
00:21:08,310 --> 00:21:10,734
The behavior has to
story understanding.

421
00:21:10,734 --> 00:21:12,150
So I'm going to
need some stories.

422
00:21:12,150 --> 00:21:16,980
And so I tend to work
with short summaries

423
00:21:16,980 --> 00:21:21,640
of Shakespearean plays,
medical cases, cyber

424
00:21:21,640 --> 00:21:28,660
warfare, classical social
studies, and psychology.

425
00:21:31,240 --> 00:21:34,950
And these stories are
written by us so as

426
00:21:34,950 --> 00:21:36,600
to get through Boris's parser.

427
00:21:39,260 --> 00:21:43,220
So they are carefully prepared.

428
00:21:43,220 --> 00:21:45,380
But they're human readable.

429
00:21:45,380 --> 00:21:47,310
We're not encoding
this stuff up.

430
00:21:47,310 --> 00:21:49,060
This is the sort of
thing you could read,

431
00:21:49,060 --> 00:21:52,200
and if you read that you say,
yeah, this is kind of funny,

432
00:21:52,200 --> 00:21:54,250
but you can understand it.

433
00:21:54,250 --> 00:21:56,910
Summary of Macbeth.

434
00:21:56,910 --> 00:22:00,840
Here is a little fragment of
it, and it is easier to read.

435
00:22:03,650 --> 00:22:04,820
So what do we want to do?

436
00:22:04,820 --> 00:22:07,270
What can we bring to bear
on understanding the story?

437
00:22:07,270 --> 00:22:09,310
If you read that
story, you'd see--

438
00:22:09,310 --> 00:22:14,390
I could ask you a question,
is Duncan dead at the end?

439
00:22:14,390 --> 00:22:15,640
And how would you know?

440
00:22:15,640 --> 00:22:17,890
It doesn't say he's
dead at the end.

441
00:22:17,890 --> 00:22:21,640
He was murdered, but--

442
00:22:21,640 --> 00:22:25,100
I could ask you is this
a story about revenge?

443
00:22:25,100 --> 00:22:27,427
The word revenge
is never mentioned,

444
00:22:27,427 --> 00:22:29,260
and you have to think
about it a little bit.

445
00:22:29,260 --> 00:22:30,910
But you'd probably
conclude in the end

446
00:22:30,910 --> 00:22:33,390
that it's about revenge.

447
00:22:33,390 --> 00:22:35,620
So now we ask ourselves
what kinds of knowledge

448
00:22:35,620 --> 00:22:41,380
is required to know that
Duncan is dead at the end

449
00:22:41,380 --> 00:22:44,410
and that it's about revenge?

450
00:22:44,410 --> 00:22:47,870
Well, first of all, we
need some common sense.

451
00:22:47,870 --> 00:22:50,710
And what we've found,
somewhat to our surprise,

452
00:22:50,710 --> 00:22:52,810
is that much of that can
be expressed in terms

453
00:22:52,810 --> 00:22:56,550
of simple if-then rules.

454
00:22:56,550 --> 00:23:04,600
And these seven rule types
arose because people building

455
00:23:04,600 --> 00:23:08,500
software to understand stories
found that they were necessary.

456
00:23:08,500 --> 00:23:11,170
We knew we needed
the first kind.

457
00:23:13,690 --> 00:23:16,210
If you kill someone,
then they are dead.

458
00:23:16,210 --> 00:23:19,970
Every other one here arose
because we reached an impasse

459
00:23:19,970 --> 00:23:23,860
in our construction of our
story understanding system.

460
00:23:23,860 --> 00:23:25,360
So the may rules.

461
00:23:25,360 --> 00:23:29,470
If I anger you, you may kill me.

462
00:23:29,470 --> 00:23:32,680
Thank god we don't always
kill people who anger us.

463
00:23:32,680 --> 00:23:37,840
But we humans always are
searching for explanations.

464
00:23:37,840 --> 00:23:40,510
So if you kill me, and I've
previously angered you,

465
00:23:40,510 --> 00:23:43,660
and you can't think of any other
reason for why the killing took

466
00:23:43,660 --> 00:23:47,140
place, then the
anger is supposed.

467
00:23:47,140 --> 00:23:51,850
So that's the explanation
for rule type number two.

468
00:23:51,850 --> 00:23:56,160
Sometimes we use abduction.

469
00:23:56,160 --> 00:23:58,660
You might have a firm belief
that anybody who kills somebody

470
00:23:58,660 --> 00:23:59,290
is crazy.

471
00:23:59,290 --> 00:24:01,210
That's abduction.

472
00:24:01,210 --> 00:24:03,940
You're presuming the
antecedent from the presence

473
00:24:03,940 --> 00:24:05,098
of a consequent.

474
00:24:10,580 --> 00:24:13,340
So those are kinds of rules
that work in the background

475
00:24:13,340 --> 00:24:14,720
to deal with the story.

476
00:24:14,720 --> 00:24:16,610
And of course, there
are things that

477
00:24:16,610 --> 00:24:18,789
are explicit in the story too.

478
00:24:18,789 --> 00:24:20,330
Here are some examples
of things that

479
00:24:20,330 --> 00:24:23,830
are-- of causal relations that
are explicit in the story.

480
00:24:23,830 --> 00:24:26,460
The first kind says that this
happens because that happened.

481
00:24:26,460 --> 00:24:28,640
A close, tight
causal connection.

482
00:24:28,640 --> 00:24:31,225
Second kind, we know
there's a causal connection,

483
00:24:31,225 --> 00:24:32,225
but it might be lengthy.

484
00:24:35,550 --> 00:24:37,560
The third kind,
the strangely kind,

485
00:24:37,560 --> 00:24:40,140
arose when one of
my students was

486
00:24:40,140 --> 00:24:45,390
working on Crow creation myths.

487
00:24:45,390 --> 00:24:48,440
He happened to be a
Crow Indian and so

488
00:24:48,440 --> 00:24:51,060
a natural interest
in that mythology.

489
00:24:51,060 --> 00:24:54,810
And what he noted was
that in Crow mythology,

490
00:24:54,810 --> 00:24:58,080
you're often told that
something is connected causally

491
00:24:58,080 --> 00:25:02,190
and also told that you'll
never understand it.

492
00:25:02,190 --> 00:25:04,860
Old Man Coyote
reached into the lake

493
00:25:04,860 --> 00:25:07,236
and pulled up a handful
of mud and made the world,

494
00:25:07,236 --> 00:25:08,610
and you will never
understand how

495
00:25:08,610 --> 00:25:13,840
that happened, is a kind of
typical expression in Crow

496
00:25:13,840 --> 00:25:15,210
creation mythology.

497
00:25:15,210 --> 00:25:18,150
So all of these arose because
we were trying to understand

498
00:25:18,150 --> 00:25:20,890
particular kinds of stories.

499
00:25:20,890 --> 00:25:21,390
OK.

500
00:25:21,390 --> 00:25:23,220
So that's all background.

501
00:25:23,220 --> 00:25:27,710
Here it is in operation.

502
00:25:27,710 --> 00:25:30,420
And that's about the
speed that it goes.

503
00:25:30,420 --> 00:25:32,310
That is reading the story--

504
00:25:32,310 --> 00:25:33,900
the summary of
Macbeth that you saw

505
00:25:33,900 --> 00:25:37,050
on the screen a few moments ago.

506
00:25:37,050 --> 00:25:39,230
But of course, it's
invisible at that scale.

507
00:25:39,230 --> 00:25:42,730
So let me blow a piece of it up.

508
00:25:42,730 --> 00:25:47,360
There you see the piece that
says, oh, Macbeth murdered

509
00:25:47,360 --> 00:25:47,960
Duncan.

510
00:25:47,960 --> 00:25:49,180
Duncan becomes dead.

511
00:25:49,180 --> 00:25:55,250
So the yellow parts are inserted
by background knowledge.

512
00:25:55,250 --> 00:25:58,850
The white parts are
explicit in the story.

513
00:26:02,890 --> 00:26:04,150
So you may have also noted--

514
00:26:04,150 --> 00:26:05,608
no, you wouldn't
have noted-- well,

515
00:26:05,608 --> 00:26:07,210
my drawing attention to it.

516
00:26:07,210 --> 00:26:09,760
We have not only the concept
that the yellow parts--

517
00:26:14,410 --> 00:26:17,760
yes, the yellow parts
there are conclusions.

518
00:26:17,760 --> 00:26:19,522
The white parts are explicit.

519
00:26:19,522 --> 00:26:20,980
And what you can
see, incidentally,

520
00:26:20,980 --> 00:26:23,140
is that-- just from
the colors-- that much

521
00:26:23,140 --> 00:26:24,580
of the understanding
of the story

522
00:26:24,580 --> 00:26:28,487
is inferred from what is there.

523
00:26:28,487 --> 00:26:30,070
It's a funny kind
of way of saying it,

524
00:26:30,070 --> 00:26:32,740
but you've seen
in computer vision

525
00:26:32,740 --> 00:26:34,290
or you will see
in computer vision

526
00:26:34,290 --> 00:26:39,006
that what you think you
see is half hallucination,

527
00:26:39,006 --> 00:26:40,630
and what you think
you see in the story

528
00:26:40,630 --> 00:26:43,252
is also half hallucinated.

529
00:26:43,252 --> 00:26:44,710
It seems that the
authors just tell

530
00:26:44,710 --> 00:26:47,850
us enough to keep us on track.

531
00:26:47,850 --> 00:26:50,860
In any event, we have
not only the yellow parts

532
00:26:50,860 --> 00:26:52,900
that are inferred,
but we also have

533
00:26:52,900 --> 00:26:55,974
the observation that one piece
may be connected to another.

534
00:26:55,974 --> 00:26:57,640
And that can only be
determined by doing

535
00:26:57,640 --> 00:27:01,100
a search through that
so-called elaboration graph.

536
00:27:01,100 --> 00:27:05,200
So here are the same sorts of
things that you can search for.

537
00:27:05,200 --> 00:27:07,630
There's a definition
of a Pyrrhic victory.

538
00:27:07,630 --> 00:27:11,310
You do something or rather you
want something at least you

539
00:27:11,310 --> 00:27:13,780
becoming happy, but
ultimately the same wanting

540
00:27:13,780 --> 00:27:17,440
leads to disaster.

541
00:27:17,440 --> 00:27:18,110
So there it is.

542
00:27:18,110 --> 00:27:20,530
That green thing down there is
reflected in the green elements

543
00:27:20,530 --> 00:27:22,570
up there that are picked
out of the entire graph

544
00:27:22,570 --> 00:27:23,653
because they're connected.

545
00:27:23,653 --> 00:27:26,680
And I'll show you how they're
connected in this one.

546
00:27:26,680 --> 00:27:29,170
So this is the Pyrrhic
victory concept

547
00:27:29,170 --> 00:27:32,560
that has been extracted from
the story by a search program.

548
00:27:32,560 --> 00:27:35,560
So we start with Macbeth
wanting to be king.

549
00:27:35,560 --> 00:27:37,240
He murders Duncan
because of that.

550
00:27:37,240 --> 00:27:39,250
He becomes happy,
because he eventually

551
00:27:39,250 --> 00:27:43,130
ends up being king himself,
but downstream he's harmed.

552
00:27:43,130 --> 00:27:44,902
So it's a kind of
Pyrrhic victory.

553
00:27:44,902 --> 00:27:46,360
And now you say to
me, well, that's

554
00:27:46,360 --> 00:27:47,901
not my definition
of Pyrrhic victory,

555
00:27:47,901 --> 00:27:51,830
and that's OK, because
we all have nuanced

556
00:27:51,830 --> 00:27:53,960
differences in our concepts.

557
00:27:53,960 --> 00:27:56,380
So this is just
one computer's idea

558
00:27:56,380 --> 00:27:59,974
of what a Pyrrhic victory is.

559
00:27:59,974 --> 00:28:01,390
Here are the kinds
of things we've

560
00:28:01,390 --> 00:28:05,120
been able to do as a
consequence of, to our surprise,

561
00:28:05,120 --> 00:28:08,471
just having a suite of rules
and a suite of concepts.

562
00:28:08,471 --> 00:28:10,720
And what I'm going to spend
the next few minutes doing

563
00:28:10,720 --> 00:28:13,480
is just taking you quickly
through a few examples

564
00:28:13,480 --> 00:28:15,070
of these kinds of things.

565
00:28:19,800 --> 00:28:22,980
This is reading Macbeth from
two different cultural points

566
00:28:22,980 --> 00:28:26,940
of view, an Asian point of
view and a US point of view.

567
00:28:26,940 --> 00:28:28,680
There were some
fabulous experiments

568
00:28:28,680 --> 00:28:33,570
conducted in the
'90s in a high school

569
00:28:33,570 --> 00:28:38,490
in the outskirts of Beijing and
in a high school in Wisconsin.

570
00:28:38,490 --> 00:28:42,390
And these experiments involved
having the students read

571
00:28:42,390 --> 00:28:45,240
stories about
violence and observing

572
00:28:45,240 --> 00:28:48,480
the reaction of the
students to those stories.

573
00:28:48,480 --> 00:28:53,520
And what they found was that
at a statistically significant

574
00:28:53,520 --> 00:28:57,870
level, not this or that, but
a statistically significant

575
00:28:57,870 --> 00:29:00,780
level, the Asian students
outside of Beijing

576
00:29:00,780 --> 00:29:03,920
attributed violence
to situations.

577
00:29:03,920 --> 00:29:08,190
And they would ask, what made
that person want to do that?

578
00:29:08,190 --> 00:29:12,450
Whereas the kids in Wisconsin
had a greater tendency to say,

579
00:29:12,450 --> 00:29:15,946
that person must be
completely crazy.

580
00:29:15,946 --> 00:29:17,820
So one attributed to
the situation, the other

581
00:29:17,820 --> 00:29:24,790
was dispositional, to
use the technical term.

582
00:29:24,790 --> 00:29:28,080
So here we see in one
reading of Macbeth--

583
00:29:28,080 --> 00:29:29,600
let me show it blown up.

584
00:29:29,600 --> 00:29:34,350
In the top version,
Macduff kills Macbeth

585
00:29:34,350 --> 00:29:38,620
because there's a revenge
situation that forces it.

586
00:29:38,620 --> 00:29:42,090
And the other interpretation
is because Macduff is crazy.

587
00:29:44,970 --> 00:29:48,490
So another kind of
similar pairing--

588
00:29:48,490 --> 00:29:48,990
oh, wow.

589
00:29:48,990 --> 00:29:50,190
That was fast.

590
00:29:50,190 --> 00:29:54,650
This is a story about the
Estonian Russian cyber

591
00:29:54,650 --> 00:29:56,179
war of 2007.

592
00:29:56,179 --> 00:29:58,720
Could you-- you probably didn't
hear the rules of engagement,

593
00:29:58,720 --> 00:30:00,600
but I don't talk to
the back of laptops.

594
00:30:00,600 --> 00:30:02,670
So if you'd put that
away, I'd appreciate it.

595
00:30:02,670 --> 00:30:12,360
So in 2007, the Estonians
moved a war memorial

596
00:30:12,360 --> 00:30:15,060
from the Soviet era out
of the center of town

597
00:30:15,060 --> 00:30:18,060
to a cemetery in the outskirts.

598
00:30:18,060 --> 00:30:23,700
And about 30% of the Estonian
population is Russian,

599
00:30:23,700 --> 00:30:28,362
and they were irritated by this.

600
00:30:28,362 --> 00:30:32,340
And it had never been
proven, but the next day

601
00:30:32,340 --> 00:30:35,790
the Estonian National
Network went down,

602
00:30:35,790 --> 00:30:38,930
and government
websites were defaced.

603
00:30:38,930 --> 00:30:41,490
And this was hurtful, because
the Estonians pride themselves

604
00:30:41,490 --> 00:30:44,220
in being very
technically advanced.

605
00:30:44,220 --> 00:30:48,330
In Estonia, it's a
right to be educated

606
00:30:48,330 --> 00:30:51,540
on how to use the internet.

607
00:30:51,540 --> 00:30:53,820
They have a national ID card.

608
00:30:53,820 --> 00:30:56,530
They have a law that says if
anybody looks at your data,

609
00:30:56,530 --> 00:30:57,960
they've got to explain why.

610
00:30:57,960 --> 00:31:01,690
They're a very technically
sophisticated country.

611
00:31:01,690 --> 00:31:06,570
And so what's the interpretation
of this attack, which

612
00:31:06,570 --> 00:31:09,630
was presumed to be done
by either ethnic Russians

613
00:31:09,630 --> 00:31:12,410
in Estonia or by
people from Russia?

614
00:31:12,410 --> 00:31:16,860
Well, was it an
aggressive revenge or

615
00:31:16,860 --> 00:31:20,490
was it teaching the
Estonians a lesson?

616
00:31:20,490 --> 00:31:23,460
It depends on what?

617
00:31:23,460 --> 00:31:25,855
It depends on whose
side you're on.

618
00:31:25,855 --> 00:31:26,980
That's the only difference.

619
00:31:26,980 --> 00:31:29,040
And that's what produced the
difference in interpretation

620
00:31:29,040 --> 00:31:30,810
on those two sides--
one being aggressive

621
00:31:30,810 --> 00:31:33,420
revenge and the
other being teaching

622
00:31:33,420 --> 00:31:34,450
the Estonians a lesson.

623
00:31:34,450 --> 00:31:38,340
By the way, I was in
Estonia in January.

624
00:31:38,340 --> 00:31:39,910
That's the statue
that wasn't there.

625
00:31:43,270 --> 00:31:44,870
Give you another example.

626
00:31:44,870 --> 00:31:48,230
I'm just trying to show you
some of the breadth of our story

627
00:31:48,230 --> 00:31:50,390
understanding activity.

628
00:31:50,390 --> 00:31:53,890
So the next example comes
about because shortly

629
00:31:53,890 --> 00:31:57,370
after the terrorist attack
on the World Trade Center,

630
00:31:57,370 --> 00:32:00,880
there was a strong
interest in bringing

631
00:32:00,880 --> 00:32:03,220
political science and
artificial intelligence

632
00:32:03,220 --> 00:32:08,830
together to make it
possible to understand how

633
00:32:08,830 --> 00:32:11,950
other people think when
they're not necessarily crazy,

634
00:32:11,950 --> 00:32:14,770
they've just got
different backgrounds.

635
00:32:14,770 --> 00:32:17,130
The thesis is that we are
the stories in our culture.

636
00:32:20,290 --> 00:32:24,360
So I was at a meeting
in Washington,

637
00:32:24,360 --> 00:32:26,520
and the only thing I
remember from that meeting

638
00:32:26,520 --> 00:32:31,530
is one of the participants
drew a parallel between the Tet

639
00:32:31,530 --> 00:32:36,000
Offensive in Vietnam and
the Arab-Israeli war that

640
00:32:36,000 --> 00:32:41,010
took place about six
or seven years later.

641
00:32:41,010 --> 00:32:42,160
And here's the story.

642
00:32:51,900 --> 00:32:52,619
OK.

643
00:32:52,619 --> 00:32:54,410
And here's what happened
seven years later.

644
00:33:06,590 --> 00:33:10,140
What do you suppose
happened next?

645
00:33:10,140 --> 00:33:12,970
And of course, the
answer is quite clear.

646
00:33:12,970 --> 00:33:19,270
And when we feel like talking
about the long-range eventual

647
00:33:19,270 --> 00:33:21,416
practical uses of the
stuff we're talking about,

648
00:33:21,416 --> 00:33:22,790
this is the kind
of thing we say.

649
00:33:22,790 --> 00:33:26,504
What we want to do is we want
to build for political analysts

650
00:33:26,504 --> 00:33:28,170
tools that would be
as important to them

651
00:33:28,170 --> 00:33:31,230
as spreadsheets are to a
financial analyst, tools that

652
00:33:31,230 --> 00:33:35,890
can enable to predict
or expect or understand

653
00:33:35,890 --> 00:33:40,190
unintended consequence of
actions you might perform.

654
00:33:40,190 --> 00:33:44,210
So this a gap and
alignment problem.

655
00:33:44,210 --> 00:33:45,760
And here is one case
in which we have

656
00:33:45,760 --> 00:33:48,450
departed from modeling humans.

657
00:33:48,450 --> 00:33:51,580
And we did it because
one of our students

658
00:33:51,580 --> 00:33:56,230
was a refugee from
bioengineering.

659
00:33:56,230 --> 00:34:01,120
And he knew a lot about aligning
proteins, sequences, and DNA

660
00:34:01,120 --> 00:34:02,560
sequences.

661
00:34:02,560 --> 00:34:06,260
And so he brought to our group
the Needleman-Wunsch algorithm

662
00:34:06,260 --> 00:34:07,620
for doing alignment.

663
00:34:07,620 --> 00:34:12,170
And we used that to
align those stories.

664
00:34:12,170 --> 00:34:15,531
So there they are
with the gaps in them.

665
00:34:15,531 --> 00:34:17,989
We took those two stories I
showed you on a previous slide.

666
00:34:17,989 --> 00:34:20,699
We put a couple of gaps in.

667
00:34:20,699 --> 00:34:23,139
The Needleman-Wunsch
algorithm aligned them,

668
00:34:23,139 --> 00:34:25,659
and then we were able to fill
in the gaps using one to fill

669
00:34:25,659 --> 00:34:26,794
in the gap in the other.

670
00:34:26,794 --> 00:34:28,210
And since you can't
see it, here's

671
00:34:28,210 --> 00:34:29,580
what it would have filled in.

672
00:34:34,110 --> 00:34:38,780
So that's an example
of how we can

673
00:34:38,780 --> 00:34:41,989
use precedence to think
about what happens next

674
00:34:41,989 --> 00:34:46,790
or what happened in the missing
piece or what led to this.

675
00:34:46,790 --> 00:34:50,330
It's a kind of
analogical reasoning.

676
00:34:50,330 --> 00:34:55,550
My next example is putting
the system in teaching mode.

677
00:34:55,550 --> 00:34:56,510
We have a system.

678
00:34:56,510 --> 00:34:57,320
We have a student.

679
00:34:57,320 --> 00:34:58,750
We want to teach the
student something.

680
00:34:58,750 --> 00:35:01,125
Maybe the student is from Mars
and doesn't know anything.

681
00:35:01,125 --> 00:35:05,090
So this is an example of
how the Genesis system can

682
00:35:05,090 --> 00:35:09,590
watch another version of
itself, not understand the story

683
00:35:09,590 --> 00:35:12,741
and supply the
missing knowledge.

684
00:35:12,741 --> 00:35:14,240
So this is a hint
at how it might be

685
00:35:14,240 --> 00:35:15,531
used in an educational context.

686
00:35:20,400 --> 00:35:23,406
And once you can have a
model of the listener,

687
00:35:23,406 --> 00:35:24,780
then you can also
think about how

688
00:35:24,780 --> 00:35:27,990
you can shape the story so
as to make some aspect of it

689
00:35:27,990 --> 00:35:31,500
more or less believable.

690
00:35:31,500 --> 00:35:33,000
So you notice I'm
carefully avoiding

691
00:35:33,000 --> 00:35:35,475
the word propaganda, which
puts a pejorative spin on it.

692
00:35:35,475 --> 00:35:37,600
But if you're just trying
to teach somebody values,

693
00:35:37,600 --> 00:35:39,910
that's another way
of thinking about it.

694
00:35:39,910 --> 00:35:42,660
So this is the Hansel
and Gretel story.

695
00:35:42,660 --> 00:35:45,750
And the system has been ordered
to make the woodcutter be

696
00:35:45,750 --> 00:35:50,930
likeable, because he does some
good things and bad things.

697
00:35:50,930 --> 00:35:52,680
So when we do that,
you'll note that there

698
00:35:52,680 --> 00:35:56,600
are some things that are
struck out, the stuff in red,

699
00:35:56,600 --> 00:36:00,120
and some things that are
marked in green for emphasis.

700
00:36:00,120 --> 00:36:02,880
And let me blow those
up so you can see them.

701
00:36:06,460 --> 00:36:08,980
So the stuff that the
woodcutter does that's good

702
00:36:08,980 --> 00:36:10,990
are highlighted and
bolded, and the things

703
00:36:10,990 --> 00:36:12,906
that we don't want to
say about the woodcutter

704
00:36:12,906 --> 00:36:16,610
because it makes him look
bad, we strike those out.

705
00:36:16,610 --> 00:36:18,110
And of course,
another way of making

706
00:36:18,110 --> 00:36:22,690
somebody-- what's another way
of making somebody look good?

707
00:36:22,690 --> 00:36:24,760
Make everybody else
look bad, right?

708
00:36:24,760 --> 00:36:27,280
So we can flip a
switch and have him

709
00:36:27,280 --> 00:36:29,160
make comments about
the witch too so

710
00:36:29,160 --> 00:36:35,335
that the woodcutter looks even
better because the bad behavior

711
00:36:35,335 --> 00:36:38,450
of the witch is highlighted.

712
00:36:38,450 --> 00:36:40,407
So these are just some
examples of the kinds

713
00:36:40,407 --> 00:36:41,240
of things we can do.

714
00:36:41,240 --> 00:36:42,430
Here's another one.

715
00:36:42,430 --> 00:36:45,720
This is that Macbeth
story played out in--

716
00:36:45,720 --> 00:36:49,250
it's about 180 80
or 100 sentences,

717
00:36:49,250 --> 00:36:52,450
and we can summarize it.

718
00:36:52,450 --> 00:36:56,050
So we can use our
understanding of the story

719
00:36:56,050 --> 00:37:00,560
to trim away all the stuff that
is not particularly important.

720
00:37:00,560 --> 00:37:02,882
So what is not
particularly important?

721
00:37:02,882 --> 00:37:04,840
Anything that's not
connected to something else

722
00:37:04,840 --> 00:37:08,141
is not particularly important.

723
00:37:08,141 --> 00:37:09,140
Think about it this way.

724
00:37:09,140 --> 00:37:11,220
The only reason
you read a story--

725
00:37:11,220 --> 00:37:14,110
if it's not just for fun-- the
only reason you read a story--

726
00:37:14,110 --> 00:37:18,010
a case study-- is because you
think it'll be useful later.

727
00:37:18,010 --> 00:37:21,420
And it's only useful later
if it exerts constraint.

728
00:37:21,420 --> 00:37:24,430
And it only exerts constraint
if there are connections--

729
00:37:24,430 --> 00:37:26,234
causal connections in this case.

730
00:37:26,234 --> 00:37:28,150
So we take all the stuff
that's not connected,

731
00:37:28,150 --> 00:37:29,904
and we get rid of it.

732
00:37:29,904 --> 00:37:31,570
Then we get rid of
anything that doesn't

733
00:37:31,570 --> 00:37:33,760
lead to a central concept.

734
00:37:33,760 --> 00:37:36,756
So in this case, we say that the
thing is about Pyrrhic victory.

735
00:37:36,756 --> 00:37:37,880
That's the central concept.

736
00:37:37,880 --> 00:37:42,340
We get rid of everything that
doesn't bear on that concept

737
00:37:42,340 --> 00:37:43,900
pattern instantiation.

738
00:37:43,900 --> 00:37:46,450
And then we can squeeze
this thing down to about 20%

739
00:37:46,450 --> 00:37:47,587
of its original size.

740
00:37:47,587 --> 00:37:50,170
And now I come to the thing that
we were talking about before,

741
00:37:50,170 --> 00:37:53,530
and that is how do you
find the right precedent?

742
00:37:53,530 --> 00:37:56,620
Well, if you do a Google
search, it's mostly--

743
00:37:56,620 --> 00:37:58,880
well, they're getting more
and more sophisticated,

744
00:37:58,880 --> 00:38:02,067
but most searches
are mostly keywords.

745
00:38:02,067 --> 00:38:04,150
But now we've got something
better than key words.

746
00:38:04,150 --> 00:38:05,415
We've got concepts.

747
00:38:05,415 --> 00:38:06,790
So what I'm going
to show you now

748
00:38:06,790 --> 00:38:11,980
is a portrayal of an
information retrieval test case

749
00:38:11,980 --> 00:38:15,580
that we did with 14 or
15 conflict stories.

750
00:38:15,580 --> 00:38:18,509
We're interested in how
close they were together.

751
00:38:18,509 --> 00:38:20,050
Because the closer
they are together,

752
00:38:20,050 --> 00:38:24,080
the more one is likely to be a
useful precedent for another.

753
00:38:24,080 --> 00:38:27,550
So in one of these
matrices, what you see

754
00:38:27,550 --> 00:38:30,470
is how close they are when
viewed from the point of view

755
00:38:30,470 --> 00:38:32,290
of key words.

756
00:38:32,290 --> 00:38:41,130
That's the one on the bottom.

757
00:38:41,130 --> 00:38:43,560
The one on the top is how
close they are with respect

758
00:38:43,560 --> 00:38:46,170
to the concepts that
they contain, you know,

759
00:38:46,170 --> 00:38:48,900
the words like revenge,
attack, and not

760
00:38:48,900 --> 00:38:51,390
present in the story as
words, but are present there

761
00:38:51,390 --> 00:38:53,110
anyway as concepts.

762
00:38:53,110 --> 00:38:55,590
And the only point
of this pairing

763
00:38:55,590 --> 00:38:58,124
is to show that the
consideration of similarity

764
00:38:58,124 --> 00:38:59,790
is different depending
on whether you're

765
00:38:59,790 --> 00:39:06,540
thinking in terms of concepts
or thinking in terms of words.

766
00:39:06,540 --> 00:39:08,400
So here's a story.

767
00:39:12,250 --> 00:39:14,490
A young man went to
work for a company.

768
00:39:14,490 --> 00:39:16,980
His boss was pretty mean.

769
00:39:16,980 --> 00:39:20,640
Wouldn't let him
go to conferences.

770
00:39:20,640 --> 00:39:23,270
One day somebody
else in the company

771
00:39:23,270 --> 00:39:25,950
arranged for him to go
to a conference anyhow.

772
00:39:25,950 --> 00:39:27,970
Provided transportation.

773
00:39:27,970 --> 00:39:31,459
He went to the conference, and
he met some interesting people.

774
00:39:31,459 --> 00:39:33,000
But unfortunately,
circumstances were

775
00:39:33,000 --> 00:39:36,744
that he had to leave early
to catch a flight back home.

776
00:39:36,744 --> 00:39:38,910
And then some of the people
he met at the conference

777
00:39:38,910 --> 00:39:40,920
started looking for
him because he was so--

778
00:39:40,920 --> 00:39:43,980
so what story am I telling?

779
00:39:43,980 --> 00:39:47,950
It's pretty obviously a
Cinderella story, right?

780
00:39:47,950 --> 00:39:50,115
But there's no pumpkin.

781
00:39:50,115 --> 00:39:51,240
There's no fairy godmother.

782
00:39:51,240 --> 00:39:54,120
It's just that even
though the agents are

783
00:39:54,120 --> 00:39:56,670
very different in terms
of their descriptions,

784
00:39:56,670 --> 00:40:01,610
the relationships between
them are pretty much the same.

785
00:40:01,610 --> 00:40:04,950
So over the years, what we've
done quite without intending

786
00:40:04,950 --> 00:40:07,110
it or expecting
it or realizing it

787
00:40:07,110 --> 00:40:12,200
is that we have duplicated in
Genesis the kinds of thinking

788
00:40:12,200 --> 00:40:15,880
that Marvin Minsky talks a lot
about in his most recent book,

789
00:40:15,880 --> 00:40:17,850
The Emotion Machine.

790
00:40:17,850 --> 00:40:21,144
He likes to talk in
terms of multiplicities.

791
00:40:21,144 --> 00:40:22,560
We have multiple
ways of thinking.

792
00:40:22,560 --> 00:40:26,130
We have multiple
representations.

793
00:40:26,130 --> 00:40:29,760
And those kinds of reasoning
occur on multiple levels,

794
00:40:29,760 --> 00:40:32,250
from instinctive
reactions at the bottom

795
00:40:32,250 --> 00:40:34,380
to self-conscious
reflection on the top.

796
00:40:38,690 --> 00:40:40,700
So quite without
our intending it,

797
00:40:40,700 --> 00:40:44,400
when we thought about
it one day by accident,

798
00:40:44,400 --> 00:40:50,450
we had this epiphany that
we've been working to implement

799
00:40:50,450 --> 00:40:54,160
much of what is in that book.

800
00:40:54,160 --> 00:40:57,000
So so far, and I'm going to
depart from story understanding

801
00:40:57,000 --> 00:40:59,840
a little bit to talk to you
about some other hypotheses

802
00:40:59,840 --> 00:41:01,130
of mine.

803
00:41:01,130 --> 00:41:03,410
So far, there are two--
the strong story hypothesis

804
00:41:03,410 --> 00:41:06,770
and then there's this
inner language hypothesis

805
00:41:06,770 --> 00:41:09,650
that Chomsky likes
to talk a lot about.

806
00:41:09,650 --> 00:41:12,920
We have an inner language,
and our inner language

807
00:41:12,920 --> 00:41:15,620
came before our outer language.

808
00:41:15,620 --> 00:41:17,840
And this is what makes
it possible to think.

809
00:41:17,840 --> 00:41:20,410
So those are two
hypotheses-- inner language

810
00:41:20,410 --> 00:41:22,250
and strong story.

811
00:41:22,250 --> 00:41:27,320
Here's another
one-- it's important

812
00:41:27,320 --> 00:41:31,400
that we're social animals,
and it's actually important

813
00:41:31,400 --> 00:41:33,450
that we talk to each other.

814
00:41:33,450 --> 00:41:38,180
Once Danny Hillis, a famous
guy, a graduate of ours,

815
00:41:38,180 --> 00:41:41,300
came into my office
and said, have you ever

816
00:41:41,300 --> 00:41:42,980
had the experience of--

817
00:41:42,980 --> 00:41:44,630
well, you often
talked to Marvin.

818
00:41:44,630 --> 00:41:45,984
Yeah, I do.

819
00:41:45,984 --> 00:41:47,900
And have you ever had
the experience, he said,

820
00:41:47,900 --> 00:41:50,480
of having Marvin guess?

821
00:41:50,480 --> 00:41:52,440
He has a very short
attention span, Marvin,

822
00:41:52,440 --> 00:41:56,540
and he'll often guess your
idea before you've fully

823
00:41:56,540 --> 00:41:57,500
explained it?

824
00:41:57,500 --> 00:41:58,100
Yes, I said.

825
00:41:58,100 --> 00:42:00,000
It happens all the time.

826
00:42:00,000 --> 00:42:02,870
Isn't it the case, Danny said,
that the idea that he guesses

827
00:42:02,870 --> 00:42:04,610
you have is better
than the idea you're

828
00:42:04,610 --> 00:42:07,520
actually trying
to tell him about?

829
00:42:07,520 --> 00:42:08,120
Yes.

830
00:42:08,120 --> 00:42:11,120
And then he pointed out, well,
maybe when we talk to ourself,

831
00:42:11,120 --> 00:42:13,440
it's doing the
same kind of thing.

832
00:42:13,440 --> 00:42:16,460
It's accessing ideas and
putting them together

833
00:42:16,460 --> 00:42:21,470
in ways that wouldn't be
possible if we weren't talking.

834
00:42:21,470 --> 00:42:24,440
So it often happens
that ideas come

835
00:42:24,440 --> 00:42:26,240
about when we talk
to each other,

836
00:42:26,240 --> 00:42:31,640
because it forces the rendering
of our thoughts and language.

837
00:42:31,640 --> 00:42:34,400
And if we don't have a friend
or don't happen to have anybody

838
00:42:34,400 --> 00:42:35,450
around we can talk to--

839
00:42:35,450 --> 00:42:38,500
I feel like I talk to
myself all the time.

840
00:42:38,500 --> 00:42:41,750
And maybe that's an
important consequence

841
00:42:41,750 --> 00:42:44,930
or important aspect
of our intelligence

842
00:42:44,930 --> 00:42:47,320
is conversation that we
carry on with ourself.

843
00:42:52,430 --> 00:42:56,060
Be careful doing this out loud.

844
00:42:56,060 --> 00:42:58,940
Some people will think
you've got a screw loose.

845
00:42:58,940 --> 00:43:01,190
But let me let me show
you an experiment I

846
00:43:01,190 --> 00:43:05,510
consider to be extraordinarily
interesting along these lines.

847
00:43:05,510 --> 00:43:08,391
It was done by a friend of mine
at the University Pittsburgh

848
00:43:08,391 --> 00:43:08,890
Michelin.

849
00:43:11,640 --> 00:43:14,600
Mickey, as she is called,
was working with students

850
00:43:14,600 --> 00:43:15,839
on physics problems.

851
00:43:15,839 --> 00:43:17,380
You've all done this
kind of problem,

852
00:43:17,380 --> 00:43:22,040
and it's about pulleys and
weights and forces and stuff.

853
00:43:22,040 --> 00:43:25,100
And so these students
were learning the subject,

854
00:43:25,100 --> 00:43:28,250
and so she gave them a quiz,
and she had them talk out loud

855
00:43:28,250 --> 00:43:30,970
as they were
working on the quiz.

856
00:43:30,970 --> 00:43:37,780
And she kept track
of how many things

857
00:43:37,780 --> 00:43:39,990
the best students
said to themselves

858
00:43:39,990 --> 00:43:43,726
and how many things the worst
students said to themselves.

859
00:43:43,726 --> 00:43:45,100
So in this particular
experiment,

860
00:43:45,100 --> 00:43:46,266
there weren't many students.

861
00:43:46,266 --> 00:43:50,170
I think eight-- four good
ones and four bad ones.

862
00:43:50,170 --> 00:43:52,960
And the good ones scored
twice as high as the bad ones,

863
00:43:52,960 --> 00:43:54,490
and here's the data.

864
00:43:57,770 --> 00:44:03,020
The better students said about
3 and 1/2 times more stuff

865
00:44:03,020 --> 00:44:06,080
to themselves than
the other ones.

866
00:44:06,080 --> 00:44:07,580
So unfortunately,
this is backwards.

867
00:44:07,580 --> 00:44:09,288
We don't know if we
took the bad students

868
00:44:09,288 --> 00:44:13,679
and encouraged them to talk
more, if they'd become smarter.

869
00:44:13,679 --> 00:44:14,720
So we're not saying that.

870
00:44:14,720 --> 00:44:16,160
But it is interesting
observation

871
00:44:16,160 --> 00:44:18,050
that the ones who talked
to themselves more

872
00:44:18,050 --> 00:44:19,275
were actually better at it.

873
00:44:19,275 --> 00:44:20,900
And what they were
saying was a mixture

874
00:44:20,900 --> 00:44:23,900
of problem-solving things
and physics things.

875
00:44:23,900 --> 00:44:28,590
Like I'm stuck or maybe
I should try that again

876
00:44:28,590 --> 00:44:31,280
or physics things like I think
I have to do a force diagram.

877
00:44:31,280 --> 00:44:34,500
A mixture of those
kinds of things.

878
00:44:34,500 --> 00:44:41,660
So talking seems to surface
a kind of capability

879
00:44:41,660 --> 00:44:43,400
that not every animal has.

880
00:44:45,737 --> 00:44:46,820
And then there's this one.

881
00:44:51,440 --> 00:44:54,630
So it isn't just that we
have a perceptual apparatus,

882
00:44:54,630 --> 00:44:59,300
it's that we can direct it
to do stuff in our behalf.

883
00:44:59,300 --> 00:45:03,170
That's what I think
is part of the magic.

884
00:45:03,170 --> 00:45:05,360
So my standard examples--

885
00:45:05,360 --> 00:45:06,870
John kissed Mary.

886
00:45:06,870 --> 00:45:07,910
Did John touch Mary?

887
00:45:11,630 --> 00:45:13,440
Everybody knows that
the answer is yes.

888
00:45:13,440 --> 00:45:14,231
How do you know it?

889
00:45:14,231 --> 00:45:17,164
Because you imagine
it and you see it.

890
00:45:17,164 --> 00:45:18,830
So there's a lot of
talk in AI about how

891
00:45:18,830 --> 00:45:21,080
you gather
common-sense knowledge

892
00:45:21,080 --> 00:45:23,774
and how you can only know
a limited number of facts.

893
00:45:23,774 --> 00:45:25,190
I think it's all
screwy, because I

894
00:45:25,190 --> 00:45:26,990
think a lot of our
common sense comes just

895
00:45:26,990 --> 00:45:34,550
in time by the engagement
of our perceptual apparatus.

896
00:45:34,550 --> 00:45:36,550
So there's John
kissing Mary, and now I

897
00:45:36,550 --> 00:45:39,080
want to give you another puzzle.

898
00:45:39,080 --> 00:45:43,450
And the next puzzle is how
many countries in Africa

899
00:45:43,450 --> 00:45:44,630
does the equator go through?

900
00:45:48,100 --> 00:45:48,850
Does anybody know?

901
00:45:51,360 --> 00:45:56,430
I've asked students who
come to MIT from Africa,

902
00:45:56,430 --> 00:45:58,350
and they don't know.

903
00:45:58,350 --> 00:45:59,850
And some of them
come from countries

904
00:45:59,850 --> 00:46:02,270
that are on the equator
and they don't know.

905
00:46:05,620 --> 00:46:07,300
But now you know.

906
00:46:07,300 --> 00:46:08,380
And what's happened?

907
00:46:11,690 --> 00:46:14,502
Your eyes scan across that
red line, and you count.

908
00:46:14,502 --> 00:46:17,630
Shimon Ullman would call
it a visual routine.

909
00:46:17,630 --> 00:46:20,490
So you're forming-- you're
creating a little program.

910
00:46:20,490 --> 00:46:23,050
Your language
system is demanding

911
00:46:23,050 --> 00:46:26,170
that your visual system
run a little program that

912
00:46:26,170 --> 00:46:28,340
scans across and counts.

913
00:46:28,340 --> 00:46:31,020
And your vision system
reports back the answer.

914
00:46:31,020 --> 00:46:32,660
And that I think is a miracle.

915
00:46:35,320 --> 00:46:36,790
So one more example.

916
00:46:36,790 --> 00:46:37,900
It's a little grizzly.

917
00:46:37,900 --> 00:46:40,200
I hope you don't mind.

918
00:46:40,200 --> 00:46:45,200
So a couple of years ago,
I installed a table saw.

919
00:46:45,200 --> 00:46:46,135
I like to--

920
00:46:46,135 --> 00:46:47,050
I'm an engineer.

921
00:46:47,050 --> 00:46:48,360
I like to build stuff.

922
00:46:48,360 --> 00:46:50,080
I like to make stuff.

923
00:46:50,080 --> 00:46:52,840
And I had a friend of mine
who's a cabinetmaker--

924
00:46:52,840 --> 00:46:56,020
a good cabinetmaker-- helped
me to install the saw.

925
00:46:56,020 --> 00:46:59,560
And he said, you must
never wear gloves

926
00:46:59,560 --> 00:47:03,280
when you operate this tool.

927
00:47:03,280 --> 00:47:04,550
And I said well--

928
00:47:04,550 --> 00:47:08,140
and before I got the first
word out, I knew why.

929
00:47:08,140 --> 00:47:10,360
No one had ever told me.

930
00:47:10,360 --> 00:47:12,670
I had never witnessed
an experience that would

931
00:47:12,670 --> 00:47:15,010
suggest that should be a rule.

932
00:47:15,010 --> 00:47:16,870
But I imagined it.

933
00:47:16,870 --> 00:47:19,430
Can you imagine it?

934
00:47:19,430 --> 00:47:21,410
What I need you to
imagine is that you're

935
00:47:21,410 --> 00:47:26,010
wearing the kind of
fluffy cotton gloves.

936
00:47:26,010 --> 00:47:27,760
Got it now?

937
00:47:27,760 --> 00:47:32,720
And the fluffy cotton glove
gets caught in the blade.

938
00:47:32,720 --> 00:47:34,440
And now you know why
you would never--

939
00:47:34,440 --> 00:47:36,360
I don't think any of you
would ever use gloves

940
00:47:36,360 --> 00:47:38,734
when you operate a table saw
now, because you can imagine

941
00:47:38,734 --> 00:47:42,474
the grisly result. So it's not
just our perceptual apparatus.

942
00:47:42,474 --> 00:47:44,640
It's our ability to deploy
our perceptual apparatus,

943
00:47:44,640 --> 00:47:48,990
and our imagination, I
think, is a great miracle.

944
00:47:48,990 --> 00:47:51,317
That vision is still hard,
as everyone in vision

945
00:47:51,317 --> 00:47:51,900
will tell you.

946
00:47:51,900 --> 00:47:54,420
Some years ago, I was involved
in a DARPA program that

947
00:47:54,420 --> 00:48:01,050
had as its objective recognizing
48 activities, 47 of which

948
00:48:01,050 --> 00:48:03,960
can be performed by humans.

949
00:48:03,960 --> 00:48:05,160
One of them is fly.

950
00:48:05,160 --> 00:48:08,370
So that doesn't count, I guess.

951
00:48:08,370 --> 00:48:12,781
After a couple of years into the
program, they retrenched to 17.

952
00:48:12,781 --> 00:48:14,280
At the end of the
program, they said

953
00:48:14,280 --> 00:48:18,420
if you could do six
reliably, you'll be a hero.

954
00:48:18,420 --> 00:48:21,540
And my team caught
everyone's attention

955
00:48:21,540 --> 00:48:24,070
by saying we wouldn't
recognize any actions that

956
00:48:24,070 --> 00:48:26,670
would distract people.

957
00:48:26,670 --> 00:48:29,160
So vision is very hard.

958
00:48:29,160 --> 00:48:31,860
And then stories do, of
course, come together

959
00:48:31,860 --> 00:48:34,266
with perception, right?

960
00:48:34,266 --> 00:48:35,640
At some point,
you've doubtlessly

961
00:48:35,640 --> 00:48:37,740
in the course of the last
two weeks seen this example.

962
00:48:37,740 --> 00:48:38,406
What am I doing?

963
00:48:41,380 --> 00:48:42,430
What am I doing?

964
00:48:42,430 --> 00:48:43,810
AUDIENCE: Drinking.

965
00:48:43,810 --> 00:48:47,460
PATRICK HENRY WINSTON: And
then there's Ullman's cat.

966
00:48:47,460 --> 00:48:49,830
What's it doing?

967
00:48:49,830 --> 00:48:57,900
So my interpretation of this
is that that cat and I are--

968
00:48:57,900 --> 00:49:00,510
it's the same story.

969
00:49:00,510 --> 00:49:02,200
You can imagine
that there's thirst,

970
00:49:02,200 --> 00:49:05,640
that there are activities
that lead to water or liquid

971
00:49:05,640 --> 00:49:06,700
passing into the mouth.

972
00:49:06,700 --> 00:49:10,530
So we give them the same label,
even though visually they're

973
00:49:10,530 --> 00:49:12,650
as different as
anything could be.

974
00:49:12,650 --> 00:49:15,300
You would never get a
deep neural net program

975
00:49:15,300 --> 00:49:17,700
that's been trained
on me to recognize

976
00:49:17,700 --> 00:49:19,710
that that's a cat drinking.

977
00:49:19,710 --> 00:49:22,120
There's visually
nothing similar at all.

978
00:49:24,760 --> 00:49:26,760
All right.

979
00:49:26,760 --> 00:49:29,710
But we might ask this question--

980
00:49:29,710 --> 00:49:31,150
can we have an
intelligent machine

981
00:49:31,150 --> 00:49:33,430
without a perceptual system?

982
00:49:33,430 --> 00:49:37,464
You know, that Genesis system
with Macbeth and all that?

983
00:49:37,464 --> 00:49:39,130
Is it really intelligent
when it doesn't

984
00:49:39,130 --> 00:49:41,110
have any perceptual
apparatus at all?

985
00:49:41,110 --> 00:49:42,150
It can't see anything.

986
00:49:42,150 --> 00:49:44,191
It doesn't know what it
feels like to be stabbed.

987
00:49:47,300 --> 00:49:50,070
I think it's an interesting
philosophical question.

988
00:49:50,070 --> 00:49:54,230
And I'm a little agnostic
on this right now,

989
00:49:54,230 --> 00:50:00,560
a little more agnostic than I
was half a year ago, because I

990
00:50:00,560 --> 00:50:03,290
went back to the republic.

991
00:50:03,290 --> 00:50:07,750
You remember that metaphor
of the cave and the republic?

992
00:50:07,750 --> 00:50:09,320
There's a metaphor of the cave.

993
00:50:09,320 --> 00:50:13,130
You have some prisoners, and
they're chained in this cave,

994
00:50:13,130 --> 00:50:14,720
and there's a fire somewhere.

995
00:50:14,720 --> 00:50:19,442
And all they can see is their
own shadows against the wall.

996
00:50:19,442 --> 00:50:20,900
That's all they've
got for reality.

997
00:50:23,490 --> 00:50:28,250
And so their reality
is extremely limited.

998
00:50:28,250 --> 00:50:31,650
So they're intelligent
but they're limited.

999
00:50:31,650 --> 00:50:34,866
So I think, generalizing
that metaphor,

1000
00:50:34,866 --> 00:50:36,740
I think a machine without
a perceptual system

1001
00:50:36,740 --> 00:50:40,440
has extraordinarily
limited reality.

1002
00:50:40,440 --> 00:50:42,920
And maybe we need a
little bit of perception

1003
00:50:42,920 --> 00:50:44,410
to have any kind--

1004
00:50:44,410 --> 00:50:46,999
to have what we would be
comfortable to calling

1005
00:50:46,999 --> 00:50:47,540
intelligence.

1006
00:50:47,540 --> 00:50:48,530
But we don't need much.

1007
00:50:48,530 --> 00:50:51,620
And another way of thinking
about it is, sort of the fact

1008
00:50:51,620 --> 00:50:56,210
that our own reality is limited.

1009
00:50:56,210 --> 00:51:00,380
If you compare our visual
system to that of a bee,

1010
00:51:00,380 --> 00:51:04,220
they have a much broader
spectrum of wavelengths

1011
00:51:04,220 --> 00:51:08,170
that they can make use of,
because they don't have--

1012
00:51:08,170 --> 00:51:09,760
do you know why?

1013
00:51:09,760 --> 00:51:13,740
We are limited because we
have water in our eyeballs.

1014
00:51:13,740 --> 00:51:15,920
And so some of that stuff
in the far ultraviolet

1015
00:51:15,920 --> 00:51:17,420
can't get through.

1016
00:51:17,420 --> 00:51:20,120
But bees can see it.

1017
00:51:20,120 --> 00:51:23,704
And then that's comparing
us humans with bees.

1018
00:51:23,704 --> 00:51:25,620
How about comparing one
human against another?

1019
00:51:31,690 --> 00:51:33,585
At this distance, I
can hardly see it.

1020
00:51:33,585 --> 00:51:34,210
Can you see it?

1021
00:51:34,210 --> 00:51:35,020
AUDIENCE: Boat.

1022
00:51:35,020 --> 00:51:36,478
PATRICK HENRY
WINSTON: It's a boat,

1023
00:51:36,478 --> 00:51:40,540
but some people can't see it,
because they're colorblind.

1024
00:51:40,540 --> 00:51:45,340
So for them, there's a
slight impairment of reality,

1025
00:51:45,340 --> 00:51:47,680
just like a computer
without a perceptual system

1026
00:51:47,680 --> 00:51:49,315
would have a big
impairment of reality.

1027
00:51:52,570 --> 00:51:57,430
So it's been said many times
that what we're doing here

1028
00:51:57,430 --> 00:52:00,025
is we're trying to understand
the science side of things.

1029
00:52:00,025 --> 00:52:02,400
And we think that that will
lead to engineering advances.

1030
00:52:02,400 --> 00:52:08,410
And people often
ask this, and those

1031
00:52:08,410 --> 00:52:13,950
who don't believe that human
intelligence is relevant

1032
00:52:13,950 --> 00:52:18,142
will say, well, airplanes
don't fly like birds.

1033
00:52:18,142 --> 00:52:19,600
What do you think
of that argument?

1034
00:52:22,300 --> 00:52:27,040
I think it has a gigantic
hole, and it turns out

1035
00:52:27,040 --> 00:52:31,180
that the Wright brothers were
extremely interested in birds,

1036
00:52:31,180 --> 00:52:34,600
because they knew that
birds could tell them

1037
00:52:34,600 --> 00:52:38,300
something about the
secrets of aerodynamics.

1038
00:52:38,300 --> 00:52:43,065
And all flying machines have
to deal with the secrets

1039
00:52:43,065 --> 00:52:44,065
of that kind of physics.

1040
00:52:46,730 --> 00:52:48,370
So we study humans,
not because we're

1041
00:52:48,370 --> 00:52:50,200
going to build a machine
that has neurons in it,

1042
00:52:50,200 --> 00:52:51,574
but because we
want to understand

1043
00:52:51,574 --> 00:52:55,330
the computational imperatives
that human intelligence can

1044
00:52:55,330 --> 00:52:57,280
shed light on.

1045
00:52:57,280 --> 00:52:59,890
That's why we think it has
engineering value, even

1046
00:52:59,890 --> 00:53:03,610
though we won't in
any likely future

1047
00:53:03,610 --> 00:53:11,170
be building computers with
synapses of the kind we have.

1048
00:53:11,170 --> 00:53:17,969
Well, now we come to the dangers
that we started out with.

1049
00:53:17,969 --> 00:53:20,260
What do you think we should--
suppose that machines can

1050
00:53:20,260 --> 00:53:21,060
become--

1051
00:53:21,060 --> 00:53:23,949
they do become really smart,
and we've got machine learning.

1052
00:53:23,949 --> 00:53:24,490
What is that?

1053
00:53:24,490 --> 00:53:26,290
That's modern statistics.

1054
00:53:26,290 --> 00:53:29,439
And of course, it's useful.

1055
00:53:29,439 --> 00:53:31,480
What if they became really
smart in the same ways

1056
00:53:31,480 --> 00:53:32,188
that we're smart?

1057
00:53:32,188 --> 00:53:34,378
What would we want to
do to protect ourselves?

1058
00:53:37,190 --> 00:53:41,270
Well, for this, I'd like
to introduce the subject

1059
00:53:41,270 --> 00:53:44,510
by asking you to read
the following story.

1060
00:53:44,510 --> 00:53:50,431
This was part of that Morrison
paying a suite of experiments.

1061
00:53:59,114 --> 00:54:00,780
I'm sorry these are
so full of violence.

1062
00:54:00,780 --> 00:54:02,770
This happens to be
what they worked with.

1063
00:54:09,730 --> 00:54:16,290
So after the students read
this story, they were asked

1064
00:54:16,290 --> 00:54:23,010
did Lu kill Sean because
America is individualistic?

1065
00:54:23,010 --> 00:54:25,750
And the Asian students would
have a tendency to say yes.

1066
00:54:25,750 --> 00:54:28,760
So how can we model
that in our system?

1067
00:54:28,760 --> 00:54:33,190
Well, to start out with, this
is the original interpretation

1068
00:54:33,190 --> 00:54:35,350
of the story as told.

1069
00:54:35,350 --> 00:54:38,170
And if you look at
where that arrow is,

1070
00:54:38,170 --> 00:54:42,460
you see that that's where Lu
kills Sean, and just in back

1071
00:54:42,460 --> 00:54:44,800
of it is the means.

1072
00:54:44,800 --> 00:54:46,082
He shot him with a gun.

1073
00:54:46,082 --> 00:54:47,540
But it's not
connected to anything.

1074
00:54:50,320 --> 00:54:52,470
And so the instantaneous
response is,

1075
00:54:52,470 --> 00:54:54,930
we don't know why
Lu killed Sean.

1076
00:54:54,930 --> 00:54:57,602
But what Genesis does
at this point is,

1077
00:54:57,602 --> 00:54:59,310
when asked the question,
did Lu kill Sean

1078
00:54:59,310 --> 00:55:01,440
because America is
individualistic?

1079
00:55:01,440 --> 00:55:04,470
It goes into its
own memory and said

1080
00:55:04,470 --> 00:55:08,340
I am modeling an Asian reader.

1081
00:55:08,340 --> 00:55:11,610
I believe that America
is individualistic.

1082
00:55:11,610 --> 00:55:14,430
I will insert that
into the story.

1083
00:55:14,430 --> 00:55:18,480
I will examine the
consequences of that insertion,

1084
00:55:18,480 --> 00:55:20,955
and then see what happens.

1085
00:55:20,955 --> 00:55:23,485
And this is what happens.

1086
00:55:26,005 --> 00:55:28,580
The question is asked and
inserts into the story--

1087
00:55:28,580 --> 00:55:30,640
boom, boom, boom.

1088
00:55:30,640 --> 00:55:35,120
And now Lu kills Sean is
connected all the way back

1089
00:55:35,120 --> 00:55:38,060
to America is individualistic.

1090
00:55:38,060 --> 00:55:40,940
And so the machine can
say yes, but that's not

1091
00:55:40,940 --> 00:55:43,239
the interesting part.

1092
00:55:43,239 --> 00:55:44,780
Now, this is what
it says, but that's

1093
00:55:44,780 --> 00:55:47,000
not the interesting part.

1094
00:55:47,000 --> 00:55:48,640
The interesting part is this--

1095
00:55:52,900 --> 00:55:57,400
it describes to
itself what it's doing

1096
00:55:57,400 --> 00:56:00,070
in its own language,
which it treats

1097
00:56:00,070 --> 00:56:04,040
as its story of
its own behavior.

1098
00:56:04,040 --> 00:56:07,420
So it now has the
capacity to introspect

1099
00:56:07,420 --> 00:56:09,164
into what it itself is doing.

1100
00:56:13,250 --> 00:56:16,010
I think that's pretty cool.

1101
00:56:16,010 --> 00:56:16,770
It's a kind of--

1102
00:56:16,770 --> 00:56:17,270
OK.

1103
00:56:17,270 --> 00:56:19,478
It's a suitcase word--
self-awareness, consciousness,

1104
00:56:19,478 --> 00:56:23,270
a big suitcase word, but you
can say that this system is

1105
00:56:23,270 --> 00:56:24,969
aware of its own behavior.

1106
00:56:24,969 --> 00:56:26,510
By the way, this is
one of the things

1107
00:56:26,510 --> 00:56:29,630
that Turing addressed
in his original paper.

1108
00:56:29,630 --> 00:56:34,400
One of the arguments
against AI was--

1109
00:56:34,400 --> 00:56:36,290
I forgot what Turing called it--

1110
00:56:36,290 --> 00:56:37,790
the disabilities argument.

1111
00:56:37,790 --> 00:56:40,640
And people were saying
computers can never

1112
00:56:40,640 --> 00:56:42,590
do these kinds of
things, one of which

1113
00:56:42,590 --> 00:56:44,360
is be the subject
of its own thought.

1114
00:56:44,360 --> 00:56:47,992
But Genesis is now reading
the story of its own behavior

1115
00:56:47,992 --> 00:56:49,700
and being the subject
of its own thought.

1116
00:56:52,331 --> 00:56:52,830
OK.

1117
00:56:52,830 --> 00:56:55,050
So what if they
really become smart?

1118
00:56:55,050 --> 00:56:57,390
Now, I will become a little
bit on the whimsical side.

1119
00:56:57,390 --> 00:56:58,680
Suppose they really get smart.

1120
00:56:58,680 --> 00:57:01,800
What will we want to do?

1121
00:57:01,800 --> 00:57:03,750
Maybe we ought to simulate
these suckers first

1122
00:57:03,750 --> 00:57:05,375
before we turn them
loose in the world.

1123
00:57:05,375 --> 00:57:07,110
Do you agree with that?

1124
00:57:07,110 --> 00:57:11,070
After all, simulation is
now a well-developed art.

1125
00:57:11,070 --> 00:57:13,600
So we can take these machines--
maybe there will be robots.

1126
00:57:13,600 --> 00:57:15,016
Maybe there will
just be programs,

1127
00:57:15,016 --> 00:57:17,640
and we can do elaborate
simulations to make sure

1128
00:57:17,640 --> 00:57:21,320
that they're not dangerous.

1129
00:57:21,320 --> 00:57:24,560
And we would want to do that in
as natural a world as possible.

1130
00:57:28,690 --> 00:57:32,620
And we'd want to do these
experiments for a long time

1131
00:57:32,620 --> 00:57:36,280
before we turned them loose
to see what kinds of behaviors

1132
00:57:36,280 --> 00:57:39,958
were to be expected.

1133
00:57:39,958 --> 00:57:41,541
And you see where
I'm going with this?

1134
00:57:44,630 --> 00:57:45,862
Maybe we're at it.

1135
00:57:49,930 --> 00:57:52,330
I think it's a pretty
interesting possibility.

1136
00:57:52,330 --> 00:57:56,400
I'm not sure any of you are it,
but I know that I might be it.

1137
00:57:56,400 --> 00:57:59,350
This is a great simulation
to see if we're dangerous.

1138
00:57:59,350 --> 00:58:01,720
And I must say, if
we are a simulation

1139
00:58:01,720 --> 00:58:05,800
to see if we're dangerous,
it's not going very well.

1140
00:58:05,800 --> 00:58:07,192
Key questions revisited.

1141
00:58:07,192 --> 00:58:08,650
Why has AI made so
little progress?

1142
00:58:08,650 --> 00:58:11,650
Because for too many years, it
was about reasoning instead of

1143
00:58:11,650 --> 00:58:13,712
about what's different.

1144
00:58:13,712 --> 00:58:14,920
How can we make progress now?

1145
00:58:14,920 --> 00:58:17,320
By focusing on what
it is that makes

1146
00:58:17,320 --> 00:58:19,960
human intelligence unique.

1147
00:58:19,960 --> 00:58:21,460
How can a computer
be really smart

1148
00:58:21,460 --> 00:58:23,270
without a perceptual
system or can it be?

1149
00:58:23,270 --> 00:58:27,490
And I think yes, but I'm
a little bit agnostic.

1150
00:58:27,490 --> 00:58:28,630
Should engineers care?

1151
00:58:28,630 --> 00:58:31,299
Absolutely, because
it's not the hardware

1152
00:58:31,299 --> 00:58:32,590
that we're trying to replicate.

1153
00:58:32,590 --> 00:58:36,171
It's the understanding of the
computational imperatives.

1154
00:58:36,171 --> 00:58:38,420
What are the dangers and
what should we do about them?

1155
00:58:38,420 --> 00:58:42,250
We need to make any system
that we depend on capable

1156
00:58:42,250 --> 00:58:44,360
of explaining itself.

1157
00:58:44,360 --> 00:58:45,970
It needs to have
a kind of ability

1158
00:58:45,970 --> 00:58:49,600
to explain what it's
doing in our terms.

1159
00:58:49,600 --> 00:58:50,260
No.

1160
00:58:50,260 --> 00:58:51,801
Don't just tell me
it's a school bus.

1161
00:58:51,801 --> 00:58:54,280
Tell me why you think
it's a school bus.

1162
00:58:54,280 --> 00:58:55,210
You did this thing.

1163
00:58:55,210 --> 00:58:57,970
You better be able to
defend yourself in something

1164
00:58:57,970 --> 00:58:59,410
analogous to a court of law.

1165
00:58:59,410 --> 00:59:02,900
These are the things
we need to do.

1166
00:59:02,900 --> 00:59:06,042
And finally, my final
slide is this one.

1167
00:59:06,042 --> 00:59:07,750
This is just a summary
of the things I've

1168
00:59:07,750 --> 00:59:11,430
tried to cover this morning.

1169
00:59:11,430 --> 00:59:17,430
And one last thing, I think it
was Cato the Elder who said,

1170
00:59:17,430 --> 00:59:20,830
carthago delenda
est. Every speech

1171
00:59:20,830 --> 00:59:24,880
to the Roman Senate ended with
Carthage must be destroyed.

1172
00:59:24,880 --> 00:59:26,690
He could be talking
about the sewer system,

1173
00:59:26,690 --> 00:59:31,240
and the final words would be
Carthage must be destroyed.

1174
00:59:31,240 --> 00:59:37,090
Well, here is my analog to
Carthage must be destroyed.

1175
00:59:37,090 --> 00:59:39,130
I think this is so,
because I think--

1176
00:59:39,130 --> 00:59:43,600
well, many people consider
artificial intelligence

1177
00:59:43,600 --> 00:59:44,950
products to be dangerous.

1178
00:59:44,950 --> 00:59:48,010
I think understanding
our own intelligence is

1179
00:59:48,010 --> 00:59:50,470
essential to the
survival of the species.

1180
00:59:50,470 --> 00:59:53,640
We really do need to
understand ourselves better.