1
00:00:00,000 --> 00:00:02,520
The following content is
provided under a Creative

2
00:00:02,520 --> 00:00:03,940
Commons license.

3
00:00:03,940 --> 00:00:06,330
Your support will help
MIT OpenCourseWare

4
00:00:06,330 --> 00:00:10,660
continue to offer high quality
educational resources for free.

5
00:00:10,660 --> 00:00:13,320
To make a donation or
view additional materials

6
00:00:13,320 --> 00:00:17,160
from hundreds of MIT courses,
visit MIT OpenCourseWare

7
00:00:17,160 --> 00:00:18,370
at ocw.mit.edu.

8
00:00:21,750 --> 00:00:24,300
RUSS TEDRAKE: OK, so your
project presentations

9
00:00:24,300 --> 00:00:26,970
we drew randomly last time.

10
00:00:26,970 --> 00:00:29,790
Phillip, unfortunately,
got the short straw.

11
00:00:29,790 --> 00:00:31,185
He's presenting today.

12
00:00:31,185 --> 00:00:32,310
Don't feel too bad for him.

13
00:00:32,310 --> 00:00:34,620
It's because he's going
to [INAUDIBLE] next week,

14
00:00:34,620 --> 00:00:35,460
to ICRA, right?

15
00:00:35,460 --> 00:00:41,100
So the plan is
eight minute talks

16
00:00:41,100 --> 00:00:54,840
plus two minutes of
questions and transitions

17
00:00:54,840 --> 00:00:56,790
between speakers.

18
00:00:56,790 --> 00:00:58,417
You're free to use
your own laptop.

19
00:00:58,417 --> 00:01:00,000
Hopefully it'll just
plug in and work.

20
00:01:00,000 --> 00:01:02,700
You're also free to send
me slides so that I can--

21
00:01:02,700 --> 00:01:04,825
I'll put on my laptop,
whichever is better for you.

22
00:01:04,825 --> 00:01:07,408
If you have movies or something,
send them to me a few minutes

23
00:01:07,408 --> 00:01:09,180
before so I can
make sure they play,

24
00:01:09,180 --> 00:01:12,780
but whatever is better for you.

25
00:01:12,780 --> 00:01:16,655
I'd like to ask you to try to
sort of-- in eight minutes--

26
00:01:16,655 --> 00:01:18,030
I mean, it's
actually really hard

27
00:01:18,030 --> 00:01:19,500
to give an eight
minute talk, right?

28
00:01:19,500 --> 00:01:21,667
It's much easier to give
an hour and a half lecture.

29
00:01:21,667 --> 00:01:24,450
You can just ramble on.

30
00:01:24,450 --> 00:01:32,105
But you should try to sort of
efficiently cover the things

31
00:01:32,105 --> 00:01:34,230
that I-- you know, the most
important things to say

32
00:01:34,230 --> 00:01:38,460
are roughly, of course,
describing the problem you

33
00:01:38,460 --> 00:01:47,430
chose, and it's always
good to say for a minute

34
00:01:47,430 --> 00:01:49,980
why it's important
or it's interesting.

35
00:01:57,180 --> 00:02:00,270
I'd like you to describe the
technical approach you took--

36
00:02:09,060 --> 00:02:10,919
you chose, and why you chose it.

37
00:02:16,830 --> 00:02:20,230
And then a quick
results, I mean,

38
00:02:20,230 --> 00:02:22,620
as much as you can
get in, and if you

39
00:02:22,620 --> 00:02:26,310
have any sort of interesting
implementation details

40
00:02:26,310 --> 00:02:29,380
that came up, that reared
their heads, that's always fun

41
00:02:29,380 --> 00:02:29,880
to say.

42
00:02:34,322 --> 00:02:36,780
You're not going to be able to
fit much into eight minutes.

43
00:02:36,780 --> 00:02:40,170
But tell us what
you've done so far.

44
00:02:40,170 --> 00:02:41,640
And it's typically pretty fun.

45
00:02:41,640 --> 00:02:43,015
You guys are not
going to believe

46
00:02:43,015 --> 00:02:45,030
how cool the projects
you've all picked are.

47
00:02:45,030 --> 00:02:46,733
It's going to be a lot of fun.

48
00:02:46,733 --> 00:02:48,900
We did-- so we're in this
room Tuesday and Thursday.

49
00:02:48,900 --> 00:02:53,330
I've asked them in the--

50
00:02:53,330 --> 00:02:55,830
because there's a real chance
that we might run a little bit

51
00:02:55,830 --> 00:02:57,150
over class, I asked
them if we could

52
00:02:57,150 --> 00:02:58,230
try to keep the room
for the next hour.

53
00:02:58,230 --> 00:02:59,580
They actually told
me that I could yet.

54
00:02:59,580 --> 00:03:01,290
But hopefully we'll
be able to stay here

55
00:03:01,290 --> 00:03:06,900
till 4:15 or something
if we need to to finish.

56
00:03:06,900 --> 00:03:15,426
And the report-- I said it
before, but the report is--

57
00:03:15,426 --> 00:03:17,610
is it working?

58
00:03:17,610 --> 00:03:19,615
AUDIENCE: No, the
same thing has--

59
00:03:19,615 --> 00:03:21,540
RUSS TEDRAKE: Why don't
you power it down.

60
00:03:21,540 --> 00:03:23,540
That's when it worked last time.

61
00:03:23,540 --> 00:03:24,040
Yeah.

62
00:03:30,885 --> 00:03:32,260
It's due May 21st,
I said, right?

63
00:03:32,260 --> 00:03:33,010
That's a Thursday.

64
00:03:41,060 --> 00:03:43,580
So if you choose
the double column--

65
00:03:43,580 --> 00:03:46,920
so I like to think of them
as ICRA format, for instance,

66
00:03:46,920 --> 00:03:49,060
would be perfect.

67
00:03:49,060 --> 00:03:52,820
That would be the
robotics conference.

68
00:03:52,820 --> 00:03:56,990
But I'm not asking for a novel,
just a quick presentation

69
00:03:56,990 --> 00:03:58,010
of what you did.

70
00:03:58,010 --> 00:03:59,927
And I think you should
have the same points as

71
00:03:59,927 --> 00:04:10,220
in the presentation,
but you'll have

72
00:04:10,220 --> 00:04:17,295
a little bit more room to
tell about the implementation.

73
00:04:20,145 --> 00:04:23,000
AUDIENCE: Is there a monetary
penalty for extra pages?

74
00:04:23,000 --> 00:04:24,418
RUSS TEDRAKE: No, not at all.

75
00:04:24,418 --> 00:04:24,960
That's right.

76
00:04:27,337 --> 00:04:29,420
No publishing costs, and
you can use color figures

77
00:04:29,420 --> 00:04:30,337
as much as you'd like.

78
00:04:33,770 --> 00:04:36,688
Any other questions
about the project?

79
00:04:36,688 --> 00:04:39,790
AUDIENCE: [INAUDIBLE]

80
00:04:39,790 --> 00:04:41,790
RUSS TEDRAKE: I'm just
giving you a ballpark six

81
00:04:41,790 --> 00:04:42,270
to eight pages.

82
00:04:42,270 --> 00:04:43,812
If you come in at
five, I'm not going

83
00:04:43,812 --> 00:04:45,450
to give you a huge
deduction if you say

84
00:04:45,450 --> 00:04:47,490
everything you needed to say.

85
00:04:47,490 --> 00:04:48,943
And you're free to--

86
00:04:48,943 --> 00:04:50,610
it at least means
there's no publication

87
00:04:50,610 --> 00:04:51,870
cost for extra pages, I guess.

88
00:04:58,140 --> 00:05:01,440
And six to eight pages
double column in ICRA format

89
00:05:01,440 --> 00:05:04,800
is more than six to eight
pages the other way.

90
00:05:04,800 --> 00:05:07,770
So it's a rough guideline.

91
00:05:07,770 --> 00:05:11,124
I'm just telling you,
you know, roughly--

92
00:05:11,124 --> 00:05:13,590
AUDIENCE: So we can just
reformat it [INAUDIBLE]

93
00:05:13,590 --> 00:05:14,493
six to eight pages?

94
00:05:14,493 --> 00:05:16,410
RUSS TEDRAKE: That's
what I'm trying to avoid.

95
00:05:16,410 --> 00:05:18,440
[LAUGHTER] Thank you.

96
00:05:18,440 --> 00:05:20,940
I'm just trying to give you a
sense of the amount of content

97
00:05:20,940 --> 00:05:22,500
that should be in there.

98
00:05:22,500 --> 00:05:26,380
And you can format
it as you would like.

99
00:05:26,380 --> 00:05:30,750
OK, excellent, so last
official lecture--

100
00:05:30,750 --> 00:05:36,270
so I wanted to, as promised, end
my part of the class, at least,

101
00:05:36,270 --> 00:05:40,050
on some success stories of
how all this stuff works

102
00:05:40,050 --> 00:05:47,210
and has worked in the research
world over the last 15 years.

103
00:05:47,210 --> 00:05:50,550
And so the ones I
picked out to talk

104
00:05:50,550 --> 00:05:55,620
about were Emilio Frazzoli's
helicopter motion planning,

105
00:05:55,620 --> 00:05:56,160
which--

106
00:05:56,160 --> 00:05:58,980
so Emilio is upstairs in LIDS.

107
00:05:58,980 --> 00:06:03,990
Some of the really primal--
some of the first good work

108
00:06:03,990 --> 00:06:05,790
on learning in robots
was by Chris Atkeson.

109
00:06:05,790 --> 00:06:07,165
And Stefan Schaal
was his student

110
00:06:07,165 --> 00:06:11,727
at the time, who is now quite
established, obviously, at USC.

111
00:06:11,727 --> 00:06:14,310
And then I'll tell you a little
bit about our perching project

112
00:06:14,310 --> 00:06:17,700
depending on how much
time we have left, OK?

113
00:06:17,700 --> 00:06:20,280
So let me start off
maybe with the lights

114
00:06:20,280 --> 00:06:27,570
on telling you some of the
big ideas from Emilio's

115
00:06:27,570 --> 00:06:32,185
planning for the helicopters.

116
00:06:32,185 --> 00:06:33,810
This was actually
Emilio's thesis work.

117
00:06:33,810 --> 00:06:35,160
And they've continued to use--

118
00:06:35,160 --> 00:06:37,830
nobody here's from
Emilio's group, are they?

119
00:06:37,830 --> 00:06:38,340
No.

120
00:06:38,340 --> 00:06:39,640
Anybody know Emilio?

121
00:06:39,640 --> 00:06:40,362
Yeah?

122
00:06:40,362 --> 00:06:41,820
He's on your
committee, OK perfect.

123
00:06:41,820 --> 00:06:43,020
That's good.

124
00:06:43,020 --> 00:06:44,385
So this work already.

125
00:06:44,385 --> 00:06:46,010
OK, correct me if I
say anything wrong.

126
00:06:53,530 --> 00:06:56,500
OK, so the big challenge
that Emilio wanted

127
00:06:56,500 --> 00:06:59,800
to address in his thesis was--

128
00:06:59,800 --> 00:07:03,610
well, he does everything very,
very rigorously and formally.

129
00:07:03,610 --> 00:07:06,460
So he's got many theoretical
contributions in his thesis.

130
00:07:06,460 --> 00:07:08,650
But the experiment that
he wanted to get working

131
00:07:08,650 --> 00:07:12,790
was we've got a helicopter,
let's say an RC class

132
00:07:12,790 --> 00:07:13,750
helicopter.

133
00:07:13,750 --> 00:07:15,610
And I want to drive
it from point A

134
00:07:15,610 --> 00:07:19,780
to point B through potentially
a very cluttered environment

135
00:07:19,780 --> 00:07:22,340
where the obstacles are
even moving in real time.

136
00:07:22,340 --> 00:07:26,710
So he really wanted a real time
lightweight planning algorithm

137
00:07:26,710 --> 00:07:29,350
that was as close to optimal
as possible in a minimum time

138
00:07:29,350 --> 00:07:31,660
optimality sense but
that could run on

139
00:07:31,660 --> 00:07:33,940
a-- you know, with 1 megabyte
of memory or something

140
00:07:33,940 --> 00:07:37,390
on a computer that's on
board the helicopter.

141
00:07:37,390 --> 00:07:39,790
And what he ended up doing
was a beautiful combination

142
00:07:39,790 --> 00:07:44,620
of feedback control and
randomized motion planning.

143
00:07:44,620 --> 00:07:47,180
And I want to tell you
some of those details.

144
00:07:47,180 --> 00:07:49,570
OK, so the first
big idea in the--

145
00:08:01,840 --> 00:08:06,880
if you talk to Emilio
about doing motion planning

146
00:08:06,880 --> 00:08:11,800
and you say I've got some
nonlinear system f of x, u,

147
00:08:11,800 --> 00:08:17,002
and I want to find a path
from start to the finish,

148
00:08:17,002 --> 00:08:19,210
and I've got obstacles,
whatever, the standard motion

149
00:08:19,210 --> 00:08:20,500
planning problem--

150
00:08:20,500 --> 00:08:22,943
but these obstacles
could be moving--

151
00:08:22,943 --> 00:08:24,610
the first thing he
would immediately say

152
00:08:24,610 --> 00:08:30,700
and he has said to me is never
ever search over u's. That's

153
00:08:30,700 --> 00:08:32,857
just seems like a
bad idea in general.

154
00:08:32,857 --> 00:08:35,440
The [INAUDIBLE] that we talked
about, the naive implementation

155
00:08:35,440 --> 00:08:38,740
is pick a point
at random and then

156
00:08:38,740 --> 00:08:41,620
try to find the closest
point, and then find

157
00:08:41,620 --> 00:08:43,962
the u which takes you as
close as possible to that.

158
00:08:43,962 --> 00:08:45,670
He says, if you're
doing that, you're not

159
00:08:45,670 --> 00:08:49,692
thinking about feedback
stabilization, for instance,

160
00:08:49,692 --> 00:08:52,150
and you're working potentially
on a harder dynamical system

161
00:08:52,150 --> 00:08:54,320
than you need to
be working with.

162
00:08:54,320 --> 00:08:59,080
So he advocates first designing
a class of feedback policies.

163
00:08:59,080 --> 00:09:03,010
Let's say it's-- they could be
time varying or time invariant,

164
00:09:03,010 --> 00:09:04,870
right?

165
00:09:04,870 --> 00:09:07,780
And they have maybe some--

166
00:09:07,780 --> 00:09:09,280
they're parameterized,
for instance,

167
00:09:09,280 --> 00:09:10,690
by some desired point.

168
00:09:23,530 --> 00:09:26,710
And these feedback policies if--

169
00:09:26,710 --> 00:09:28,750
ideally you generate
them carefully

170
00:09:28,750 --> 00:09:32,810
and have some provable
stability guarantees about them.

171
00:09:32,810 --> 00:09:36,098
And instead, then, of picking in
some u, which could potentially

172
00:09:36,098 --> 00:09:38,140
be working on a system
that's very unstable, open

173
00:09:38,140 --> 00:09:41,530
loop unstable, it might
require tight integration,

174
00:09:41,530 --> 00:09:43,510
all these things.

175
00:09:43,510 --> 00:09:46,835
Instead, you should sample over
the parameters of your feedback

176
00:09:46,835 --> 00:09:47,335
policy.

177
00:09:49,960 --> 00:09:51,770
So, for instance--

178
00:09:51,770 --> 00:09:53,590
I'll make it more
specific in a second,

179
00:09:53,590 --> 00:09:55,840
but for instance, if
I take a point here,

180
00:09:55,840 --> 00:09:58,637
I find my closest
point on the map,

181
00:09:58,637 --> 00:10:00,220
and I want to grow
towards it, instead

182
00:10:00,220 --> 00:10:04,510
of trying a bunch of u's,
I say, what if I ran pi 1

183
00:10:04,510 --> 00:10:06,220
with x desired of this?

184
00:10:06,220 --> 00:10:09,160
How close-- x desired equals,
let's say, this point,

185
00:10:09,160 --> 00:10:10,660
how close would it get me?

186
00:10:10,660 --> 00:10:13,810
Or let's say I
run-- or maybe pi 10

187
00:10:13,810 --> 00:10:15,190
would have been the closest one.

188
00:10:15,190 --> 00:10:16,990
I'll pick the closest
feedback controller

189
00:10:16,990 --> 00:10:19,720
that's parameterized
that's going to try

190
00:10:19,720 --> 00:10:22,680
to get me close to that point.

191
00:10:22,680 --> 00:10:25,780
OK, there's a lot of
benefits to doing that.

192
00:10:25,780 --> 00:10:28,030
In a very real
way, it's talking--

193
00:10:28,030 --> 00:10:30,933
we talked about feedback motion
planning as having advantages.

194
00:10:30,933 --> 00:10:31,600
You know, if I--

195
00:10:31,600 --> 00:10:34,000
I could come up with
a trajectory here

196
00:10:34,000 --> 00:10:36,340
in the open loop sense that
if I went to stabilize,

197
00:10:36,340 --> 00:10:37,660
it was not stabilizable.

198
00:10:37,660 --> 00:10:40,900
This avoids that shortcoming
by actually searching

199
00:10:40,900 --> 00:10:42,160
in the space of feedback laws.

200
00:10:44,890 --> 00:10:48,070
And it also has
advantages, for instance,

201
00:10:48,070 --> 00:10:52,570
that the dynamical system you're
simulating is probably stable.

202
00:10:52,570 --> 00:10:55,443
So you could take larger
steps, integration steps.

203
00:10:55,443 --> 00:10:57,610
You don't have to integrate
very carefully if you've

204
00:10:57,610 --> 00:10:58,510
got a stable system.

205
00:11:03,050 --> 00:11:09,020
So Emilio has advocated this
sort of trim and maneuver

206
00:11:09,020 --> 00:11:10,940
view of the world
in motion planning,

207
00:11:10,940 --> 00:11:12,320
especially on the helicopters.

208
00:11:25,040 --> 00:11:27,770
So each of these feedback laws
that he considers at the motion

209
00:11:27,770 --> 00:11:32,510
planning time are some
carefully designed controller.

210
00:11:32,510 --> 00:11:34,590
They're of two types.

211
00:11:34,590 --> 00:11:36,718
One of them is a
trim controller.

212
00:11:36,718 --> 00:11:38,760
A trim controller for a
helicopter, for instance,

213
00:11:38,760 --> 00:11:42,080
would be the hover, or
something that flies forward

214
00:11:42,080 --> 00:11:46,860
with a steady speed, something
that banks with a steady speed.

215
00:11:46,860 --> 00:11:49,460
All of these things
are trims where,

216
00:11:49,460 --> 00:11:53,780
in the relative frame of the
helicopter, it's a fixed point.

217
00:11:56,820 --> 00:12:23,000
All right, so-- and Emilio
was very careful in his work

218
00:12:23,000 --> 00:12:27,560
to point out that you can
make heavy use of symmetries

219
00:12:27,560 --> 00:12:29,420
and invariants in
those dynamics.

220
00:12:29,420 --> 00:12:33,723
So, for instance, the horizontal
position of the helicopter

221
00:12:33,723 --> 00:12:34,640
doesn't really matter.

222
00:12:34,640 --> 00:12:39,560
If I know how to go forward from
x equals 0 in a trim condition,

223
00:12:39,560 --> 00:12:42,260
then I also know how to
go forward from x equals 1

224
00:12:42,260 --> 00:12:44,000
in a position, right?

225
00:12:44,000 --> 00:12:46,850
If I know how to bank
at a certain speed,

226
00:12:46,850 --> 00:12:48,980
I can do that from
any x location.

227
00:12:48,980 --> 00:12:51,590
The location of the helicopter
doesn't affect that feedback

228
00:12:51,590 --> 00:12:52,557
policy, OK?

229
00:13:06,050 --> 00:13:07,683
All right, so if
you have a few--

230
00:13:07,683 --> 00:13:09,350
what that allows you
to do, essentially,

231
00:13:09,350 --> 00:13:13,220
is come up with a handful
of trim controllers

232
00:13:13,220 --> 00:13:15,230
which do a lot of things.

233
00:13:15,230 --> 00:13:18,080
In fact, what I think
he does in the end

234
00:13:18,080 --> 00:13:23,210
is he comes up
with at around 10--

235
00:13:23,210 --> 00:13:24,443
I think 25 was the number.

236
00:13:24,443 --> 00:13:26,360
I remember-- I brought
his paper just in case.

237
00:13:36,340 --> 00:13:50,780
Yeah, so he comes up with
25 trim trajectories, which

238
00:13:50,780 --> 00:13:55,850
in his case was level
flight, no sideslip,

239
00:13:55,850 --> 00:14:02,510
with forward velocities tiled
over, just to give you a sense

240
00:14:02,510 --> 00:14:15,080
here, 0, 1.25, 2.5, 5, and
10 meters per second and then

241
00:14:15,080 --> 00:14:23,360
angular heading
rates that are sort

242
00:14:23,360 --> 00:14:32,480
of similar from negative 1,
negative 0.5, 0, 0.5, and 1

243
00:14:32,480 --> 00:14:35,530
radians per second.

244
00:14:35,530 --> 00:14:38,830
OK, so if he knows
how to do those 25

245
00:14:38,830 --> 00:14:41,200
things, every
combination of those,

246
00:14:41,200 --> 00:14:45,550
then he's got a pretty darn
good library of trim behaviors

247
00:14:45,550 --> 00:14:48,100
that he can use as
possible controllers

248
00:14:48,100 --> 00:14:51,460
that he could evaluate
on his system.

249
00:14:51,460 --> 00:14:59,770
Now those are the
steady state behaviors.

250
00:14:59,770 --> 00:15:02,350
And in order to
transition between trims

251
00:15:02,350 --> 00:15:10,420
he does these
maneuvers, which are

252
00:15:10,420 --> 00:15:27,040
finite time
stabilized transitions

253
00:15:27,040 --> 00:15:28,655
between trim conditions.

254
00:15:41,125 --> 00:15:43,000
And those maneuvers are
actually pretty cool.

255
00:15:43,000 --> 00:15:43,583
I'll show you.

256
00:15:43,583 --> 00:15:46,480
So some of them can be--
so, for instance, he

257
00:15:46,480 --> 00:15:50,470
learned a maneuver that was the
transition from going forward

258
00:15:50,470 --> 00:15:53,440
this way to being in the
opposite orientation going

259
00:15:53,440 --> 00:15:54,685
forward this way.

260
00:15:54,685 --> 00:15:56,560
And that maneuver-- I
shouldn't say learned--

261
00:15:56,560 --> 00:15:58,780
he solved using a model
of the helicopter, right,

262
00:15:58,780 --> 00:16:03,740
with dercall or something like
that, a trajectory that was

263
00:16:03,740 --> 00:16:04,990
actually a snap roll, I think.

264
00:16:04,990 --> 00:16:07,330
It kind of went up, and did this
thing, and then came back down

265
00:16:07,330 --> 00:16:08,288
and went the other way.

266
00:16:08,288 --> 00:16:10,848
And I'll show you a video that.

267
00:16:10,848 --> 00:16:12,640
I had lunch with Emilio
and get his videos.

268
00:16:16,947 --> 00:16:19,030
OK, so how does he design
the feedback controllers

269
00:16:19,030 --> 00:16:21,310
for the trims?

270
00:16:21,310 --> 00:16:24,520
He uses value iteration, right?

271
00:16:24,520 --> 00:16:28,015
So we're current
and relevant, see?

272
00:16:28,015 --> 00:16:29,890
So because he's got
actually a relatively low

273
00:16:29,890 --> 00:16:31,860
dimensional space by
the time he exploits

274
00:16:31,860 --> 00:16:33,610
these feedback invariants
and just worries

275
00:16:33,610 --> 00:16:37,000
about the coordinates that
matter in these things,

276
00:16:37,000 --> 00:16:40,810
he's actually able to tile the
space and run value iteration.

277
00:16:40,810 --> 00:16:44,110
And the advantage of that,
of getting these trims,

278
00:16:44,110 --> 00:16:49,000
is that he not only gets
the nominal controller,

279
00:16:49,000 --> 00:16:53,260
but he gets the cost to
go from value iteration.

280
00:16:53,260 --> 00:16:55,030
For the maneuvers,
he uses some sort

281
00:16:55,030 --> 00:17:00,520
of back stepping optimization,
and he feedback stabilizes.

282
00:17:00,520 --> 00:17:04,250
And he uses as an estimate of
the cost to go on the maneuvers

283
00:17:04,250 --> 00:17:05,500
just the duration of the time.

284
00:17:05,500 --> 00:17:07,250
Everything's a
minimum time problem.

285
00:17:07,250 --> 00:17:09,730
So he uses the duration of
the trajectory as the cost

286
00:17:09,730 --> 00:17:13,210
to go of these maneuvers.

287
00:17:13,210 --> 00:17:22,930
And now when you go back
into the planning problem,

288
00:17:22,930 --> 00:17:24,069
the search goes like this.

289
00:17:24,069 --> 00:17:26,109
I pick a random point.

290
00:17:26,109 --> 00:17:28,662
I start now asking every--

291
00:17:28,662 --> 00:17:30,370
each of the existing
points in my graph--

292
00:17:30,370 --> 00:17:32,050
let me give it an
initial graph here,

293
00:17:32,050 --> 00:17:35,000
just to make it a little
bit more interesting.

294
00:17:35,000 --> 00:17:36,520
I ask all of the
points on my graph

295
00:17:36,520 --> 00:17:39,250
not if they're close
in the Euclidean sense,

296
00:17:39,250 --> 00:17:42,400
but what's the
cost to go, what's

297
00:17:42,400 --> 00:17:46,870
the value function of trying
to go from here to here.

298
00:17:46,870 --> 00:17:49,510
Each of them will vote
on the cost to go.

299
00:17:49,510 --> 00:17:51,670
And he takes-- he
starts-- he considers--

300
00:17:51,670 --> 00:17:54,580
he tries to go in order
of the minimum cost

301
00:17:54,580 --> 00:17:55,690
to the maximum cost.

302
00:17:58,750 --> 00:18:02,260
Once he decides to try-- this
one has the lowest cost to go.

303
00:18:02,260 --> 00:18:04,210
He actually tries to
expand that controller

304
00:18:04,210 --> 00:18:06,880
in the direction
of here and checks

305
00:18:06,880 --> 00:18:10,960
whether it's actually
feasible to running this pi 1

306
00:18:10,960 --> 00:18:14,670
and with this x desired, whether
it can feasibly get here,

307
00:18:14,670 --> 00:18:17,170
for instance, without-- if there
was an obstacle right here,

308
00:18:17,170 --> 00:18:20,860
let's say, then it might be
that the motion plan is not

309
00:18:20,860 --> 00:18:23,140
feasible to get
from there to there.

310
00:18:23,140 --> 00:18:26,140
So he throws it out and
takes the next lowest cost.

311
00:18:26,140 --> 00:18:28,540
And every time he
expands a node,

312
00:18:28,540 --> 00:18:30,040
he'll pick the next lowest.

313
00:18:30,040 --> 00:18:32,420
He'll search in these,
try all the policies

314
00:18:32,420 --> 00:18:34,090
until he finds one
that's feasible

315
00:18:34,090 --> 00:18:35,440
and tries to grow to there.

316
00:18:38,090 --> 00:18:40,798
The result, actually, when
he puts it all together--

317
00:18:40,798 --> 00:18:42,340
you should read the
papers, actually.

318
00:18:42,340 --> 00:18:44,050
This is a-- it's
hard to do justice

319
00:18:44,050 --> 00:18:47,092
in such a complicated
piece of work.

320
00:18:47,092 --> 00:18:49,300
He's really-- he thinks of
everything, it feels like,

321
00:18:49,300 --> 00:18:51,257
in his thesis.

322
00:18:51,257 --> 00:18:53,590
The five minute presentation
here doesn't do it justice.

323
00:18:53,590 --> 00:18:55,360
But he's got an
algorithm that works

324
00:18:55,360 --> 00:18:58,720
fast enough that it
works in real time

325
00:18:58,720 --> 00:19:01,090
even if the
obstacles are moving.

326
00:19:01,090 --> 00:19:03,070
Because he's
computed-- the big idea

327
00:19:03,070 --> 00:19:05,920
there is that you
used value iteration

328
00:19:05,920 --> 00:19:10,607
to compute the cost to go for
the non-obstacle case, right?

329
00:19:10,607 --> 00:19:12,190
You know how to
control the helicopter

330
00:19:12,190 --> 00:19:13,658
when there's no obstacles.

331
00:19:13,658 --> 00:19:15,700
And you use that as your
heuristic, your distance

332
00:19:15,700 --> 00:19:20,705
metric, for the case where
you're planning with obstacles.

333
00:19:20,705 --> 00:19:22,330
And it works fast
enough that he can be

334
00:19:22,330 --> 00:19:23,980
flying his helicopter around--

335
00:19:23,980 --> 00:19:26,110
he's got simulations of
like windows closing,

336
00:19:26,110 --> 00:19:27,340
and it goes a different way.

337
00:19:27,340 --> 00:19:30,260
Or the window's open, and he
goes through this minimum time

338
00:19:30,260 --> 00:19:30,760
way.

339
00:19:30,760 --> 00:19:32,657
And a lot of times
they still accomplish

340
00:19:32,657 --> 00:19:34,240
finding a minimum
time trajectory from

341
00:19:34,240 --> 00:19:37,270
start to the goal, even
with dynamic obstacles.

342
00:19:37,270 --> 00:19:39,670
It's pretty good,
pretty good stuff.

343
00:19:39,670 --> 00:19:41,856
Yeah?

344
00:19:41,856 --> 00:19:46,490
AUDIENCE: So when you decide to
expand [INAUDIBLE] tree based

345
00:19:46,490 --> 00:19:48,910
on the value iteration
result, and then you

346
00:19:48,910 --> 00:19:53,860
simulate it in [INAUDIBLE]
came across an obstacle.

347
00:19:53,860 --> 00:19:56,470
So the values there, is that
actually the q function,

348
00:19:56,470 --> 00:19:58,030
or is it the v function?

349
00:19:58,030 --> 00:20:01,050
RUSS TEDRAKE: It's a value
function, v function--

350
00:20:01,050 --> 00:20:01,788
j.

351
00:20:01,788 --> 00:20:06,640
AUDIENCE: OK, so
if that actually

352
00:20:06,640 --> 00:20:09,010
happens to cross
an obstacle, there

353
00:20:09,010 --> 00:20:13,498
might be another action getting
[INAUDIBLE] from that space.

354
00:20:13,498 --> 00:20:15,290
And it wouldn't still
have the lowest cost.

355
00:20:15,290 --> 00:20:18,730
But because the value iteration,
you'd just say the main--

356
00:20:18,730 --> 00:20:22,420
[INAUDIBLE] another state
which has the lowest value.

357
00:20:22,420 --> 00:20:23,420
RUSS TEDRAKE: It's true.

358
00:20:23,420 --> 00:20:29,005
So he's careful to say that
when you go to this trims

359
00:20:29,005 --> 00:20:30,880
and maneuver-- whenever
you're using feedback

360
00:20:30,880 --> 00:20:33,610
controllers in here, one of the
things, you have to sacrifice,

361
00:20:33,610 --> 00:20:36,940
actually, not only optimality,
but completeness, actually,

362
00:20:36,940 --> 00:20:39,280
is that you can
only now be complete

363
00:20:39,280 --> 00:20:43,540
if the solution lies in this
class of feedback policies.

364
00:20:43,540 --> 00:20:44,680
So you're sort of--

365
00:20:44,680 --> 00:20:48,820
he calls it pi complete
or something like this.

366
00:20:48,820 --> 00:20:51,490
If there's a path from
the start to the goal that

367
00:20:51,490 --> 00:20:55,450
is executed by these predesigned
feedback-- parameterized

368
00:20:55,450 --> 00:20:57,790
feedback policies,
then you'll find it.

369
00:20:57,790 --> 00:20:59,110
But you give up completeness.

370
00:20:59,110 --> 00:21:00,568
And potentially
give up optimality.

371
00:21:00,568 --> 00:21:03,040
Now, I would guess
that if it was--

372
00:21:03,040 --> 00:21:05,998
there are cases where he
can try multiple things

373
00:21:05,998 --> 00:21:08,290
and actually get around the
immediate problem you said.

374
00:21:08,290 --> 00:21:09,665
But I'm sure-- I
would guess also

375
00:21:09,665 --> 00:21:11,530
that there's cases
where he does say

376
00:21:11,530 --> 00:21:13,072
I've just missed that solution.

377
00:21:13,072 --> 00:21:15,280
So I'm sure there's some
things where you could try--

378
00:21:15,280 --> 00:21:17,740
for instance, maybe pi 1
would have said it's feasible,

379
00:21:17,740 --> 00:21:19,238
but there's an obstacle.

380
00:21:19,238 --> 00:21:21,780
But then maybe if I ran pi 10
it would actually get me there.

381
00:21:25,205 --> 00:21:26,330
Let me see what he gave me.

382
00:21:26,330 --> 00:21:34,950
He gave-- so this is views
from his small helicopter.

383
00:21:44,095 --> 00:21:45,470
AUDIENCE: Does--
this state space

384
00:21:45,470 --> 00:21:47,746
has a velocity inside,
or [INAUDIBLE]??

385
00:21:50,867 --> 00:21:53,200
RUSS TEDRAKE: The state space
absolutely has velocities.

386
00:21:53,200 --> 00:21:55,990
It's very much about learning
a dynamic controller.

387
00:21:55,990 --> 00:21:58,300
Some of these are invariants.

388
00:21:58,300 --> 00:22:02,068
The obvious invariants are
in position and translation.

389
00:22:04,410 --> 00:22:06,910
So the velocities maybe are not
one of the invariant-- maybe

390
00:22:06,910 --> 00:22:08,470
they're supposedly
reasoned about.

391
00:22:08,470 --> 00:22:12,790
But yes, it's very much a
[INAUDIBLE] dynamic planner.

392
00:22:12,790 --> 00:22:13,753
So here's the on board.

393
00:22:13,753 --> 00:22:15,170
This is sort of
the success story.

394
00:22:15,170 --> 00:22:21,478
He's-- an on board video,
and you'll see a few of the--

395
00:22:21,478 --> 00:22:23,770
I think this is the longer
version that we already had,

396
00:22:23,770 --> 00:22:27,730
because you'll see some flips
and stuff in the middle.

397
00:22:31,880 --> 00:22:33,790
So this is, I think, the--

398
00:22:33,790 --> 00:22:35,357
no he called it a hammerhead.

399
00:22:35,357 --> 00:22:36,940
I don't know all my
helicopter stunts.

400
00:22:36,940 --> 00:22:39,490
This is the maneuver which
changed directions pretty

401
00:22:39,490 --> 00:22:41,310
quickly, going back and around.

402
00:22:48,290 --> 00:22:50,240
So they're doing pretty
aggressive things here

403
00:22:50,240 --> 00:22:57,020
with, given the model, very
strong convergence guarantees.

404
00:23:00,470 --> 00:23:03,687
OK, so especially
in these vehicles--

405
00:23:03,687 --> 00:23:05,270
they're doing it now
with the forklift

406
00:23:05,270 --> 00:23:08,420
to do inserting into a pallet,
and lift up pallets, and sort

407
00:23:08,420 --> 00:23:10,790
of moving around
boxes and everything.

408
00:23:10,790 --> 00:23:13,850
These real-time motion
planning strategies

409
00:23:13,850 --> 00:23:17,210
based on clever implementations
of the RRTs, they're real.

410
00:23:17,210 --> 00:23:18,980
They work.

411
00:23:18,980 --> 00:23:21,660
And value iteration is
sort of real, and it works.

412
00:23:21,660 --> 00:23:23,637
I asked Emilio what he
thought about the fact

413
00:23:23,637 --> 00:23:25,970
that value iteration doesn't
solve the double integrator

414
00:23:25,970 --> 00:23:27,770
rate.

415
00:23:27,770 --> 00:23:29,780
And he was actually
very interested.

416
00:23:29,780 --> 00:23:32,510
But it's sort of--

417
00:23:32,510 --> 00:23:33,800
it's good enough.

418
00:23:33,800 --> 00:23:36,710
For the purposes of this,
it wasn't-- it needn't be

419
00:23:36,710 --> 00:23:40,520
a perfectly optimal controller,
just a coarse discretization,

420
00:23:40,520 --> 00:23:41,522
solve it pretty fast.

421
00:23:41,522 --> 00:23:43,730
That was good enough to give
you a working controller

422
00:23:43,730 --> 00:23:45,920
with some stability
guarantees to use

423
00:23:45,920 --> 00:23:49,280
in the rest of the
planning algorithms.

424
00:23:49,280 --> 00:23:54,233
AUDIENCE: [INAUDIBLE]

425
00:23:54,233 --> 00:23:56,150
RUSS TEDRAKE: It's just
dynamically replanning

426
00:23:56,150 --> 00:23:56,692
all the time.

427
00:23:56,692 --> 00:24:01,700
AUDIENCE: Did he have like all
the information [INAUDIBLE]??

428
00:24:01,700 --> 00:24:04,377
RUSS TEDRAKE: So I think
that the strongest results

429
00:24:04,377 --> 00:24:06,710
in the moving, I think, were
in a simulation environment

430
00:24:06,710 --> 00:24:09,020
where it was fully known.

431
00:24:09,020 --> 00:24:09,830
I don't know if--

432
00:24:09,830 --> 00:24:12,620
I don't honestly know if
they had moving obstacles

433
00:24:12,620 --> 00:24:14,935
in their physical demos.

434
00:24:14,935 --> 00:24:15,560
That'd be cool.

435
00:24:20,700 --> 00:24:24,570
OK, good, so evidence one that
these things really work, yeah?

436
00:24:27,390 --> 00:24:29,490
I can't sort of talk about--

437
00:24:29,490 --> 00:24:32,130
I can't do a whole class
talking about learning control

438
00:24:32,130 --> 00:24:35,315
without mentioning Chris
and Stefan who are really

439
00:24:35,315 --> 00:24:36,690
the guys that got
a lot of people

440
00:24:36,690 --> 00:24:39,940
excited about these things.

441
00:24:39,940 --> 00:24:42,120
So let me show you
some of their work.

442
00:24:54,190 --> 00:24:57,040
Chris apparently also had
some helicopter stunts

443
00:24:57,040 --> 00:24:58,390
that he did while he was at MIT.

444
00:24:58,390 --> 00:24:59,998
And I haven't seen those videos.

445
00:24:59,998 --> 00:25:01,540
But I hear they were
pretty good too.

446
00:25:01,540 --> 00:25:05,120
So helicopters seem
to be a popular thing.

447
00:25:05,120 --> 00:25:09,510
So this is a case of learning
sort of a cart pull problem.

448
00:25:09,510 --> 00:25:15,700
This is a pole balancing task
with a big arm, a Sarcos arm.

449
00:25:15,700 --> 00:25:17,830
The arm has actually
been sort of

450
00:25:17,830 --> 00:25:21,130
mapped out with inverse
kinematics and inverse dynamics

451
00:25:21,130 --> 00:25:21,630
control.

452
00:25:21,630 --> 00:25:23,710
So really, he's moving the sled.

453
00:25:23,710 --> 00:25:27,428
You can think of it almost
equivalently as a 2D cart--

454
00:25:27,428 --> 00:25:28,720
and a pole that's not attached.

455
00:25:28,720 --> 00:25:30,303
It's not a pin joint.

456
00:25:30,303 --> 00:25:31,720
There's vision
coming off the side

457
00:25:31,720 --> 00:25:34,030
with-- that's why there's
a bright pink ball

458
00:25:34,030 --> 00:25:35,840
and a bright yellow ball.

459
00:25:35,840 --> 00:25:37,900
So there's a vision
system watching,

460
00:25:37,900 --> 00:25:39,340
doing the state estimation.

461
00:25:39,340 --> 00:25:42,530
And they did a lot of work.

462
00:25:42,530 --> 00:25:45,550
This was, really, some of the
first people that were making

463
00:25:45,550 --> 00:25:47,740
real robots use these tools.

464
00:25:47,740 --> 00:25:50,193
They were using cue
learning, value learning,

465
00:25:50,193 --> 00:25:52,360
and they call it model-based
reinforcement learning.

466
00:25:52,360 --> 00:25:55,297
Because they were trying
to distinguish between two

467
00:25:55,297 --> 00:25:56,380
versions of the algorithm.

468
00:25:56,380 --> 00:26:00,460
One was do system
identification on every trial.

469
00:26:00,460 --> 00:26:04,210
Between trials,
solve your offline--

470
00:26:04,210 --> 00:26:06,838
on that model, solve your
reinforcement learning problem.

471
00:26:06,838 --> 00:26:08,380
And then put the
best controller back

472
00:26:08,380 --> 00:26:13,580
on the robot versus doing the
trials exactly on the robot.

473
00:26:13,580 --> 00:26:15,220
So this was their
model-based case.

474
00:26:15,220 --> 00:26:17,920
And they argued heavily
that in these robots

475
00:26:17,920 --> 00:26:20,170
it made a lot more
sense to use a model,

476
00:26:20,170 --> 00:26:24,220
because even if the
model was designed

477
00:26:24,220 --> 00:26:26,200
from a very few
trajectories, they

478
00:26:26,200 --> 00:26:28,510
could learn a good
enough model of the robot

479
00:26:28,510 --> 00:26:31,810
that the task would be
completed in just a few trials.

480
00:26:49,600 --> 00:26:51,498
OK, so eight real
trials-- and that's

481
00:26:51,498 --> 00:26:53,290
the only data they had
from the real robot.

482
00:26:53,290 --> 00:26:55,060
They had some initial
guess at the parameters.

483
00:26:55,060 --> 00:26:56,477
But then the system
identification

484
00:26:56,477 --> 00:27:00,680
really happened each time.

485
00:27:00,680 --> 00:27:01,930
They took the tennis ball off.

486
00:27:12,658 --> 00:27:14,450
AUDIENCE: Is the range
of motion of the arm

487
00:27:14,450 --> 00:27:18,280
limited, or is it just losing
control when the pole falls?

488
00:27:18,280 --> 00:27:20,030
RUSS TEDRAKE: I think
it's losing control.

489
00:27:20,030 --> 00:27:21,488
I think they're
trying to be safely

490
00:27:21,488 --> 00:27:22,922
in the middle of the workspace.

491
00:27:26,160 --> 00:27:29,040
OK, now the modeling,
the system identification

492
00:27:29,040 --> 00:27:31,380
was only on the pole.

493
00:27:31,380 --> 00:27:34,213
The arm was well known,
well characterized.

494
00:27:34,213 --> 00:27:36,630
So they just learned a handful
of parameters for the pole.

495
00:27:36,630 --> 00:27:37,963
But they did it in a few trials.

496
00:27:37,963 --> 00:27:40,260
And then they showed
that the policy

497
00:27:40,260 --> 00:27:43,020
could adjust that quickly.

498
00:27:43,020 --> 00:27:45,780
And they became strong
advocates for model-based,

499
00:27:45,780 --> 00:27:46,780
doing it offline.

500
00:27:46,780 --> 00:27:48,240
Now, I would say that
in some of the problems

501
00:27:48,240 --> 00:27:49,990
we've talked about in
class, for instance,

502
00:27:49,990 --> 00:27:52,080
the fluid stuff that
Jon told you about,

503
00:27:52,080 --> 00:27:54,660
that's a clear case--

504
00:27:54,660 --> 00:27:55,980
I think, we think--

505
00:27:55,980 --> 00:27:58,380
that learning the model
is not the shortest path

506
00:27:58,380 --> 00:28:01,080
to getting success.

507
00:28:01,080 --> 00:28:04,702
But on a robot arm, maybe it is.

508
00:28:04,702 --> 00:28:06,660
They did that in a lot
of different cool tasks.

509
00:28:06,660 --> 00:28:12,000
So they did, for
instance, double sticking.

510
00:28:12,000 --> 00:28:14,310
They had a bunch
of different tasks

511
00:28:14,310 --> 00:28:16,410
that worked with
sort of similar--

512
00:28:31,982 --> 00:28:33,190
AUDIENCE: Is it using vision?

513
00:28:35,853 --> 00:28:37,270
RUSS TEDRAKE: The
other ones were.

514
00:28:37,270 --> 00:28:38,890
I would guess, since there's
no bright orange balls

515
00:28:38,890 --> 00:28:40,150
and there is a stick,
that there's probably

516
00:28:40,150 --> 00:28:41,492
some sensor on the other side.

517
00:28:41,492 --> 00:28:42,200
But I don't know.

518
00:28:42,200 --> 00:28:43,325
I don't remember, actually.

519
00:28:43,325 --> 00:28:45,545
AUDIENCE: It's got
strings [INAUDIBLE]..

520
00:28:47,137 --> 00:28:49,720
RUSS TEDRAKE: Yeah, I don't know
how they're doing the sensor.

521
00:28:49,720 --> 00:28:50,240
AUDIENCE: I thought
that was just

522
00:28:50,240 --> 00:28:51,410
to keep it from [INAUDIBLE].

523
00:28:51,410 --> 00:28:52,910
RUSS TEDRAKE: From
wiping out, yeah,

524
00:28:52,910 --> 00:28:54,350
from hitting the experimenter.

525
00:28:57,710 --> 00:29:00,630
OK, so this was really a
cool time in robotics, right?

526
00:29:00,630 --> 00:29:02,030
So the people, they're coming
in, and they're doing all--

527
00:29:02,030 --> 00:29:04,010
making these robots do
things they hadn't done.

528
00:29:04,010 --> 00:29:05,840
I'm sure cracking
a whip or something

529
00:29:05,840 --> 00:29:09,920
was on the near horizon.

530
00:29:09,920 --> 00:29:12,290
But interestingly--
so I have to tell you

531
00:29:12,290 --> 00:29:15,860
that after a bunch of
experiments here and pushing

532
00:29:15,860 --> 00:29:18,590
it and pushing it, I think
it's fair to say that Chris

533
00:29:18,590 --> 00:29:21,470
and Stefan decided that
reinforcement learning

534
00:29:21,470 --> 00:29:24,560
by itself was not the route
to super advanced robots.

535
00:29:24,560 --> 00:29:26,840
And they switched to
working more heavily

536
00:29:26,840 --> 00:29:29,210
on imitation learning,
which was still done

537
00:29:29,210 --> 00:29:31,430
sort of in a reinforcement
learning context.

538
00:29:31,430 --> 00:29:33,710
But they tried to
speed up learning

539
00:29:33,710 --> 00:29:38,640
by getting trials from humans.

540
00:29:38,640 --> 00:29:48,080
So there's a couple
of ways they did that.

541
00:29:48,080 --> 00:29:50,510
First of all, they'd try to
learn the reward function

542
00:29:50,510 --> 00:29:51,710
from the human.

543
00:29:51,710 --> 00:29:54,200
Sometimes they'd try to
learn the dynamic model

544
00:29:54,200 --> 00:29:55,160
from the human.

545
00:29:55,160 --> 00:29:57,320
Sometimes they would try
to prime a value function

546
00:29:57,320 --> 00:29:59,507
from a human, from
trials from a human.

547
00:29:59,507 --> 00:30:01,340
They'd play all these
sort of-- at any place

548
00:30:01,340 --> 00:30:02,965
in the reinforcement
learning framework

549
00:30:02,965 --> 00:30:05,750
where you could expect to try
to use information from humans,

550
00:30:05,750 --> 00:30:07,550
these are the kind of games
that people were playing

551
00:30:07,550 --> 00:30:08,425
and continue to play.

552
00:30:08,425 --> 00:30:12,410
The new helicopter results by
Pieter Abbeel and Andrew Ng

553
00:30:12,410 --> 00:30:15,530
are really a success story
for imitation learning,

554
00:30:15,530 --> 00:30:17,270
because they used
human pilots to drive

555
00:30:17,270 --> 00:30:21,780
those initial trajectories.

556
00:30:21,780 --> 00:30:24,710
So this is the pole
balancing with a human.

557
00:30:30,614 --> 00:30:32,262
AUDIENCE: [INAUDIBLE]

558
00:30:32,262 --> 00:30:33,720
RUSS TEDRAKE:
They're just watching

559
00:30:33,720 --> 00:30:38,160
the dynamics of the pole, but
with a controller that works.

560
00:30:38,160 --> 00:30:40,500
And they can use
that to sort of--

561
00:30:40,500 --> 00:30:43,050
for instance, you could
just find the policy that--

562
00:30:43,050 --> 00:30:48,420
[LAUGHTER]

563
00:30:48,420 --> 00:30:49,230
That's Chris.

564
00:31:35,090 --> 00:31:39,200
This is Auk, who was
just working with Stefan.

565
00:31:39,200 --> 00:31:44,330
They were having DB play tennis,
and do drums, all these things.

566
00:31:49,300 --> 00:31:53,773
[LAUGHTER]

567
00:32:01,230 --> 00:32:03,610
Now, this particular example--
the other one obviously

568
00:32:03,610 --> 00:32:04,900
was not-- but this
particular example

569
00:32:04,900 --> 00:32:06,370
was more about
trajectory learning.

570
00:32:06,370 --> 00:32:08,170
And they could
execute the trajectory

571
00:32:08,170 --> 00:32:09,510
on a fully actuated robot.

572
00:32:12,250 --> 00:32:14,320
But these are definitely
the success stories

573
00:32:14,320 --> 00:32:15,310
in imitation learning.

574
00:32:17,641 --> 00:32:19,433
AUDIENCE: He's no
terminator, for instance.

575
00:32:19,433 --> 00:32:20,077
[LAUGHTER]

576
00:32:20,077 --> 00:32:21,952
RUSS TEDRAKE: Oh, you
should see the new one.

577
00:32:21,952 --> 00:32:23,600
[LAUGHTER]

578
00:32:23,600 --> 00:32:26,280
Yeah, it doesn't have teeth.

579
00:32:26,280 --> 00:32:29,610
But it could crush
through walls.

580
00:32:29,610 --> 00:32:32,340
You know, it's this huge
hydraulic-- you've seen it,

581
00:32:32,340 --> 00:32:33,250
Rick.

582
00:32:33,250 --> 00:32:34,513
It's scary looking, right?

583
00:32:34,513 --> 00:32:36,180
And the videos they
have, they're like--

584
00:32:36,180 --> 00:32:39,680
AUDIENCE: They have one where
it's learning to kick box.

585
00:32:39,680 --> 00:32:42,180
[LAUGHTER]

586
00:32:42,180 --> 00:32:44,055
The researcher was a
kick boxer [INAUDIBLE]..

587
00:32:44,055 --> 00:32:46,555
RUSS TEDRAKE: So that's the one
you've got to watch out for.

588
00:32:46,555 --> 00:32:48,700
They're made by Sarcos,
all these big hydraulic--

589
00:32:48,700 --> 00:32:51,540
I mean they're
powerful tethered--

590
00:32:51,540 --> 00:32:56,250
serious power output robots.

591
00:32:56,250 --> 00:32:59,520
Yeah, so the same--

592
00:32:59,520 --> 00:33:02,820
back to the non-imitation,
but the learning.

593
00:33:02,820 --> 00:33:07,260
Jun Morimoto working with Kenji
Doya and with Chris Atkeson

594
00:33:07,260 --> 00:33:11,190
did things like
reinforcement learning

595
00:33:11,190 --> 00:33:12,910
to make this little
robot stand up.

596
00:33:12,910 --> 00:33:13,660
It's kind of cute.

597
00:33:13,660 --> 00:33:14,580
I have to show it.

598
00:33:18,220 --> 00:33:24,260
[LAUGHTER]

599
00:33:24,260 --> 00:33:29,450
And then after a few trials
it would successfully

600
00:33:29,450 --> 00:33:31,460
stand up, right?

601
00:33:31,460 --> 00:33:34,010
And these are really-- these
are the cue learning kind

602
00:33:34,010 --> 00:33:35,960
of algorithms I told you about.

603
00:33:38,570 --> 00:33:41,810
So there's a lot of
good work out there

604
00:33:41,810 --> 00:33:44,483
in that line of thinking.

605
00:33:44,483 --> 00:33:46,650
Interestingly, I wanted to
show, to find the videos.

606
00:33:46,650 --> 00:33:47,858
I couldn't find Jan's videos.

607
00:33:47,858 --> 00:33:51,350
But I told you
Stefan and Chris had

608
00:33:51,350 --> 00:33:54,470
started going towards imitation
learning and away from pure RL

609
00:33:54,470 --> 00:33:55,580
in some ways.

610
00:33:55,580 --> 00:33:56,878
But Stefan, I think, is back.

611
00:33:56,878 --> 00:33:57,920
I don't know about Chris.

612
00:33:57,920 --> 00:33:59,337
Chris is still on
the fence maybe.

613
00:33:59,337 --> 00:34:02,810
But Stefan and Jan
Peters are the ones

614
00:34:02,810 --> 00:34:06,260
that did this natural actor
critic algorithm which I just

615
00:34:06,260 --> 00:34:08,820
wrote the reference on
the board on Tuesday.

616
00:34:08,820 --> 00:34:12,260
And they-- now there's a new
wave of videos of a batting--

617
00:34:12,260 --> 00:34:14,360
from a few years
ago, you know it's

618
00:34:14,360 --> 00:34:16,040
playing tee ball or something.

619
00:34:16,040 --> 00:34:18,728
It's hitting a bat to
try to hit a ball off.

620
00:34:18,728 --> 00:34:19,520
And they've got a--

621
00:34:19,520 --> 00:34:22,880
Jan's got a wham arm
doing a ball and cup

622
00:34:22,880 --> 00:34:25,820
task like this, and maybe paddle
ball or these kind of tasks.

623
00:34:25,820 --> 00:34:27,853
And it's the next
generation of these.

624
00:34:27,853 --> 00:34:29,520
But it's without the
imitation learning.

625
00:34:29,520 --> 00:34:30,770
It's the natural actor critic.

626
00:34:30,770 --> 00:34:32,449
And they believe
that these algorithms

627
00:34:32,449 --> 00:34:34,699
have gotten better enough
that it's interesting again.

628
00:34:39,650 --> 00:34:43,190
OK, so that-- I mean,
that's actually--

629
00:34:43,190 --> 00:34:45,170
I told you there's
[INAUDIBLE] that

630
00:34:45,170 --> 00:34:50,030
run back and forth between
pegs to optimize their gaits.

631
00:34:50,030 --> 00:34:54,230
There's a handful of stories
from robotics of these learning

632
00:34:54,230 --> 00:34:55,670
algorithms working well.

633
00:34:55,670 --> 00:34:58,580
There's a lot of success
stories of the motion planning

634
00:34:58,580 --> 00:35:00,590
algorithms working well.

635
00:35:00,590 --> 00:35:04,280
But they're sort of still not
quite mainstream robotics,

636
00:35:04,280 --> 00:35:05,900
I'd say, the learning stuff.

637
00:35:05,900 --> 00:35:10,088
If you really wanted to design
a controller, we'd still--

638
00:35:10,088 --> 00:35:11,630
there's a handful
of people out there

639
00:35:11,630 --> 00:35:12,838
that are still working on it.

640
00:35:12,838 --> 00:35:16,850
But I obviously believe that's
one of the key ingredients

641
00:35:16,850 --> 00:35:19,410
going forward.

642
00:35:19,410 --> 00:35:22,640
So let me take a minute and
show you some of the stuff

643
00:35:22,640 --> 00:35:25,190
that we've been doing in
a little bit more detail.

644
00:35:37,195 --> 00:35:38,570
What you guys saw
yesterday still

645
00:35:38,570 --> 00:35:39,950
has my yesterday data, yeah?

646
00:35:44,830 --> 00:35:47,860
OK, so in our lab, we're--

647
00:35:47,860 --> 00:35:50,050
I told you this in
snippets throughout.

648
00:35:50,050 --> 00:35:53,650
But just to show
you really how we're

649
00:35:53,650 --> 00:35:56,710
going at this today
in the research,

650
00:35:56,710 --> 00:35:59,740
I really believe
that fluid dynamics

651
00:35:59,740 --> 00:36:03,430
is a great domain where there's
tons of good control problems

652
00:36:03,430 --> 00:36:06,070
to be solved.

653
00:36:06,070 --> 00:36:09,460
And most of the problems
that are unsolved

654
00:36:09,460 --> 00:36:11,830
are unsolved because
we don't have

655
00:36:11,830 --> 00:36:16,420
good models that are useful
for designing controllers.

656
00:36:16,420 --> 00:36:19,895
So this is, I think,
a particular domain

657
00:36:19,895 --> 00:36:21,770
where I think the machine
learning algorithms

658
00:36:21,770 --> 00:36:24,190
and the model-free
section of the course

659
00:36:24,190 --> 00:36:25,870
is really going to help out.

660
00:36:25,870 --> 00:36:27,490
You can do system
identification,

661
00:36:27,490 --> 00:36:29,800
but you can also do
reinforcement learning to take

662
00:36:29,800 --> 00:36:33,760
your best model-based controller
and improve it in a fluid,

663
00:36:33,760 --> 00:36:36,190
for instance, setting where--

664
00:36:36,190 --> 00:36:40,740
with just approximate models and
model-free control synthesis.

665
00:36:40,740 --> 00:36:41,990
There's actually other places.

666
00:36:41,990 --> 00:36:44,500
So Elena in our
group is now-- she

667
00:36:44,500 --> 00:36:46,570
says she wants to do
plasmas or something.

668
00:36:46,570 --> 00:36:48,820
So there are plenty
of other domains

669
00:36:48,820 --> 00:36:51,820
where the dynamics are still
just ridiculously poorly

670
00:36:51,820 --> 00:36:52,455
understood.

671
00:36:52,455 --> 00:36:53,830
And you can imagine
if you wanted

672
00:36:53,830 --> 00:36:58,150
to do control of some high
energy something or something

673
00:36:58,150 --> 00:37:00,010
where the models
aren't good, there's

674
00:37:00,010 --> 00:37:03,820
really not a lot of good
control work in those domains.

675
00:37:03,820 --> 00:37:05,680
You could do some
of the first stuff.

676
00:37:05,680 --> 00:37:09,770
So we've got two major fluid
projects in the lab right now.

677
00:37:09,770 --> 00:37:10,730
One is the perching.

678
00:37:10,730 --> 00:37:11,980
I showed you the video before.

679
00:37:11,980 --> 00:37:13,180
I'll tell you the story now.

680
00:37:13,180 --> 00:37:15,940
And I'll tell you a little bit
more about our robotic birds.

681
00:37:15,940 --> 00:37:21,940
So perching is a
hard fluids problem.

682
00:37:21,940 --> 00:37:24,640
Because if you go
to a high angle

683
00:37:24,640 --> 00:37:28,173
of attack with your
airfoil then the flow

684
00:37:28,173 --> 00:37:29,215
becomes very complicated.

685
00:37:29,215 --> 00:37:31,750
It becomes nonlinear
and unsteady.

686
00:37:31,750 --> 00:37:35,890
And linear control
and classical control

687
00:37:35,890 --> 00:37:38,440
works well and in these
sort of low angle of attack

688
00:37:38,440 --> 00:37:39,950
regimes on the left.

689
00:37:39,950 --> 00:37:41,900
And that's what
fighter jets use today.

690
00:37:41,900 --> 00:37:43,803
And when you go to the
high angle of attack,

691
00:37:43,803 --> 00:37:45,220
the fighter jets
aren't doing bad.

692
00:37:45,220 --> 00:37:47,560
We actually-- Rick was
just showing me today

693
00:37:47,560 --> 00:37:50,440
he found the instruction
manual for doing

694
00:37:50,440 --> 00:37:53,860
a Pugachev's Cobra, which
is this ridiculous task they

695
00:37:53,860 --> 00:37:55,150
do in airshows.

696
00:37:55,150 --> 00:37:57,714
And step one is like turn off
your flight control systems,

697
00:37:57,714 --> 00:38:00,047
turn off all the warnings or
something like that, right?

698
00:38:00,047 --> 00:38:02,680
[LAUGHTER] It's like, wow, OK.

699
00:38:02,680 --> 00:38:05,380
But then it's just a
pretty rote maneuver.

700
00:38:05,380 --> 00:38:08,050
You go through a
handful of trajectory--

701
00:38:08,050 --> 00:38:08,620
controls.

702
00:38:08,620 --> 00:38:13,000
And it does this sort of
nose up, nose back down.

703
00:38:13,000 --> 00:38:15,850
So it's hard to say that that's
a high performance control

704
00:38:15,850 --> 00:38:16,630
system just yet.

705
00:38:16,630 --> 00:38:21,340
It's just sort of an loop thing
the pilots do in air shows.

706
00:38:21,340 --> 00:38:26,140
Birds all the time are
doing beautifully complex

707
00:38:26,140 --> 00:38:28,040
dynamic maneuvers.

708
00:38:28,040 --> 00:38:30,460
My favorite now is when
they land on a perch,

709
00:38:30,460 --> 00:38:33,040
because they go up to
incredibly high angle of attack

710
00:38:33,040 --> 00:38:36,850
always in order to
actually generate

711
00:38:36,850 --> 00:38:41,327
that complicated flow behind
the wing to generate more drag.

712
00:38:41,327 --> 00:38:42,160
So when they get a--

713
00:38:42,160 --> 00:38:44,080
when you get separation
on the back of your wing,

714
00:38:44,080 --> 00:38:45,455
you get a pocket
of low pressure.

715
00:38:45,455 --> 00:38:49,060
And you get a pressure drag,
that they sort of go up,

716
00:38:49,060 --> 00:38:50,350
hit their wings out like this.

717
00:38:50,350 --> 00:38:51,725
It's like hitting
the air brakes,

718
00:38:51,725 --> 00:38:54,050
and they stop
dramatically faster.

719
00:38:54,050 --> 00:39:01,060
So there's a handful of
projects out there doing this.

720
00:39:01,060 --> 00:39:03,580
This is the Cornell
morphing plane project.

721
00:39:03,580 --> 00:39:07,660
It's-- the idea in there is that
you actually morph your body up

722
00:39:07,660 --> 00:39:08,980
to try to get a lot of drag.

723
00:39:08,980 --> 00:39:11,470
But you keep your
wings morphed back down

724
00:39:11,470 --> 00:39:13,360
so that they have
attached flow, and you can

725
00:39:13,360 --> 00:39:14,670
do your linear control on that.

726
00:39:14,670 --> 00:39:16,420
And then they move the
tail out of the way

727
00:39:16,420 --> 00:39:19,570
so that the airflow on the
tail is again attached.

728
00:39:19,570 --> 00:39:22,120
And they can do
standard linear control.

729
00:39:22,120 --> 00:39:25,390
And John Howe does these prop
hang perching trajectories,

730
00:39:25,390 --> 00:39:31,330
where he uses a lot of thrust to
stabilize his plane to go over

731
00:39:31,330 --> 00:39:35,800
and sort of do a helicopter
kind of landing on the perch.

732
00:39:35,800 --> 00:39:39,130
But these are sort
of not the approaches

733
00:39:39,130 --> 00:39:40,900
that we would
advocate in the class.

734
00:39:40,900 --> 00:39:43,450
This is-- my take on that
is that a lot of people

735
00:39:43,450 --> 00:39:45,730
are trying to perch
by making planes that

736
00:39:45,730 --> 00:39:49,120
work with old school control.

737
00:39:49,120 --> 00:39:52,180
And I'd like to
advocate using our newer

738
00:39:52,180 --> 00:39:55,630
tools to do nonlinear control
on more exciting vehicles,

739
00:39:55,630 --> 00:39:56,530
let's say.

740
00:39:56,530 --> 00:39:58,072
There's a couple of
military vehicles

741
00:39:58,072 --> 00:40:00,810
that do these kind
of things too.

742
00:40:00,810 --> 00:40:04,410
So Woody has been--
and the group

743
00:40:04,410 --> 00:40:06,390
have been thinking
about how to make

744
00:40:06,390 --> 00:40:08,100
a fair comparison
between the performance

745
00:40:08,100 --> 00:40:10,590
of a bird and a plane.

746
00:40:10,590 --> 00:40:13,170
And they point out that the
important variables here

747
00:40:13,170 --> 00:40:17,100
are the mass, the wing area,
the density of the fluid.

748
00:40:17,100 --> 00:40:21,840
If you're willing to sort of
characterize all these terms,

749
00:40:21,840 --> 00:40:24,540
then you can come up with a
dimensionless quantity which

750
00:40:24,540 --> 00:40:28,132
scores how effectively a
plane would land on a-- how

751
00:40:28,132 --> 00:40:30,090
quickly a plane, for
instance, would decelerate

752
00:40:30,090 --> 00:40:32,400
or a bird would decelerate
relative to what

753
00:40:32,400 --> 00:40:37,410
you'd expect if you were a
flat plate and peak drag.

754
00:40:37,410 --> 00:40:39,750
That's a fair way
to sort of back out

755
00:40:39,750 --> 00:40:42,462
the stopping abilities of a
small bird versus a big plane

756
00:40:42,462 --> 00:40:43,420
or something like this.

757
00:40:43,420 --> 00:40:45,060
So it gives you a
dimensionless number.

758
00:40:45,060 --> 00:40:46,830
Woody's not quite happy with us
calling it the perching number.

759
00:40:46,830 --> 00:40:49,050
But I still call it
the perching number.

760
00:40:49,050 --> 00:40:50,760
It means you're impressive if--

761
00:40:50,760 --> 00:40:53,630
if you stop well, it's
impressive to be heavy, operate

762
00:40:53,630 --> 00:40:55,890
in a low density fluid,
have a small wing area,

763
00:40:55,890 --> 00:40:57,288
or stop in a short distance.

764
00:40:57,288 --> 00:40:59,830
And now these numbers, you have
to take with a grain of salt,

765
00:40:59,830 --> 00:41:02,372
because it's hard to get these
numbers out of the literature.

766
00:41:02,372 --> 00:41:05,490
But our first crack at trying to
take these numbers from papers

767
00:41:05,490 --> 00:41:06,550
and things like that--

768
00:41:06,550 --> 00:41:09,300
a Boeing 747 not
trying to stop quickly,

769
00:41:09,300 --> 00:41:12,990
this is trying to get you
safely to your destination, is--

770
00:41:12,990 --> 00:41:14,970
if you look at as
it's decelerating,

771
00:41:14,970 --> 00:41:19,217
just before it touches down,
it gets a number of 0.08.

772
00:41:19,217 --> 00:41:21,300
So it's roughly using--
it's a very small fraction

773
00:41:21,300 --> 00:41:22,560
of its available drag.

774
00:41:22,560 --> 00:41:25,500
Because it's taking a
conservative approach.

775
00:41:25,500 --> 00:41:28,020
Now the X31 was
trying to stop fast.

776
00:41:28,020 --> 00:41:31,050
It's getting a perching number,
again, very roughly estimated,

777
00:41:31,050 --> 00:41:36,120
of about 0.15 with sort of a 24
degree angle of attack landing.

778
00:41:36,120 --> 00:41:39,180
The Cornell perching plane's
somewhere around 0.125.

779
00:41:39,180 --> 00:41:42,030
That number we sort of
believe is accurate,

780
00:41:42,030 --> 00:41:43,120
because we know the guys.

781
00:41:43,120 --> 00:41:45,120
But they pointed out that
they were more worried

782
00:41:45,120 --> 00:41:46,920
about, in their
perching trajectory,

783
00:41:46,920 --> 00:41:49,003
not hitting the ground on
the way to the building.

784
00:41:49,003 --> 00:41:51,462
So they optimized for something
other than perching number,

785
00:41:51,462 --> 00:41:52,332
of course, too.

786
00:41:52,332 --> 00:41:53,940
AUDIENCE: [INAUDIBLE]
from biology?

787
00:41:53,940 --> 00:41:54,420
RUSS TEDRAKE: What's that?

788
00:41:54,420 --> 00:41:55,140
Good.

789
00:41:55,140 --> 00:41:58,140
So the birds are
more like this, yeah?

790
00:41:58,140 --> 00:42:00,040
Now, that's using
a lot of thrust,

791
00:42:00,040 --> 00:42:03,120
which we can penalize if we know
how to use metabolic-- measure

792
00:42:03,120 --> 00:42:04,830
their metabolic cost.

793
00:42:04,830 --> 00:42:07,765
And we're trying to get
numbers of it without thrust.

794
00:42:07,765 --> 00:42:10,140
It's actually hard to get a
bird to land without flapping

795
00:42:10,140 --> 00:42:11,028
its wings, right?

796
00:42:11,028 --> 00:42:12,555
[LAUGHTER]

797
00:42:12,555 --> 00:42:15,180
But we're actually working with
biologists at the field station

798
00:42:15,180 --> 00:42:17,055
hopefully to get some
real numbers like that.

799
00:42:17,055 --> 00:42:18,900
So what we'd love
to have is sort

800
00:42:18,900 --> 00:42:23,760
of a plot of a length scale or
something over perching number

801
00:42:23,760 --> 00:42:25,115
and show that it's invariant.

802
00:42:25,115 --> 00:42:26,490
You know, the
small, small things

803
00:42:26,490 --> 00:42:28,823
that fly like planes get
a bad perching number.

804
00:42:28,823 --> 00:42:30,240
And big things
that fly like birds

805
00:42:30,240 --> 00:42:33,627
get a small-- or a high perching
number and things like that.

806
00:42:33,627 --> 00:42:34,710
And we're working on that.

807
00:42:34,710 --> 00:42:36,610
AUDIENCE: [INAUDIBLE]
birds that naturally

808
00:42:36,610 --> 00:42:38,643
land without flapping.

809
00:42:38,643 --> 00:42:40,060
RUSS TEDRAKE: Are
there any birds?

810
00:42:40,060 --> 00:42:42,810
So the case, I think, where
they land without flapping

811
00:42:42,810 --> 00:42:43,898
is when they wind hover.

812
00:42:43,898 --> 00:42:45,690
So it's not so much
about the bird as about

813
00:42:45,690 --> 00:42:47,812
if there's enough
wind over the perch,

814
00:42:47,812 --> 00:42:49,020
then they'll sometimes hover.

815
00:42:49,020 --> 00:42:51,880
And then we could try to
find that kind of data.

816
00:42:51,880 --> 00:42:53,130
And there are people that do--

817
00:42:53,130 --> 00:42:55,470
so Rick had a
picture of a parrot

818
00:42:55,470 --> 00:42:57,330
with-- a parakeet
with an oxygen mask

819
00:42:57,330 --> 00:43:00,510
on trying to do energy
metabolics on a bird.

820
00:43:00,510 --> 00:43:06,170
And, you know, everything's
possible, but--

821
00:43:06,170 --> 00:43:09,520
AUDIENCE: There is an
s in the denominator.

822
00:43:09,520 --> 00:43:13,562
That basically means like that
bigger planes [INAUDIBLE]??

823
00:43:16,295 --> 00:43:18,670
RUSS TEDRAKE: Yes, so it's
impressive if you stop quickly

824
00:43:18,670 --> 00:43:19,420
with a small wing.

825
00:43:22,850 --> 00:43:23,930
Right?

826
00:43:23,930 --> 00:43:26,330
AUDIENCE: But there is a
b zero and b f as well.

827
00:43:28,307 --> 00:43:30,140
RUSS TEDRAKE: It's all
factored out in order

828
00:43:30,140 --> 00:43:34,220
to be sort of-- so the initial
case, actually, was actually

829
00:43:34,220 --> 00:43:37,340
m over rho s x, but that doesn't
handle the case where you don't

830
00:43:37,340 --> 00:43:39,315
stop, go to zero speed.

831
00:43:39,315 --> 00:43:40,940
So you actually need
this extra factor,

832
00:43:40,940 --> 00:43:42,482
which is also
dimensionless in itself

833
00:43:42,482 --> 00:43:48,380
to handle the case where you go
between some relative speeds.

834
00:43:48,380 --> 00:43:50,210
I think it's-- yep?

835
00:43:50,210 --> 00:43:52,530
AUDIENCE: Distance,
to me, [INAUDIBLE]

836
00:43:52,530 --> 00:43:55,452
usually things that we make
are really bigger than birds

837
00:43:55,452 --> 00:43:56,330
[INAUDIBLE].

838
00:43:56,330 --> 00:43:58,330
RUSS TEDRAKE: That's why
we need to have a plot.

839
00:43:58,330 --> 00:44:00,860
And we found that we have
a-- in our very sparse data

840
00:44:00,860 --> 00:44:02,810
collection, we have
some inclination

841
00:44:02,810 --> 00:44:05,720
that there are some
small planes that

842
00:44:05,720 --> 00:44:07,880
have a worse perching
number than some big birds

843
00:44:07,880 --> 00:44:10,160
and that it's actually--
we have successfully

844
00:44:10,160 --> 00:44:12,830
taken out the dimensionality.

845
00:44:12,830 --> 00:44:15,410
But this is work that we're
still trying to finish.

846
00:44:15,410 --> 00:44:19,760
But just for fun, if you
wanted your Boeing 747

847
00:44:19,760 --> 00:44:22,820
to get a perching
number of 11.7,

848
00:44:22,820 --> 00:44:25,640
it means you'd have to
go from 450 mile an hour

849
00:44:25,640 --> 00:44:29,840
cruise speed to
zero in 20 meters.

850
00:44:29,840 --> 00:44:30,800
[LAUGHTER]

851
00:44:30,800 --> 00:44:37,450
Right, so that's
pretty good, right?

852
00:44:37,450 --> 00:44:44,750
OK, so why is it a
good control problem?

853
00:44:44,750 --> 00:44:47,180
So this is all the reasons
that we've used in class.

854
00:44:47,180 --> 00:44:49,520
But in these domains, the
fluid dynamics I think

855
00:44:49,520 --> 00:44:51,320
are just prohibitively
complicated.

856
00:44:51,320 --> 00:44:53,870
Their time varying
nonlinear, CFD simulations

857
00:44:53,870 --> 00:44:54,740
are often very slow.

858
00:44:54,740 --> 00:44:55,580
We're in a very--

859
00:44:55,580 --> 00:44:57,800
we're around 10 to
the 5 Reynolds number

860
00:44:57,800 --> 00:44:59,900
where CFD craps out.

861
00:44:59,900 --> 00:45:02,695
And there's just not a lot of
good design accessible models.

862
00:45:02,695 --> 00:45:04,070
Design accessible
means something

863
00:45:04,070 --> 00:45:07,330
I could run any of my
control synthesis tools on.

864
00:45:07,330 --> 00:45:09,230
AUDIENCE: Can the
wings of most planes

865
00:45:09,230 --> 00:45:11,420
handle the kind of
drag that would be--

866
00:45:11,420 --> 00:45:13,430
RUSS TEDRAKE: So
UAVs, I think, can.

867
00:45:13,430 --> 00:45:15,590
But I think, yeah, I
mean, no question that

868
00:45:15,590 --> 00:45:17,900
if we took-- even a
fighter jet, probably,

869
00:45:17,900 --> 00:45:18,900
would rip its wings off.

870
00:45:18,900 --> 00:45:23,360
But a 747 could not
physically stop in 20 meters

871
00:45:23,360 --> 00:45:26,150
without ripping its
wings off, right?

872
00:45:26,150 --> 00:45:29,780
So I'm not advocating
that we ride in planes

873
00:45:29,780 --> 00:45:31,880
with perching numbers of this.

874
00:45:31,880 --> 00:45:34,430
But UAVs, I think,
could do this--

875
00:45:34,430 --> 00:45:37,450
small UAVs.

876
00:45:37,450 --> 00:45:39,575
OK, so even if we did
have a perfect model,

877
00:45:39,575 --> 00:45:41,450
it's still a hard problem
for all the reasons

878
00:45:41,450 --> 00:45:42,170
we've talked about in class.

879
00:45:42,170 --> 00:45:43,587
We have limited
control authority.

880
00:45:43,587 --> 00:45:44,930
We have partial observability.

881
00:45:44,930 --> 00:45:46,430
We didn't spend a
long time on that.

882
00:45:46,430 --> 00:45:47,720
But if the dynamics of--

883
00:45:47,720 --> 00:45:49,670
the state space, the
state of the fluid

884
00:45:49,670 --> 00:45:52,670
matters when the things
are unsteady and nonlinear.

885
00:45:52,670 --> 00:45:54,860
And we don't have
sensors for that.

886
00:45:54,860 --> 00:45:56,780
The control forces
take time to develop.

887
00:45:56,780 --> 00:45:58,520
So it's under-actuated
in that sense.

888
00:45:58,520 --> 00:46:00,500
There's intermittent
control authority

889
00:46:00,500 --> 00:46:03,890
from the stalling on the wings.

890
00:46:03,890 --> 00:46:05,180
The control surfaces saturate.

891
00:46:05,180 --> 00:46:07,400
There's a lot of good reasons
why it's still a hard control

892
00:46:07,400 --> 00:46:07,900
problem.

893
00:46:07,900 --> 00:46:11,450
You have to use the kind
of tools we work with.

894
00:46:11,450 --> 00:46:14,240
So we did this
first by doing all

895
00:46:14,240 --> 00:46:15,580
these model-based approaches.

896
00:46:15,580 --> 00:46:17,330
We actually-- when we
started the project,

897
00:46:17,330 --> 00:46:19,970
I didn't expect to have a
good model of the glider.

898
00:46:19,970 --> 00:46:22,262
We thought this was immediately
a case where model-free

899
00:46:22,262 --> 00:46:23,540
was going to be the solution.

900
00:46:23,540 --> 00:46:26,090
But Rick's initial
data collection

901
00:46:26,090 --> 00:46:28,250
started coming out
beautifully clean data.

902
00:46:28,250 --> 00:46:30,055
And we tried to learn
an aerodynamic model

903
00:46:30,055 --> 00:46:31,920
and started doing
model-based control.

904
00:46:31,920 --> 00:46:33,480
So in this case,
it's actually maybe

905
00:46:33,480 --> 00:46:35,270
what Chris and Stefan
would have called

906
00:46:35,270 --> 00:46:37,103
as model-based reinforcement
learning, where

907
00:46:37,103 --> 00:46:39,140
you learn a model and
then run the best policy.

908
00:46:42,350 --> 00:46:44,540
It's in a motion capture
environment, these planes.

909
00:46:44,540 --> 00:46:47,780
We-- Rick's built a
lot of foam planes.

910
00:46:47,780 --> 00:46:50,480
Woody now has two.

911
00:46:50,480 --> 00:46:53,510
So we've got a small foam plane.

912
00:46:53,510 --> 00:46:57,260
It's got markers on here so that
the cameras offboard can sense

913
00:46:57,260 --> 00:47:01,280
the position, orientation of
the plane relative to the perch

914
00:47:01,280 --> 00:47:02,780
at about 119 Hertz.

915
00:47:02,780 --> 00:47:05,780
The only thing onboard is that
servo motor you can just barely

916
00:47:05,780 --> 00:47:07,450
see in the middle,
which is actuating

917
00:47:07,450 --> 00:47:10,870
the elevator in the tail and
a battery and a receiver.

918
00:47:10,870 --> 00:47:11,460
That's it.

919
00:47:11,460 --> 00:47:15,650
Everything else is
offboard control.

920
00:47:15,650 --> 00:47:18,170
You see that the wings are
bent up with some dihedral.

921
00:47:18,170 --> 00:47:19,850
That gives it passive
role stability,

922
00:47:19,850 --> 00:47:22,340
just the center of pressure
is above the center of mass.

923
00:47:22,340 --> 00:47:23,390
And it's got a big tail.

924
00:47:23,390 --> 00:47:24,933
So it's got sort
of yaw stability.

925
00:47:24,933 --> 00:47:25,850
It's basically a dart.

926
00:47:25,850 --> 00:47:27,600
It would never do
anything interesting out

927
00:47:27,600 --> 00:47:31,130
of the plane, which is
the way they designed it.

928
00:47:31,130 --> 00:47:33,680
But it's got perfectly
interesting and rich

929
00:47:33,680 --> 00:47:36,290
longitudinal dynamics.

930
00:47:36,290 --> 00:47:39,020
And that allows us
to simplify the model

931
00:47:39,020 --> 00:47:43,910
to be sort of a planar
model, rigid body model

932
00:47:43,910 --> 00:47:45,180
like we've been using a class.

933
00:47:45,180 --> 00:47:49,280
In fact, in my notes when I
get back to posting the most

934
00:47:49,280 --> 00:47:52,010
recent chapters, you'll
see our simple rigid body

935
00:47:52,010 --> 00:47:54,410
model of the plane.

936
00:47:54,410 --> 00:47:57,860
So Rick shot the
plane like 240 times

937
00:47:57,860 --> 00:48:00,333
into the motion capture
environment with--

938
00:48:00,333 --> 00:48:02,250
we basically flew into
the middle of the room,

939
00:48:02,250 --> 00:48:04,820
then kicked its elevator
up, and then fell down,

940
00:48:04,820 --> 00:48:06,590
or kicked its elevator
to a random place,

941
00:48:06,590 --> 00:48:08,900
fell down, and
collected a lot of data.

942
00:48:08,900 --> 00:48:12,470
We built a careful rigid
body vehicle model,

943
00:48:12,470 --> 00:48:13,973
a careful model the equator.

944
00:48:13,973 --> 00:48:15,390
We're making it
better and better.

945
00:48:15,390 --> 00:48:19,040
Woody's got the next wave of
system identification now.

946
00:48:19,040 --> 00:48:23,090
And you plot that
data over angle

947
00:48:23,090 --> 00:48:24,890
of attack, the lift
and drag coefficients,

948
00:48:24,890 --> 00:48:27,740
which are the standard
ways to do the system

949
00:48:27,740 --> 00:48:29,007
identification in aircraft.

950
00:48:29,007 --> 00:48:30,590
And the first thing
you'll notice here

951
00:48:30,590 --> 00:48:35,210
is that it's data over
very large range of angles

952
00:48:35,210 --> 00:48:36,890
of attack, which
is kind of cool,

953
00:48:36,890 --> 00:48:39,050
because you don't get
that in real flight data

954
00:48:39,050 --> 00:48:39,980
from real planes.

955
00:48:39,980 --> 00:48:42,590
Because pilots don't do that.

956
00:48:42,590 --> 00:48:47,793
It's the luxury of working with
foam planes and motion capture.

957
00:48:47,793 --> 00:48:50,210
And the other cool thing is,
at least on our initial plane

958
00:48:50,210 --> 00:48:52,340
designs, we've actually
broken it-- we built a plane

959
00:48:52,340 --> 00:48:53,923
we thought was going
to be more ideal,

960
00:48:53,923 --> 00:48:55,670
and we now have worse models.

961
00:48:55,670 --> 00:48:58,637
But at least in
these initial planes,

962
00:48:58,637 --> 00:49:00,470
we seem to have gotten
very, very good match

963
00:49:00,470 --> 00:49:02,750
to some of the
textbook flat plate

964
00:49:02,750 --> 00:49:05,180
models of quasi-steady flow.

965
00:49:05,180 --> 00:49:07,730
Now, the drag coefficient
looks a little bit messier.

966
00:49:07,730 --> 00:49:09,650
There's a-- it looks
like noise there.

967
00:49:09,650 --> 00:49:11,690
But actually, we think
that's not noise.

968
00:49:11,690 --> 00:49:14,360
We think if you watch a
time trajectory of this,

969
00:49:14,360 --> 00:49:16,970
it's actually a
periodic oscillation,

970
00:49:16,970 --> 00:49:20,090
which looks like the
vortex shedding of the--

971
00:49:20,090 --> 00:49:21,837
it has the same
characteristic frequency

972
00:49:21,837 --> 00:49:23,420
as the vortices we
see in the pictures

973
00:49:23,420 --> 00:49:26,530
I'll show you in a minute.

974
00:49:26,530 --> 00:49:29,780
OK, so we have a model where the
aerodynamics are fit from data.

975
00:49:29,780 --> 00:49:33,560
The state space is 7,
roughly, sometimes 8,

976
00:49:33,560 --> 00:49:37,220
depending on how we
model the actuator.

977
00:49:37,220 --> 00:49:38,595
But that's just sort of--

978
00:49:38,595 --> 00:49:41,095
I told you when we talked about
value iteration, I told you,

979
00:49:41,095 --> 00:49:43,460
ah, you could do
it in 6 dimensions.

980
00:49:43,460 --> 00:49:45,650
If you're crazy, you
could do it in 7.

981
00:49:45,650 --> 00:49:48,170
So Rick's crazy, right?

982
00:49:48,170 --> 00:49:50,300
And he did value
iteration on it first,

983
00:49:50,300 --> 00:49:53,060
which I think is the
right thing to start with.

984
00:49:53,060 --> 00:49:55,370
To find a cost function,
we said get to the perch

985
00:49:55,370 --> 00:49:58,160
as close as you possibly can and
designed a nonlinear feedback

986
00:49:58,160 --> 00:50:00,410
policy over the state space
that required discretizing

987
00:50:00,410 --> 00:50:04,430
the entire big state
space very coarsely

988
00:50:04,430 --> 00:50:06,830
and then doing some
optimized dynamic programming

989
00:50:06,830 --> 00:50:08,900
on the cluster and
on all these things.

990
00:50:08,900 --> 00:50:10,610
And he came back
with a policy which,

991
00:50:10,610 --> 00:50:18,125
when we play it out could nicely
do these maneuvers, the post

992
00:50:18,125 --> 00:50:21,450
stall, and we say
one out of five times

993
00:50:21,450 --> 00:50:22,640
it would catch the perch.

994
00:50:27,590 --> 00:50:29,480
Now those specifications
at the top there,

995
00:50:29,480 --> 00:50:32,420
it's entering motion capture
at 6 meters per second.

996
00:50:32,420 --> 00:50:34,190
The perch is 3 1/2 meters away.

997
00:50:34,190 --> 00:50:36,860
Those are designed to try
to be a trajectory that you

998
00:50:36,860 --> 00:50:38,840
could only acquire
the perch if you're

999
00:50:38,840 --> 00:50:40,040
willing to go post stall.

1000
00:50:40,040 --> 00:50:44,747
You could only slow down fast
enough if you're post stall.

1001
00:50:44,747 --> 00:50:45,830
So these are the pictures.

1002
00:50:45,830 --> 00:50:48,050
We actually now have a
wind tunnel downstairs.

1003
00:50:48,050 --> 00:50:50,270
Rick, can I show everybody
your wind tunnel, Rick?

1004
00:50:58,630 --> 00:51:01,330
This is our wind
tunnel downstairs.

1005
00:51:01,330 --> 00:51:03,790
AUDIENCE: [INAUDIBLE]

1006
00:51:03,790 --> 00:51:05,230
RUSS TEDRAKE: Box fans--

1007
00:51:05,230 --> 00:51:07,780
so you have to-- so
my life changed when I

1008
00:51:07,780 --> 00:51:09,560
met Jun Zhang who works at NYU.

1009
00:51:09,560 --> 00:51:12,880
He's one of the best fluid
dynamicists you'll ever meet,

1010
00:51:12,880 --> 00:51:14,470
experimental fluid dynamicist.

1011
00:51:14,470 --> 00:51:15,700
And you go to his lab.

1012
00:51:15,700 --> 00:51:19,660
And he has box fans, and
like he takes pictures

1013
00:51:19,660 --> 00:51:20,920
with his digital camera.

1014
00:51:20,920 --> 00:51:25,113
And he's got, like-- it's the
most humble lab I've ever seen.

1015
00:51:25,113 --> 00:51:26,530
And he ends up
with fluid pictures

1016
00:51:26,530 --> 00:51:28,960
that are on the cover of
Science and you name it.

1017
00:51:28,960 --> 00:51:31,788
There's a real art
form to doing things,

1018
00:51:31,788 --> 00:51:34,330
I don't know if minimally, on
the cheap, something like that.

1019
00:51:34,330 --> 00:51:38,830
But we were very much
trying to do these sort of--

1020
00:51:38,830 --> 00:51:42,160
we're trying to channel Jun
a little bit with box fans

1021
00:51:42,160 --> 00:51:44,110
and you name it.

1022
00:51:44,110 --> 00:51:47,740
But we put the-- in that sort
of chamber, we emit smoke.

1023
00:51:47,740 --> 00:51:51,250
This is with still air,
before we put the box fans on.

1024
00:51:51,250 --> 00:51:53,980
You emit smoke from the
leading edge of the plane.

1025
00:51:53,980 --> 00:51:56,620
And you can watch
the vortices pull off

1026
00:51:56,620 --> 00:51:57,760
during a perching maneuver.

1027
00:51:57,760 --> 00:51:58,870
And we count those vortices.

1028
00:51:58,870 --> 00:52:00,287
And they're pretty
close alignment

1029
00:52:00,287 --> 00:52:02,200
to what we saw in
the [INAUDIBLE] data.

1030
00:52:02,200 --> 00:52:05,970
This is what-- this is a simpler
picture with a better fluid.

1031
00:52:05,970 --> 00:52:07,720
This is actually moving
the plane by hand.

1032
00:52:07,720 --> 00:52:12,460
But it had the vortices
with a lot less smoke.

1033
00:52:12,460 --> 00:52:13,600
And we were trying to--

1034
00:52:13,600 --> 00:52:17,260
tried to capture-- we're
trying to get quality like this

1035
00:52:17,260 --> 00:52:18,910
in pictures like this now.

1036
00:52:18,910 --> 00:52:19,802
AUDIENCE: Could you
just like burn something

1037
00:52:19,802 --> 00:52:21,100
at the front of the wing?

1038
00:52:21,100 --> 00:52:22,540
RUSS TEDRAKE: It's
titanium tetrachloride.

1039
00:52:22,540 --> 00:52:24,457
It reacts with the vapor
in the air and just--

1040
00:52:24,457 --> 00:52:28,626
yeah, and you don't want to
breathe it too much, right?

1041
00:52:28,626 --> 00:52:31,450
Rick says-- and you don't
want to leave it open

1042
00:52:31,450 --> 00:52:33,610
next to your wrench, right?

1043
00:52:33,610 --> 00:52:36,080
It corroded the wrench like
in an hour or something.

1044
00:52:36,080 --> 00:52:38,772
It was pretty scary.

1045
00:52:38,772 --> 00:52:44,230
[LAUGHTER]

1046
00:52:44,230 --> 00:52:48,245
OK, so how are we doing so far
in this dimensionless analysis?

1047
00:52:48,245 --> 00:52:49,870
We've got these planes
doing this much.

1048
00:52:49,870 --> 00:52:52,960
And our glider is
getting about a 0.55

1049
00:52:52,960 --> 00:52:55,112
in our current best estimates.

1050
00:52:58,210 --> 00:53:01,420
Again, the pigeon's
still whomping us.

1051
00:53:01,420 --> 00:53:03,760
So we're working on it.

1052
00:53:03,760 --> 00:53:07,270
So nowadays we're actually
trying to do LQR trees on it.

1053
00:53:07,270 --> 00:53:10,420
So the model-based approach
we're using is this,

1054
00:53:10,420 --> 00:53:13,240
is exactly what I
presented on the LQR trees.

1055
00:53:13,240 --> 00:53:15,490
In fact, we started
thinking about LQR trees

1056
00:53:15,490 --> 00:53:19,196
because we were trying to
figure out how to do a better--

1057
00:53:19,196 --> 00:53:20,860
the value iteration
felt like it was

1058
00:53:20,860 --> 00:53:23,443
wasting all of its resolution
in parts of the state space that

1059
00:53:23,443 --> 00:53:24,640
were completely irrelevant.

1060
00:53:24,640 --> 00:53:27,370
And the LQR trees were a
way to design trajectories

1061
00:53:27,370 --> 00:53:30,880
exactly in the relevant parts
of state space and nowhere else.

1062
00:53:30,880 --> 00:53:32,890
so that's where
that idea came from.

1063
00:53:35,825 --> 00:53:36,700
It's not working yet.

1064
00:53:36,700 --> 00:53:42,550
We've got a-- our model-based
LQR stabilization,

1065
00:53:42,550 --> 00:53:45,380
Woody's best thing is hitting
the perch about half the time

1066
00:53:45,380 --> 00:53:46,210
now.

1067
00:53:46,210 --> 00:53:47,980
But still, there's
sort of this--

1068
00:53:47,980 --> 00:53:49,960
the model-based-- when
the model's a little bit

1069
00:53:49,960 --> 00:53:51,858
off the real system,
we're still trying

1070
00:53:51,858 --> 00:53:54,400
to figure out how to handle the
differences between the model

1071
00:53:54,400 --> 00:53:57,438
and the real system.

1072
00:53:57,438 --> 00:54:02,410
AUDIENCE: [INAUDIBLE] perching
a real perch [INAUDIBLE]??

1073
00:54:02,410 --> 00:54:03,880
RUSS TEDRAKE: In
the simulator, we

1074
00:54:03,880 --> 00:54:05,980
can hit it from some
small initial conditions.

1075
00:54:05,980 --> 00:54:06,980
It depends on the model.

1076
00:54:06,980 --> 00:54:08,983
In Ricks' flat plate
model, it hits it

1077
00:54:08,983 --> 00:54:10,900
from a beautiful number
of initial conditions.

1078
00:54:10,900 --> 00:54:14,020
In sort of the more careful
system identification,

1079
00:54:14,020 --> 00:54:16,420
the problem is harder, because
the second derivatives are

1080
00:54:16,420 --> 00:54:18,430
larger, it seems.

1081
00:54:18,430 --> 00:54:19,580
So we're hitting it--

1082
00:54:19,580 --> 00:54:21,970
I mean, it still hits it
from the nominal trajectory

1083
00:54:21,970 --> 00:54:25,300
and hits it from a smaller
range of initial conditions.

1084
00:54:25,300 --> 00:54:26,810
But what we can't
do, for instance,

1085
00:54:26,810 --> 00:54:29,710
is take a simulated
trajectory and put it

1086
00:54:29,710 --> 00:54:34,030
into our model, which
is not quite a feasible

1087
00:54:34,030 --> 00:54:35,240
trajectory of the model.

1088
00:54:35,240 --> 00:54:38,920
It's close, but we
can't stabilize that.

1089
00:54:38,920 --> 00:54:40,750
We've got a big dynamical
system with lots

1090
00:54:40,750 --> 00:54:41,750
of interesting dynamics.

1091
00:54:41,750 --> 00:54:42,812
We've got one actuator.

1092
00:54:42,812 --> 00:54:44,770
And it's just-- it's hard
to make it do things.

1093
00:54:44,770 --> 00:54:46,348
So we're still
trying to figure out

1094
00:54:46,348 --> 00:54:47,890
if LQR trees are
the best way for it.

1095
00:54:47,890 --> 00:54:48,917
We'll see.

1096
00:54:48,917 --> 00:54:50,500
And Phillip's trying
to push LQR trees

1097
00:54:50,500 --> 00:54:52,390
on the compass [INAUDIBLE].

1098
00:54:52,390 --> 00:54:55,700
And Michael's trying
to do it on the dog.

1099
00:54:55,700 --> 00:55:00,490
OK, so now we're trying to do
it with flapping wings too.

1100
00:55:00,490 --> 00:55:03,920
So Rick's thesis is going to be
perching with flapping wings.

1101
00:55:03,920 --> 00:55:08,974
This is his simulation, which
I think looks like a Dragon.

1102
00:55:08,974 --> 00:55:11,513
[LAUGHTER]

1103
00:55:11,513 --> 00:55:13,930
So were those actually the
initial parameters of our bird,

1104
00:55:13,930 --> 00:55:15,627
or did you pick
them arbitrarily?

1105
00:55:15,627 --> 00:55:18,195
AUDIENCE: They're actually
closer to what the glider would

1106
00:55:18,195 --> 00:55:19,370
do if it was flapping.

1107
00:55:19,370 --> 00:55:20,245
RUSS TEDRAKE: Really?

1108
00:55:20,245 --> 00:55:21,050
Yeah.

1109
00:55:21,050 --> 00:55:21,550
So--

1110
00:55:21,550 --> 00:55:23,140
AUDIENCE: [INAUDIBLE]

1111
00:55:23,140 --> 00:55:24,297
RUSS TEDRAKE: The time--

1112
00:55:24,297 --> 00:55:25,630
oh, because it's a whole second.

1113
00:55:25,630 --> 00:55:29,497
AUDIENCE: A 1-second trajectory
played over [INAUDIBLE]..

1114
00:55:29,497 --> 00:55:31,330
AUDIENCE: Oh, so it's
flapping [INAUDIBLE]..

1115
00:55:31,330 --> 00:55:32,730
RUSS TEDRAKE: Yeah.

1116
00:55:32,730 --> 00:55:35,370
AUDIENCE: It has the same
flapping frequency as the bird.

1117
00:55:35,370 --> 00:55:39,320
RUSS TEDRAKE: As the
ornithopter, yeah.

1118
00:55:39,320 --> 00:55:42,960
So this previous plot that
I went past quickly is--

1119
00:55:42,960 --> 00:55:45,000
basically says that
if you look at the--

1120
00:55:45,000 --> 00:55:48,390
if you do an LQR stabilization
of a perching trajectory, given

1121
00:55:48,390 --> 00:55:53,970
our original model, and you
look at the cost to go matrix,

1122
00:55:53,970 --> 00:55:55,470
and you look at the single--

1123
00:55:55,470 --> 00:55:57,345
I'm going to represent
that cost to go matrix

1124
00:55:57,345 --> 00:56:00,300
is the single costliest
direction of the s matrix going

1125
00:56:00,300 --> 00:56:03,600
backwards, yeah?

1126
00:56:03,600 --> 00:56:05,672
That's a quantifier
of how much error

1127
00:56:05,672 --> 00:56:08,130
you should expect to get if
you were to get a gust of wind,

1128
00:56:08,130 --> 00:56:11,970
let's say, in the worst possible
direction during your perch.

1129
00:56:11,970 --> 00:56:14,820
And it's a function of
time, because the s evolves

1130
00:56:14,820 --> 00:56:15,760
backwards.

1131
00:56:15,760 --> 00:56:18,010
So if you get a gust in the
worst possible direction

1132
00:56:18,010 --> 00:56:21,270
at your initial condition
in the model stabilized,

1133
00:56:21,270 --> 00:56:23,280
it can reject that nicely.

1134
00:56:23,280 --> 00:56:28,447
But in the last tenths of
a second, a gust of wind

1135
00:56:28,447 --> 00:56:30,030
is going to be very
hard to stabilize,

1136
00:56:30,030 --> 00:56:32,363
because you have limited
control authority, particularly

1137
00:56:32,363 --> 00:56:34,420
because your airspeed
is very small.

1138
00:56:34,420 --> 00:56:36,510
And therefore your
aerodynamic control authority

1139
00:56:36,510 --> 00:56:37,680
is very small.

1140
00:56:37,680 --> 00:56:40,110
And even if you put in
thrust or thrust vectoring,

1141
00:56:40,110 --> 00:56:42,330
it's still a big problem.

1142
00:56:42,330 --> 00:56:45,485
If your airspeed goes to 0, your
control authority goes to 0.

1143
00:56:45,485 --> 00:56:47,610
And that's one of the
reasons we think the flapping

1144
00:56:47,610 --> 00:56:48,810
is such a good idea.

1145
00:56:48,810 --> 00:56:51,143
That's one of the reasons we
think that the birds always

1146
00:56:51,143 --> 00:56:53,862
flap when they're
landing, because they're

1147
00:56:53,862 --> 00:56:56,070
trying to maintain air speed
on the control surfaces.

1148
00:56:56,070 --> 00:56:59,460
It's actually also a very
good way to decelerate.

1149
00:56:59,460 --> 00:57:01,680
So this is the big bird
upstairs in the lab.

1150
00:57:01,680 --> 00:57:04,060
You're welcome to come
see it any time, Zach,

1151
00:57:04,060 --> 00:57:08,067
who you remember coming in with
some robots at the beginning,

1152
00:57:08,067 --> 00:57:09,150
he's the one who built it.

1153
00:57:09,150 --> 00:57:12,018
He's an amazing designer.

1154
00:57:12,018 --> 00:57:12,810
He's also a welder.

1155
00:57:12,810 --> 00:57:15,650
This is a titanium welded
gearbox in the front.

1156
00:57:15,650 --> 00:57:18,523
It's a 300 watt outrunner motor.

1157
00:57:18,523 --> 00:57:20,690
He's designed in all the
failure points of the bird.

1158
00:57:20,690 --> 00:57:23,160
So when it inevitably crashes,
it's a breakaway beak.

1159
00:57:23,160 --> 00:57:26,880
The wing spars have
breakaway parts.

1160
00:57:26,880 --> 00:57:29,280
So you bring bucket
of parts out.

1161
00:57:29,280 --> 00:57:30,300
And this thing is--

1162
00:57:30,300 --> 00:57:33,840
it flew for the first
time last summer.

1163
00:57:33,840 --> 00:57:35,460
That's your TA
throwing it for you.

1164
00:57:39,250 --> 00:57:39,837
And it worked.

1165
00:57:39,837 --> 00:57:40,420
And it worked.

1166
00:57:40,420 --> 00:57:41,830
And then, oh my God, it's
going to hit the building!

1167
00:57:41,830 --> 00:57:42,955
So we turned it off, right?

1168
00:57:45,983 --> 00:57:47,650
But that was our first
successful flight

1169
00:57:47,650 --> 00:57:49,540
with an autonomous
ornithopter that

1170
00:57:49,540 --> 00:57:50,980
was stabilized just in pitch.

1171
00:57:53,900 --> 00:57:58,460
All right, so I'll
tell you quickly just

1172
00:57:58,460 --> 00:58:00,830
why we care about birds here.

1173
00:58:00,830 --> 00:58:02,610
And that'll be a
fun note to end on.

1174
00:58:02,610 --> 00:58:06,983
So basically, we care--

1175
00:58:06,983 --> 00:58:08,900
I care about birds not
so much about flapping,

1176
00:58:08,900 --> 00:58:10,640
but just about doing
all the great things

1177
00:58:10,640 --> 00:58:13,820
that birds can do
because I think

1178
00:58:13,820 --> 00:58:14,930
birds do fantastic things.

1179
00:58:14,930 --> 00:58:16,930
But you have to be a
little careful saying that.

1180
00:58:16,930 --> 00:58:20,390
Because if you look at speed
or efficiency sort of in still

1181
00:58:20,390 --> 00:58:23,000
air steady level
flight, then a propeller

1182
00:58:23,000 --> 00:58:24,940
is very, very efficient.

1183
00:58:24,940 --> 00:58:27,998
An airplane wing is
very highly optimized.

1184
00:58:27,998 --> 00:58:29,540
So you have to be
very careful trying

1185
00:58:29,540 --> 00:58:32,240
to say that birds are
going to be more efficient

1186
00:58:32,240 --> 00:58:35,780
or something than a plane
unless you go to the more

1187
00:58:35,780 --> 00:58:38,060
interesting flight regimes.

1188
00:58:38,060 --> 00:58:40,430
So birds are actually
incredibly efficient-- this

1189
00:58:40,430 --> 00:58:41,420
is in the notes.

1190
00:58:41,420 --> 00:58:43,730
An albatross can fly
for hours or days

1191
00:58:43,730 --> 00:58:47,600
without flapping
its wings just by--

1192
00:58:47,600 --> 00:58:50,300
even upwind-- just by exploiting
gradients in the sheer layer,

1193
00:58:50,300 --> 00:58:50,870
right?

1194
00:58:50,870 --> 00:58:52,592
So when the wind is
blowing, then there's

1195
00:58:52,592 --> 00:58:54,050
a lot more beautiful
things you can

1196
00:58:54,050 --> 00:58:56,120
do than a plane isn't doing.

1197
00:58:56,120 --> 00:58:58,700
So the cost of
transport of this guy,

1198
00:58:58,700 --> 00:59:02,330
the dimensionless
quantity, it's 0.1,

1199
00:59:02,330 --> 00:59:06,840
which is about the same as a
747 flying across the Atlantic,

1200
00:59:06,840 --> 00:59:07,820
which is cool.

1201
00:59:07,820 --> 00:59:10,370
But what's cooler is the fact
that the cost of transport

1202
00:59:10,370 --> 00:59:12,770
for this thing seems
to be the same when

1203
00:59:12,770 --> 00:59:16,072
it's sitting on the beach
versus flying across the ocean.

1204
00:59:16,072 --> 00:59:17,780
So it's basically
flying across the ocean

1205
00:59:17,780 --> 00:59:19,790
with almost no control energy.

1206
00:59:19,790 --> 00:59:22,220
It just locks its wings
in, uses the wind power,

1207
00:59:22,220 --> 00:59:23,570
and off it goes.

1208
00:59:23,570 --> 00:59:27,590
Butterflies go thousands of
kilometers carried by the wind.

1209
00:59:27,590 --> 00:59:29,870
Falcons can dive really fast.

1210
00:59:29,870 --> 00:59:32,180
The speed isn't the
impressive thing,

1211
00:59:32,180 --> 00:59:35,120
but the fact that they can
do super agile maneuvers

1212
00:59:35,120 --> 00:59:37,670
during these incredible dives,
like catching a sparrow out

1213
00:59:37,670 --> 00:59:41,810
of the air while it's
diving is really impressive.

1214
00:59:41,810 --> 00:59:44,240
Bats can catch prey
on their wings.

1215
00:59:44,240 --> 00:59:48,290
They've been studying down
at Brown, Kenny Breuer's lab,

1216
00:59:48,290 --> 00:59:49,730
they talk about
how these bats can

1217
00:59:49,730 --> 00:59:51,813
fly through thick rainforest
with 1,000 other bats

1218
00:59:51,813 --> 00:59:54,230
on their back, through
caves with stalactites

1219
00:59:54,230 --> 00:59:54,920
and stalagmites.

1220
00:59:54,920 --> 00:59:57,050
And they're doing all
these crazy things.

1221
00:59:57,050 --> 00:59:58,940
They've got a video of
this bat that can go--

1222
00:59:58,940 --> 01:00:00,710
he's going full speed this way.

1223
01:00:00,710 --> 01:00:01,877
And in five laps, or about--

1224
01:00:01,877 --> 01:00:04,418
actually, he said that he does
most of the turn in two flaps.

1225
01:00:04,418 --> 01:00:06,230
And they say in five
it's completely done.

1226
01:00:06,230 --> 01:00:08,990
In basically two flaps, he's
able to go full speed again

1227
01:00:08,990 --> 01:00:10,220
the other way.

1228
01:00:10,220 --> 01:00:13,910
And the whole maneuver takes
just under half a wingspan--

1229
01:00:13,910 --> 01:00:15,890
full speed this way,
full speed the other way

1230
01:00:15,890 --> 01:00:20,600
in half a wingspan, wow.

1231
01:00:20,600 --> 01:00:23,600
And the story goes,
and our mission

1232
01:00:23,600 --> 01:00:26,570
here in my robot
locomotion group upstairs

1233
01:00:26,570 --> 01:00:30,350
is to try to make machines
that can do this kind of stuff,

1234
01:00:30,350 --> 01:00:33,200
birds that are far surpassing
the performance of our best

1235
01:00:33,200 --> 01:00:35,870
engineered systems in a
lot of the performance

1236
01:00:35,870 --> 01:00:38,000
and efficiency, acceleration,
maneuverability,

1237
01:00:38,000 --> 01:00:39,980
but not in steady level
flight in still air,

1238
01:00:39,980 --> 01:00:42,830
but in the interesting
fluid cases.

1239
01:00:42,830 --> 01:00:45,320
And the trick is that
they're exploiting

1240
01:00:45,320 --> 01:00:46,362
unsteady aerodynamics.

1241
01:00:46,362 --> 01:00:48,320
They're doing control in
places where we're not

1242
01:00:48,320 --> 01:00:50,030
doing good control yet.

1243
01:00:50,030 --> 01:00:53,570
And it's a lot like, for
the robotics community,

1244
01:00:53,570 --> 01:00:56,540
I like to try to tell them
it's a manipulation problem.

1245
01:00:56,540 --> 01:00:59,282
If you're manipulating a
piece of chalk or something,

1246
01:00:59,282 --> 01:01:00,240
that's relatively easy.

1247
01:01:00,240 --> 01:01:01,157
You can see the chalk.

1248
01:01:01,157 --> 01:01:03,320
You know what your fingers
are supposed to do.

1249
01:01:03,320 --> 01:01:05,362
This is manipulating
vortices that you can't see.

1250
01:01:05,362 --> 01:01:06,680
You don't know what's coming.

1251
01:01:06,680 --> 01:01:09,305
It's a much nicer manipulation,
maybe the ultimate manipulation

1252
01:01:09,305 --> 01:01:09,968
problem.

1253
01:01:09,968 --> 01:01:11,510
And I like to say
that once you start

1254
01:01:11,510 --> 01:01:13,400
thinking of it as
manipulating vortices

1255
01:01:13,400 --> 01:01:15,260
and doing these sort
of clever things,

1256
01:01:15,260 --> 01:01:18,590
then suddenly you can see
why fixed wings aren't

1257
01:01:18,590 --> 01:01:19,470
as exciting anymore.

1258
01:01:19,470 --> 01:01:20,887
It's almost like
trying to pick up

1259
01:01:20,887 --> 01:01:24,882
a coffee cup with flippers on
or something like this, right?

1260
01:01:24,882 --> 01:01:26,840
So I don't actually care
about flapping per se.

1261
01:01:26,840 --> 01:01:28,160
And sometimes it matters.

1262
01:01:28,160 --> 01:01:30,380
But I care about having a
really delicate interaction

1263
01:01:30,380 --> 01:01:31,010
with the fluid.

1264
01:01:31,010 --> 01:01:33,900
And we're trying to
figure out how to do that.

1265
01:01:33,900 --> 01:01:36,862
So here's the-- did I show
this in the beginning?

1266
01:01:36,862 --> 01:01:38,570
Yeah, good, so you've
seen the dead fish.

1267
01:01:38,570 --> 01:01:41,510
So that's another example
of the efficient swimming.

1268
01:01:41,510 --> 01:01:47,900
And it's the story of our
pursuit of these controllers.

1269
01:01:47,900 --> 01:01:50,408
Awesome, OK, so I'm done.

1270
01:01:50,408 --> 01:01:51,200
Now it's your turn.

1271
01:01:51,200 --> 01:01:55,250
And we'll be-- let me
know if you have any more

1272
01:01:55,250 --> 01:01:57,860
questions about the projects.

1273
01:01:57,860 --> 01:01:59,722
We're going to pass
out evaluations

1274
01:01:59,722 --> 01:02:00,680
next week too through--

1275
01:02:00,680 --> 01:02:03,500
we have an online system
in EECS for evaluations.

1276
01:02:03,500 --> 01:02:05,570
I hope you give us
all the feedback.

1277
01:02:05,570 --> 01:02:07,440
And I hope you
enjoyed everything.

1278
01:02:07,440 --> 01:02:10,270
And I'll see you next week.