WEBVTT

00:00:01.640 --> 00:00:04.040
The following content is
provided under a Creative

00:00:04.040 --> 00:00:05.580
Commons license.

00:00:05.580 --> 00:00:07.880
Your support will help
MIT OpenCourseWare

00:00:07.880 --> 00:00:12.270
continue to offer high quality
educational resources for free.

00:00:12.270 --> 00:00:14.870
To make a donation or
view additional materials

00:00:14.870 --> 00:00:18.830
from hundreds of MIT courses,
visit MIT OpenCourseWare

00:00:18.830 --> 00:00:20.000
at ocw.mit.edu.

00:00:22.525 --> 00:00:24.150
RUSS TEDRAKE: I've
been getting to play

00:00:24.150 --> 00:00:26.040
with this robot for
a few years now--

00:00:26.040 --> 00:00:29.490
three years of my life
basically devoted to that robot.

00:00:29.490 --> 00:00:34.500
It was one of the most exciting,
technically challenging,

00:00:34.500 --> 00:00:39.390
exhausting, stressful, but
ultimately fulfilling things

00:00:39.390 --> 00:00:41.020
I've ever done.

00:00:41.020 --> 00:00:44.520
We got to basically take this
robot, make it drive a car,

00:00:44.520 --> 00:00:45.900
get out of the car--

00:00:45.900 --> 00:00:51.135
that was tough-- open the door,
turn valves, pick up a drill,

00:00:51.135 --> 00:00:53.050
cut a hole out of the wall.

00:00:53.050 --> 00:00:54.660
Notice there's no
safety hardness.

00:00:54.660 --> 00:00:56.730
It's battery autonomous.

00:00:56.730 --> 00:00:59.910
It has a walk over
some rough terrain,

00:00:59.910 --> 00:01:02.010
climbed some stairs at the end.

00:01:02.010 --> 00:01:06.090
It had to do this in
front of an audience.

00:01:06.090 --> 00:01:07.920
Basically, we got two tries.

00:01:07.920 --> 00:01:11.640
And if your robot
breaks, it breaks, right?

00:01:11.640 --> 00:01:14.820
And there was a $2
million prize at the end.

00:01:14.820 --> 00:01:17.340
We wanted to do it not
for the $2 million prize,

00:01:17.340 --> 00:01:18.750
but for the technical challenge.

00:01:18.750 --> 00:01:22.710
And myself and a group
of students, just

00:01:22.710 --> 00:01:24.872
like I said, absolutely
devoted our lives to this.

00:01:24.872 --> 00:01:26.580
We spent all of our
waking hours on this.

00:01:26.580 --> 00:01:29.670
We worked incredibly,
incredibly hard.

00:01:29.670 --> 00:01:32.350
So just to give you a
little bit of context,

00:01:32.350 --> 00:01:36.180
DARPA, our national
defense funding agency,

00:01:36.180 --> 00:01:37.599
has gotten excited
about the idea

00:01:37.599 --> 00:01:39.390
of these grant challenges,
which get people

00:01:39.390 --> 00:01:41.030
to work really, really hard.

00:01:41.030 --> 00:01:43.940
The self-driving cars
were the first one.

00:01:43.940 --> 00:01:46.500
MIT had a very successful
team in the Urban Challenge

00:01:46.500 --> 00:01:48.240
led by John.

00:01:48.240 --> 00:01:52.590
And then it's unquestionably
had transition impact

00:01:52.590 --> 00:01:56.070
into the world via Google,
Uber, Apple, and John

00:01:56.070 --> 00:01:58.260
will tell you all about it.

00:01:58.260 --> 00:02:00.780
I think in 2012, DARPA was
scratching their heads, saying,

00:02:00.780 --> 00:02:02.196
people haven't
worked hard enough.

00:02:02.196 --> 00:02:05.100
And what's the new
challenge going to be?

00:02:05.100 --> 00:02:07.080
And right around
that time, there

00:02:07.080 --> 00:02:11.430
was a disaster that maybe
helped focus their attention

00:02:11.430 --> 00:02:13.390
towards disaster response.

00:02:13.390 --> 00:02:15.750
So ultimately, it
was October 2012

00:02:15.750 --> 00:02:17.940
that everything started
with this kickoff

00:02:17.940 --> 00:02:20.610
for the DARPA
Robotics Challenge.

00:02:20.610 --> 00:02:24.690
The official challenge was
cast in the light of disaster

00:02:24.690 --> 00:02:30.030
response using the scenario
of the nuclear disaster

00:02:30.030 --> 00:02:31.680
as a backdrop.

00:02:31.680 --> 00:02:33.960
But I think really
their goal was

00:02:33.960 --> 00:02:36.150
to evaluate and advance
the state of the art

00:02:36.150 --> 00:02:38.010
in mobile manipulation.

00:02:38.010 --> 00:02:40.050
So if I'm the funding
agency, what I think

00:02:40.050 --> 00:02:42.780
is that you see
hardware coming out

00:02:42.780 --> 00:02:44.680
of industry that is fantastic.

00:02:44.680 --> 00:02:48.090
So Boston Dynamics was
building these walking robots

00:02:48.090 --> 00:02:48.866
and the like.

00:02:48.866 --> 00:02:50.490
This one is the one
we've been playing,

00:02:50.490 --> 00:02:54.226
Atlas, built by Boston
Dynamics, which is now Google.

00:02:54.226 --> 00:02:55.510
AUDIENCE: Alphabet.

00:02:55.510 --> 00:02:57.500
RUSS TEDRAKE: Alphabet, yeah.

00:02:57.500 --> 00:02:59.789
And then I think from
the research labs,

00:02:59.789 --> 00:03:01.830
we've been seeing really
sophisticated algorithms

00:03:01.830 --> 00:03:04.860
coming out but on
relatively modest hardware.

00:03:04.860 --> 00:03:07.170
And I think it was time
for a mash up, right?

00:03:07.170 --> 00:03:10.560
So they were very
interesting in the way

00:03:10.560 --> 00:03:11.760
they set up the competition.

00:03:11.760 --> 00:03:15.454
It wasn't about making it a
completely autonomous robot.

00:03:15.454 --> 00:03:16.620
There was there was a twist.

00:03:16.620 --> 00:03:18.600
You could have a human
operator, but they

00:03:18.600 --> 00:03:20.122
wanted to encourage autonomy.

00:03:20.122 --> 00:03:22.080
So what they did is they
had a degraded network

00:03:22.080 --> 00:03:25.409
link between the human and
the robot and some reward

00:03:25.409 --> 00:03:27.450
for going a little bit
faster than the other guy.

00:03:27.450 --> 00:03:29.659
So the idea would be that
if you had to stop and work

00:03:29.659 --> 00:03:31.408
over the degraded
network link and control

00:03:31.408 --> 00:03:33.120
every joint of your
robot, then you're

00:03:33.120 --> 00:03:35.036
going to be slower than
the guy whose robot is

00:03:35.036 --> 00:03:37.260
making the decisions by itself.

00:03:37.260 --> 00:03:39.240
That didn't play out
as much as we expected,

00:03:39.240 --> 00:03:41.040
but that was the setup.

00:03:41.040 --> 00:03:42.930
That set up a
spectrum where people

00:03:42.930 --> 00:03:45.720
could do full
teleoperation, meaning

00:03:45.720 --> 00:03:48.660
joystick control of each of
the joints if they wanted to.

00:03:48.660 --> 00:03:51.360
And maybe the goal is to
have complete autonomy,

00:03:51.360 --> 00:03:54.000
and you can pick your
place on the spectrum.

00:03:54.000 --> 00:03:57.660
So MIT, possibly to a fault,
aimed for the full autonomy

00:03:57.660 --> 00:03:58.590
side.

00:03:58.590 --> 00:04:03.480
The idea was, let's just get
a few clicks of information

00:04:03.480 --> 00:04:04.630
from the human.

00:04:04.630 --> 00:04:06.876
Let the human solve the
really, really hard problems

00:04:06.876 --> 00:04:08.250
that he could
solve efficiently--

00:04:08.250 --> 00:04:09.090
object recognition.

00:04:09.090 --> 00:04:11.260
Scene understanding-- we
don't have to do that,

00:04:11.260 --> 00:04:13.860
but a few clicks from the
human can communicate that.

00:04:13.860 --> 00:04:15.870
But let the robot do
all of the dynamics

00:04:15.870 --> 00:04:18.810
and control and
planning side of things.

00:04:18.810 --> 00:04:21.600
So those few clicks should see
nearly autonomous algorithms

00:04:21.600 --> 00:04:24.410
for perception,
planning, and control.

00:04:24.410 --> 00:04:24.990
OK.

00:04:24.990 --> 00:04:30.322
So technically, I don't intend
to go into too many details,

00:04:30.322 --> 00:04:32.530
but I would love to answer
questions if you guys ask.

00:04:32.530 --> 00:04:35.520
And we can talk as much
as we want about it.

00:04:35.520 --> 00:04:38.950
But the overarching
theme to our approach

00:04:38.950 --> 00:04:41.250
when we're controlling,
perceiving, everything

00:04:41.250 --> 00:04:44.890
is to formulate everything
as an optimization problem.

00:04:44.890 --> 00:04:48.410
So even the simplest
example in robotics

00:04:48.410 --> 00:04:49.980
is the inverse
kinematics problem

00:04:49.980 --> 00:04:51.570
where you're just
trying to decide

00:04:51.570 --> 00:04:53.760
if I want to put my hand
in some particular place.

00:04:53.760 --> 00:04:56.555
I have to figure out if I have
a goal in the world coordinates.

00:04:56.555 --> 00:04:58.805
I have to figure out what
the joint coordinates should

00:04:58.805 --> 00:05:00.930
be to make that happen.

00:05:00.930 --> 00:05:03.660
So we have joint positions
in some vector q,

00:05:03.660 --> 00:05:06.270
and we just say, I'd like
to be as close as possible.

00:05:06.270 --> 00:05:09.827
I have some comfortable
position for my robot.

00:05:09.827 --> 00:05:11.910
We formulate the problem
as an optimization-- say,

00:05:11.910 --> 00:05:14.440
I'd like to be as close
to comfortable as possible

00:05:14.440 --> 00:05:17.250
in some simple cost function.

00:05:17.250 --> 00:05:19.440
And then I'm going to start
putting in constraints,

00:05:19.440 --> 00:05:23.100
like my hand is in the
desired configuration.

00:05:23.100 --> 00:05:25.190
But we have very
advanced constraints.

00:05:25.190 --> 00:05:26.899
So especially for the
balancing humanoid,

00:05:26.899 --> 00:05:28.606
we can say, for
instance, that the center

00:05:28.606 --> 00:05:30.430
mass has to be inside
the support polygon.

00:05:30.430 --> 00:05:32.430
We can say, we're about
to manipulate something.

00:05:32.430 --> 00:05:34.320
So I'd like the thing
I'm going to manipulate

00:05:34.320 --> 00:05:38.850
to be in the cone of visibility
of my vision sensors.

00:05:38.850 --> 00:05:40.220
I'd like my hand to approach.

00:05:40.220 --> 00:05:42.470
It doesn't matter where it
approaches along the table,

00:05:42.470 --> 00:05:45.570
maybe, but the palm should
be orthogonal to the table

00:05:45.570 --> 00:05:46.860
and should approach like this.

00:05:46.860 --> 00:05:50.521
And we put in more
and more sophisticated

00:05:50.521 --> 00:05:52.020
collision avoidance
type constraints

00:05:52.020 --> 00:05:54.810
and everything like this, and
the optimization framework

00:05:54.810 --> 00:05:59.631
as is general and can accept
those type of constraints.

00:05:59.631 --> 00:06:01.630
And then we can solve
them extremely efficiently

00:06:01.630 --> 00:06:04.592
with highly
optimized algorithms.

00:06:04.592 --> 00:06:06.300
So for instance, that
helped us with what

00:06:06.300 --> 00:06:08.500
I like to call the big
robot little car problem.

00:06:08.500 --> 00:06:10.100
So we have a very big robot.

00:06:10.100 --> 00:06:13.440
It's a 400 pound, six
foot something machine.

00:06:13.440 --> 00:06:15.810
And they asked us to
drive a very little car

00:06:15.810 --> 00:06:17.820
to the point where the
robot physically does not

00:06:17.820 --> 00:06:19.611
fit behind the steering
wheel-- impossible.

00:06:19.611 --> 00:06:21.540
It just doesn't kinematically.

00:06:21.540 --> 00:06:24.710
Torso's too big, steering
wheel's right there, no chance.

00:06:24.710 --> 00:06:26.760
So you have to drive
from the passenger seat.

00:06:26.760 --> 00:06:28.570
You have to put your
foot over the console.

00:06:28.570 --> 00:06:30.810
You have to drive like this,
and then our only option

00:06:30.810 --> 00:06:33.000
was to get out of
the passenger side.

00:06:33.000 --> 00:06:35.490
So that was a hard
problem kinematically,

00:06:35.490 --> 00:06:38.280
but we have this rich
library of optimizations.

00:06:38.280 --> 00:06:39.220
We can drag it around.

00:06:39.220 --> 00:06:41.910
We can explore different
kinematic configurations

00:06:41.910 --> 00:06:43.590
of the robot.

00:06:43.590 --> 00:06:46.080
But we also use the same
language of optimization

00:06:46.080 --> 00:06:49.260
and constraints, and then we
put in the dynamics of the robot

00:06:49.260 --> 00:06:50.580
as another constraint.

00:06:50.580 --> 00:06:53.430
And we can start doing
efficient dynamic motion

00:06:53.430 --> 00:06:55.060
planning with the same tools.

00:06:55.060 --> 00:06:57.360
So for instance, if we wanted
Atlas to suddenly start

00:06:57.360 --> 00:07:01.800
jumping off cinder blocks or
running, we did a lot of work

00:07:01.800 --> 00:07:03.990
in that regard to
make our optimization

00:07:03.990 --> 00:07:06.450
algorithms efficient
enough to scale

00:07:06.450 --> 00:07:08.100
to very complex
motions that could

00:07:08.100 --> 00:07:11.370
be planned on the fly
at interactive rates.

00:07:11.370 --> 00:07:16.200
So one of the things you
might be familiar with--

00:07:16.200 --> 00:07:19.170
Honda ASIMO is one of
the famous robots that

00:07:19.170 --> 00:07:23.610
walks around like this, and
it's a beautiful machine.

00:07:23.610 --> 00:07:27.240
They are extremely good at
real time planning using

00:07:27.240 --> 00:07:29.682
limiting assumptions
of keeping your center

00:07:29.682 --> 00:07:31.640
mass at a constant height
and things like this.

00:07:31.640 --> 00:07:33.619
And one of the
questions we asked is,

00:07:33.619 --> 00:07:35.160
can we take some of
the insights that

00:07:35.160 --> 00:07:37.080
have worked so well
on those robots

00:07:37.080 --> 00:07:40.530
and generalize them to
more general dynamic tasks?

00:07:40.530 --> 00:07:44.100
And one of the big ideas I want
to try to communicate quickly

00:07:44.100 --> 00:07:48.779
is that even though our robot
is extremely complicated,

00:07:48.779 --> 00:07:50.820
there's sort of a low
dimensional problem sitting

00:07:50.820 --> 00:07:52.590
inside the big high
dimensional problem.

00:07:52.590 --> 00:07:55.320
So if I start worrying about
every joint angle in my hand

00:07:55.320 --> 00:07:57.720
while I'm thinking about
walking, I'm dead, right?

00:07:57.720 --> 00:08:00.120
So actually, when you're
thinking about walking, even

00:08:00.120 --> 00:08:02.070
doing gymnastics or
something like this,

00:08:02.070 --> 00:08:03.870
I think the fundamental
representation

00:08:03.870 --> 00:08:07.020
is the dynamics of your
center of mass, your angular

00:08:07.020 --> 00:08:10.440
momentum, some bulk
dynamics of your robot,

00:08:10.440 --> 00:08:13.110
and the contact forces you're
exerting on the world, which

00:08:13.110 --> 00:08:14.539
are also constrained.

00:08:14.539 --> 00:08:16.080
And in this sort of
six dimensional--

00:08:16.080 --> 00:08:19.890
12 dimensional if you
have velocities-- space

00:08:19.890 --> 00:08:22.860
with these relatively
limited constraints,

00:08:22.860 --> 00:08:25.560
you can actually do
very efficient planning

00:08:25.560 --> 00:08:29.730
and then map that in a second
pass back to the full figure

00:08:29.730 --> 00:08:31.650
out what my pinky's going to do.

00:08:31.650 --> 00:08:35.204
So we do that.

00:08:35.204 --> 00:08:36.620
We spent a lot of
time doing that,

00:08:36.620 --> 00:08:40.559
and we can now plan motions
for complicated humanoids that

00:08:40.559 --> 00:08:43.539
were far beyond our ability
to do it a few years ago.

00:08:43.539 --> 00:08:45.410
This was a major effort for us.

00:08:45.410 --> 00:08:47.460
My kids and I were
watching American Ninja

00:08:47.460 --> 00:08:51.870
Warrior at the time, so we did
all the Ninja Warrior tasks.

00:08:51.870 --> 00:08:54.960
So there were some
algorithmic ideas

00:08:54.960 --> 00:08:56.580
that were required for that.

00:08:56.580 --> 00:08:58.950
It was also just a software
engineering exercise

00:08:58.950 --> 00:09:02.714
to build a dynamics engine that
provided analytical gradients,

00:09:02.714 --> 00:09:04.380
exposed all the
sparsity in the problem,

00:09:04.380 --> 00:09:06.330
and wrote custom solvers
and things like that

00:09:06.330 --> 00:09:08.081
to make that work.

00:09:08.081 --> 00:09:09.330
It's not just about humanoids.

00:09:09.330 --> 00:09:13.890
We spent a day after we got
Atlas doing those things

00:09:13.890 --> 00:09:16.470
to show that we could make
a quadruped run around using

00:09:16.470 --> 00:09:17.616
the same exact algorithms.

00:09:17.616 --> 00:09:18.990
It took literally
less than a day

00:09:18.990 --> 00:09:22.820
to make all these examples work.

00:09:22.820 --> 00:09:24.430
There's another
level of optimization

00:09:24.430 --> 00:09:26.260
that's kicking around in here.

00:09:26.260 --> 00:09:29.290
So the humanoid, in some
sense when it's moving around,

00:09:29.290 --> 00:09:32.390
is a fairly continuous
dynamical system.

00:09:32.390 --> 00:09:34.750
There's punctuations when
your foot hits the ground

00:09:34.750 --> 00:09:36.640
or something like
this, so you think

00:09:36.640 --> 00:09:39.770
of that as sort of a smooth
optimization problem.

00:09:39.770 --> 00:09:42.124
There's also a discrete
optimization problem

00:09:42.124 --> 00:09:43.790
sitting in there,
too, even for walking.

00:09:43.790 --> 00:09:46.614
So if you think
about it, the methods

00:09:46.614 --> 00:09:48.655
I just talked about--
we're really talking about,

00:09:48.655 --> 00:09:49.849
OK, I move like this.

00:09:49.849 --> 00:09:51.640
I would prefer to move
something like this,

00:09:51.640 --> 00:09:53.140
but there's a
continuum of solutions

00:09:53.140 --> 00:09:54.460
I could possibly take.

00:09:54.460 --> 00:09:57.499
For walking, there's also
this problem of just saying,

00:09:57.499 --> 00:10:00.040
am I going to move my right foot
first or my left foot first?

00:10:00.040 --> 00:10:03.540
Am I going to step on cinder
block one or cinder block two?

00:10:03.540 --> 00:10:05.122
There really is a
discrete problem

00:10:05.122 --> 00:10:06.580
which gives a
combinatorial problem

00:10:06.580 --> 00:10:09.070
if you have to make
long-term decisions on that.

00:10:09.070 --> 00:10:11.210
And one of the things
we've tried to do well

00:10:11.210 --> 00:10:14.680
is be very explicit about
modeling the discrete aspects

00:10:14.680 --> 00:10:16.690
and the continuous
aspects of the problem

00:10:16.690 --> 00:10:21.280
individually and using the
right solvers that could think

00:10:21.280 --> 00:10:23.420
about both of those together.

00:10:23.420 --> 00:10:27.670
So here's an example of how
we do interactive footstep

00:10:27.670 --> 00:10:28.850
planning with the robot.

00:10:28.850 --> 00:10:32.320
If it's standing in front of
some perceived cinder blocks,

00:10:32.320 --> 00:10:35.980
for instance, the human can
quickly label discrete regions

00:10:35.980 --> 00:10:38.200
just by moving a mouse around.

00:10:38.200 --> 00:10:42.352
The regions that come out are
actually fit by an algorithm.

00:10:42.352 --> 00:10:44.560
They look small, because
they're trying to figure out

00:10:44.560 --> 00:10:46.601
if the center of the foot
was inside that region,

00:10:46.601 --> 00:10:48.165
the whole foot
would fit on that.

00:10:48.165 --> 00:10:50.290
And they're also thinking
about balance constraints

00:10:50.290 --> 00:10:51.830
and other things like that.

00:10:51.830 --> 00:10:54.410
But now we have discrete
regions to possibly step in.

00:10:54.410 --> 00:10:56.770
We have a combinatorial
problem and the smooth problem

00:10:56.770 --> 00:10:58.640
of moving my center
of mass and the like,

00:10:58.640 --> 00:11:00.850
and we have very good
new solvers to do that.

00:11:00.850 --> 00:11:02.950
And seeded inside
that, I just want

00:11:02.950 --> 00:11:06.280
to communicate that there's all
these little technical nuggets.

00:11:06.280 --> 00:11:10.270
We had to find a new way to
make really fast approximations

00:11:10.270 --> 00:11:13.060
of big convex regions
of free space.

00:11:13.060 --> 00:11:15.700
So we have optimizations
that just figured out--

00:11:15.700 --> 00:11:17.230
the problem of
finding the biggest

00:11:17.230 --> 00:11:20.780
polygon that fits inside all
those obstacles is NP hard.

00:11:20.780 --> 00:11:22.520
We're not going to solve that.

00:11:22.520 --> 00:11:26.620
But it turns out finding
a pretty good polygon can

00:11:26.620 --> 00:11:28.450
be done extremely fast now.

00:11:28.450 --> 00:11:30.700
And the particular
way we did it scales

00:11:30.700 --> 00:11:33.610
to very high dimensions
and complicated obstacles

00:11:33.610 --> 00:11:36.430
to the point where we could
do it on raw sensor data,

00:11:36.430 --> 00:11:40.060
and that was an enabling
technology for us.

00:11:40.060 --> 00:11:43.090
So our robot now, when
it's making plans--

00:11:43.090 --> 00:11:46.060
so the one on the left is
just walking towards the goal.

00:11:46.060 --> 00:11:48.222
The one on the right, we
removed a cinder block.

00:11:48.222 --> 00:11:50.680
And normally, a robot would
kind of get confused and stuck,

00:11:50.680 --> 00:11:52.721
because it's just thinking
about this local plan,

00:11:52.721 --> 00:11:54.460
local plan, local plan.

00:11:54.460 --> 00:11:56.335
It wouldn't be able to
stop and go completely

00:11:56.335 --> 00:11:57.470
in the other direction.

00:11:57.470 --> 00:11:59.720
But now, since we have this
higher level combinatorial

00:11:59.720 --> 00:12:03.820
planning on top, we can make
these big, long-term decision

00:12:03.820 --> 00:12:07.930
making tasks at
interactive rates.

00:12:07.930 --> 00:12:10.390
Also, the robot was too
big to walk through a door,

00:12:10.390 --> 00:12:12.440
so we had to walk
sideways through a door.

00:12:12.440 --> 00:12:14.190
And that was sort of
a standing challenge.

00:12:14.190 --> 00:12:17.380
The guy who started the program
putting footsteps down by hand

00:12:17.380 --> 00:12:19.660
said, whatever I do
in footstep planning,

00:12:19.660 --> 00:12:22.780
I will never lay down footsteps
to walk through a door again.

00:12:22.780 --> 00:12:25.600
That was the challenge.

00:12:25.600 --> 00:12:27.520
We did a lot of work on
the balancing control

00:12:27.520 --> 00:12:30.670
for the robot, so it's
a force controlled robot

00:12:30.670 --> 00:12:33.670
using hydraulic
actuators everywhere.

00:12:33.670 --> 00:12:35.242
Again, I won't go
into the details,

00:12:35.242 --> 00:12:37.450
but we thought a lot about
the dynamics of the robot.

00:12:37.450 --> 00:12:40.720
How do you cast that as
an efficient optimization

00:12:40.720 --> 00:12:43.810
that we can solve on the fly?

00:12:43.810 --> 00:12:46.510
And we were solving an
optimization at a kilohertz

00:12:46.510 --> 00:12:48.370
to balance the robot.

00:12:48.370 --> 00:12:50.870
So you put it all together.

00:12:50.870 --> 00:12:53.650
And as a basic competency,
how well does our robot

00:12:53.650 --> 00:12:55.020
walk around and balance?

00:12:55.020 --> 00:12:59.260
Here's one of the
examples at a normal speed

00:12:59.260 --> 00:13:02.380
from the challenge.

00:13:02.380 --> 00:13:04.420
So the robot just puts
its footsteps down ahead.

00:13:04.420 --> 00:13:06.280
The operator is mostly
just watching and giving

00:13:06.280 --> 00:13:07.196
high level directions.

00:13:07.196 --> 00:13:09.820
I want to go over here, and the
robot's doing its own thing.

00:13:14.609 --> 00:13:16.150
Now, all the other
teams I know about

00:13:16.150 --> 00:13:20.440
were putting down the footsteps
by hand on the obstacles.

00:13:20.440 --> 00:13:23.530
I don't know if someone else
was doing it autonomously.

00:13:23.530 --> 00:13:26.530
We chose to do it autonomously.

00:13:26.530 --> 00:13:28.390
We were a little bit
faster because of it,

00:13:28.390 --> 00:13:34.110
but I don't know
if it was enabling.

00:13:34.110 --> 00:13:36.270
But very proud of our
walking, even though it's

00:13:36.270 --> 00:13:37.150
still conservative.

00:13:37.150 --> 00:13:39.530
This is lousy
compared to a human.

00:13:39.530 --> 00:13:41.674
Yeah?

00:13:41.674 --> 00:13:45.076
AUDIENCE: So the obstacles are
modeled by the robot's vision,

00:13:45.076 --> 00:13:46.899
or do you actually preset them?

00:13:46.899 --> 00:13:49.440
RUSS TEDRAKE: So we knew they
were going to be cinder blocks.

00:13:49.440 --> 00:13:51.840
We didn't know the orientation
or positions of them,

00:13:51.840 --> 00:13:53.946
so we had a cinder block
fitting algorithm that

00:13:53.946 --> 00:13:55.320
would run on the
fly, snap things

00:13:55.320 --> 00:13:58.050
into place with the cameras--

00:13:58.050 --> 00:13:59.970
actually, laser scanner.

00:13:59.970 --> 00:14:02.120
And then we walk up stairs.

00:14:02.120 --> 00:14:03.870
Little things-- if you
care about walking,

00:14:03.870 --> 00:14:06.750
the heels are
hanging off the back.

00:14:06.750 --> 00:14:08.310
There's special
algorithms in there

00:14:08.310 --> 00:14:11.010
to balance on partial foot
contact and things like that.

00:14:11.010 --> 00:14:12.910
And that made the difference.

00:14:12.910 --> 00:14:17.770
We could go up there
efficiently, robustly.

00:14:17.770 --> 00:14:20.980
So I would say though,
for conservative walking,

00:14:20.980 --> 00:14:22.750
it really works well.

00:14:22.750 --> 00:14:24.832
We could plant these
things on the fly.

00:14:24.832 --> 00:14:26.290
And we also had
this user interface

00:14:26.290 --> 00:14:28.624
that if the foot step planner
ever did something stupid,

00:14:28.624 --> 00:14:31.165
the human could just drag a foot
around, add a new constraint

00:14:31.165 --> 00:14:31.750
to the solver.

00:14:31.750 --> 00:14:34.150
It would continue to solve
with a new constraint

00:14:34.150 --> 00:14:37.000
and adjust its solutions.

00:14:37.000 --> 00:14:38.950
We could do more dynamic plans.

00:14:38.950 --> 00:14:40.910
We could have it run
everything like that.

00:14:40.910 --> 00:14:42.618
We actually never
tried this on the robot

00:14:42.618 --> 00:14:44.290
before the
competition, because we

00:14:44.290 --> 00:14:45.831
were terrified of
breaking the robot,

00:14:45.831 --> 00:14:47.561
and we couldn't
accept the downtime.

00:14:47.561 --> 00:14:49.060
But now that the
competition's over,

00:14:49.060 --> 00:14:52.240
this is exactly
what we're trying.

00:14:52.240 --> 00:14:55.650
But the optimizations are slower
and didn't always succeed.

00:14:55.650 --> 00:14:59.590
So in the real scenario, we were
putting some more constraints

00:14:59.590 --> 00:15:02.040
on and doing much more
conservative gaits.

00:15:02.040 --> 00:15:04.540
The balance control I'd
say worked extremely well.

00:15:04.540 --> 00:15:08.680
So the hardest task was this
getting out of the car task.

00:15:08.680 --> 00:15:09.790
We worked like crazy.

00:15:09.790 --> 00:15:11.380
We didn't work on
it until the end.

00:15:11.380 --> 00:15:13.540
I thought DARPA was going
to scratch it, honestly.

00:15:13.540 --> 00:15:16.674
But in the last month, it became
clear that we had to do it.

00:15:16.674 --> 00:15:18.340
And then we spent a
lot of effort on it.

00:15:18.340 --> 00:15:21.097
And we put the car in
every possible situation.

00:15:21.097 --> 00:15:22.180
This was on cinder blocks.

00:15:22.180 --> 00:15:23.800
It's way high.

00:15:23.800 --> 00:15:26.580
It has to step down almost
beyond its reachability

00:15:26.580 --> 00:15:27.790
in the leg.

00:15:27.790 --> 00:15:30.340
This thing was just super solid.

00:15:30.340 --> 00:15:32.410
So Andres and Lucas
were the main designers

00:15:32.410 --> 00:15:33.910
of this algorithm.

00:15:33.910 --> 00:15:36.970
I'd say it's superhuman
in this regard, right?

00:15:36.970 --> 00:15:39.230
A human would not
do that, of course,

00:15:39.230 --> 00:15:41.530
but standing on one
foot while someone's

00:15:41.530 --> 00:15:45.075
jumping on the car like this--

00:15:45.075 --> 00:15:46.305
it really works well.

00:15:46.305 --> 00:15:48.430
In fact, the hardest part
of that for the algorithm

00:15:48.430 --> 00:15:51.346
was the fact that it's trying
to find out where the ground is,

00:15:51.346 --> 00:15:52.720
and the camera's
going like this.

00:15:52.720 --> 00:15:55.420
So that was the reason
it had this long pause

00:15:55.420 --> 00:15:58.120
before it went down.

00:15:58.120 --> 00:16:04.012
But there was one time
that it didn't work well,

00:16:04.012 --> 00:16:05.470
and it's hard for
me to watch this.

00:16:05.470 --> 00:16:11.590
But it turns out on the first--
you saw that little kick?

00:16:11.590 --> 00:16:13.060
This was horrible.

00:16:15.818 --> 00:16:17.917
I'll tell you exactly
what happened,

00:16:17.917 --> 00:16:19.750
but I think it really
exposed the limitation

00:16:19.750 --> 00:16:22.660
of the state of the art.

00:16:22.660 --> 00:16:26.710
So what happened in that
particular situation was

00:16:26.710 --> 00:16:30.310
the robot was almost
autonomous in some ways,

00:16:30.310 --> 00:16:33.010
and we basically tried
to have the human

00:16:33.010 --> 00:16:34.420
have to do almost nothing.

00:16:34.420 --> 00:16:36.880
And in the end, we got
the humans checklist

00:16:36.880 --> 00:16:40.000
down to about five items,
which was probably a mistake,

00:16:40.000 --> 00:16:42.100
because we screwed
up on the checklist.

00:16:42.100 --> 00:16:45.120
So one of the five
items was to--

00:16:45.120 --> 00:16:47.260
we have one set of programs
that are running when

00:16:47.260 --> 00:16:49.150
the robot's driving the car.

00:16:49.150 --> 00:16:51.335
And then all the
human had to do was

00:16:51.335 --> 00:16:53.710
turn off the driving controller
and turn on the balancing

00:16:53.710 --> 00:16:54.670
controller.

00:16:54.670 --> 00:16:57.130
But it was exciting and the
first day of the competition.

00:16:57.130 --> 00:16:59.752
And we turned on the balancing
controller, forgot to turn off

00:16:59.752 --> 00:17:00.710
the driving controller.

00:17:00.710 --> 00:17:04.450
So the ankle was still
trying to drive the car.

00:17:04.450 --> 00:17:06.290
Even that, the controller
was robust enough.

00:17:06.290 --> 00:17:10.300
So I really think there's
this fundamental thing

00:17:10.300 --> 00:17:12.430
that if you're close
to your nominal plan,

00:17:12.430 --> 00:17:13.750
things were very robust.

00:17:13.750 --> 00:17:16.136
But what happened is the ankle
is still driving the car.

00:17:16.136 --> 00:17:18.010
I think we could balance
with the ankle doing

00:17:18.010 --> 00:17:20.650
the wrong thing, except the
ankle did the wrong thing

00:17:20.650 --> 00:17:24.550
just enough that the tailbone
hit the seat of the car.

00:17:24.550 --> 00:17:27.550
That was no longer something
we could handle, right?

00:17:27.550 --> 00:17:30.860
So there was no contact
sensor in the butt.

00:17:30.860 --> 00:17:33.300
That meant the dynamics
model was very wrong.

00:17:33.300 --> 00:17:35.090
The state estimator
got very confused.

00:17:35.090 --> 00:17:36.790
The foot came off the ground,
and the state estimator

00:17:36.790 --> 00:17:39.123
had an assumption that the
feet should be on the ground.

00:17:39.123 --> 00:17:41.500
That's how it knew where
it was in the world.

00:17:41.500 --> 00:17:43.930
And basically, the
controller was hosed, right?

00:17:43.930 --> 00:17:46.971
And that was the only time we
could have done that badly--

00:17:46.971 --> 00:17:48.220
the vibrations and everything.

00:17:48.220 --> 00:17:50.482
I had emails from people
of all walks of life

00:17:50.482 --> 00:17:52.690
telling me what they thought
was wrong with the brain

00:17:52.690 --> 00:17:56.140
of the robot from
shaking like that.

00:17:56.140 --> 00:17:57.560
But that was a bad thing.

00:17:57.560 --> 00:17:59.970
So you know I think
fundamentally,

00:17:59.970 --> 00:18:01.726
if we're thinking about plans--

00:18:01.726 --> 00:18:03.100
and that's what
we know how to do

00:18:03.100 --> 00:18:07.772
at scale for high dimensional
systems is single solutions--

00:18:07.772 --> 00:18:08.980
then we're close to the plan.

00:18:08.980 --> 00:18:09.646
Things are good.

00:18:09.646 --> 00:18:12.160
When we're far from the
plan, we're not very good.

00:18:12.160 --> 00:18:15.370
And a change in the
contact situation--

00:18:15.370 --> 00:18:18.710
even if it's in a Cartesian
space very close--

00:18:18.710 --> 00:18:22.180
change in the contact situation
is a big change to the plan.

00:18:22.180 --> 00:18:24.580
There's lots of
ways to address it.

00:18:24.580 --> 00:18:26.784
We're doing all of them now.

00:18:26.784 --> 00:18:28.450
It's all fundamentally
about robustness.

00:18:28.450 --> 00:18:31.069
But ironically, the
car was the only time

00:18:31.069 --> 00:18:32.610
we could have done
that badly, right?

00:18:32.610 --> 00:18:35.770
So every other place, we
worked out all these situations

00:18:35.770 --> 00:18:38.830
where, OK, the robot's walking,
and then something bad happens

00:18:38.830 --> 00:18:42.220
and someone lances
you or something.

00:18:42.220 --> 00:18:43.420
We had recovery.

00:18:43.420 --> 00:18:45.640
And then even if it
tried to take a step--

00:18:45.640 --> 00:18:47.920
even if that failed, it
would go into a gentle mode

00:18:47.920 --> 00:18:49.090
where it would
protect its hands,

00:18:49.090 --> 00:18:50.130
because we were afraid
of breaking the hands.

00:18:50.130 --> 00:18:52.300
It would fall very
generally to the ground.

00:18:52.300 --> 00:18:54.070
All that was good.

00:18:54.070 --> 00:18:56.320
We turned it off exactly
once in the competition.

00:18:56.320 --> 00:18:57.490
We turned it off when
we were in the car,

00:18:57.490 --> 00:18:59.800
because we can't take a step to
recover when you're in the car

00:18:59.800 --> 00:19:01.420
and you're the same
size of the car.

00:19:01.420 --> 00:19:03.430
And we didn't even want
to protect our hands,

00:19:03.430 --> 00:19:07.532
because once we got our hand
stuck on the steering wheel.

00:19:07.532 --> 00:19:08.990
So anyways, that
was the only thing

00:19:08.990 --> 00:19:12.041
we could have sort of shaken
ourselves silly and fallen.

00:19:12.041 --> 00:19:12.790
And what happened?

00:19:12.790 --> 00:19:15.220
We fell down with
our 400 pound robot.

00:19:15.220 --> 00:19:17.630
We broke the arm--

00:19:17.630 --> 00:19:18.970
the right arm.

00:19:18.970 --> 00:19:21.360
Sadly, all of our
practices ever were

00:19:21.360 --> 00:19:23.821
doing all the
tasks right handed,

00:19:23.821 --> 00:19:26.070
but we got to show off a
different form of robustness.

00:19:26.070 --> 00:19:29.250
So actually, because we had so
much autonomy in the system,

00:19:29.250 --> 00:19:31.920
we flipped a bit
and said, let's use

00:19:31.920 --> 00:19:33.150
the left arm for everything.

00:19:33.150 --> 00:19:36.506
Which is more than just map the
joint coordinates over here.

00:19:36.506 --> 00:19:38.130
It meant you had to
walk up to the door

00:19:38.130 --> 00:19:40.470
on the other side of the door.

00:19:40.470 --> 00:19:43.680
The implications
back up quite a bit.

00:19:43.680 --> 00:19:46.970
After having our arm
just completely hosed,

00:19:46.970 --> 00:19:50.310
we were able to go through and
do all the rest of the tasks

00:19:50.310 --> 00:19:52.460
except for the drill,
which required two hands.

00:19:52.460 --> 00:19:53.460
We couldn't do that one.

00:19:53.460 --> 00:19:55.290
We had to pick up the
drill and turn it on.

00:19:55.290 --> 00:19:58.650
So we ended the
day in second place

00:19:58.650 --> 00:20:01.176
with a different
display of robustness.

00:20:01.176 --> 00:20:02.856
AUDIENCE: That's still
pretty damn good.

00:20:02.856 --> 00:20:04.730
RUSS TEDRAKE: We were
happy, but not as happy

00:20:04.730 --> 00:20:05.729
as if we had not fallen.

00:20:08.050 --> 00:20:08.550
OK.

00:20:08.550 --> 00:20:11.829
So I think walking around,
balancing-- we're pretty good,

00:20:11.829 --> 00:20:12.870
but there's a limitation.

00:20:12.870 --> 00:20:18.070
I really do think everybody has
that limitation to some extent.

00:20:18.070 --> 00:20:19.820
The manipulation
capabilities of the robot

00:20:19.820 --> 00:20:23.060
were pretty limited, just
because we didn't need

00:20:23.060 --> 00:20:24.920
to do it for the challenge.

00:20:24.920 --> 00:20:27.670
The manipulation
requirements were minimal.

00:20:27.670 --> 00:20:29.060
You had to open doors.

00:20:29.060 --> 00:20:32.150
Picking up a drill was the
most complicated thing.

00:20:32.150 --> 00:20:34.820
We actually had a lot of
really nice robotic hands

00:20:34.820 --> 00:20:38.580
to play with, but they all broke
when you started really running

00:20:38.580 --> 00:20:39.920
them through these hard tests.

00:20:39.920 --> 00:20:42.950
So we ended up with these
sort of lobster claw

00:20:42.950 --> 00:20:44.940
kind of grippers, because
they didn't break.

00:20:44.940 --> 00:20:46.790
And they were robust,
and they worked well.

00:20:46.790 --> 00:20:50.420
But it limited what we
could do in manipulation.

00:20:50.420 --> 00:20:53.030
Again, the planning
worked very well.

00:20:53.030 --> 00:20:54.950
We could pick up
a board and even

00:20:54.950 --> 00:20:58.490
plan to make sure that the
board now didn't intersect

00:20:58.490 --> 00:20:59.810
with other boards in the world.

00:20:59.810 --> 00:21:02.120
And we have really good
planning capabilities,

00:21:02.120 --> 00:21:06.230
and those worked at interactive
rates-- the kinematic plans.

00:21:06.230 --> 00:21:09.330
But the grasping was open loop,
so there's really no feedback.

00:21:09.330 --> 00:21:12.800
So there's current sensing
just to not overheat the hands.

00:21:12.800 --> 00:21:14.780
But basically, you
do a lot of thinking

00:21:14.780 --> 00:21:17.120
to figure out how to get
your hand near the board.

00:21:17.120 --> 00:21:19.369
And then you kind of close
your eyes and go like this,

00:21:19.369 --> 00:21:20.810
and hope it lands in the hand.

00:21:20.810 --> 00:21:22.070
And most of the time, it does.

00:21:22.070 --> 00:21:24.890
Every once in
awhile, it doesn't.

00:21:24.890 --> 00:21:26.990
We experimented with
every touch sensor

00:21:26.990 --> 00:21:28.230
we could get our hands on.

00:21:28.230 --> 00:21:29.930
That wasn't meant to be a pun.

00:21:29.930 --> 00:21:31.460
And we tried cameras
and everything,

00:21:31.460 --> 00:21:33.800
but they were all just too
fragile and difficult to use

00:21:33.800 --> 00:21:36.091
for the competition.

00:21:36.091 --> 00:21:38.090
We're doing a lot of work
now doing optimization

00:21:38.090 --> 00:21:42.260
for grasping, but I'll
skip over that for time.

00:21:42.260 --> 00:21:44.060
So the other piece
was, how does the human

00:21:44.060 --> 00:21:47.360
come into the perception
side of the story?

00:21:47.360 --> 00:21:50.690
So one of these tasks
was moving debris

00:21:50.690 --> 00:21:52.430
out from in front of a door.

00:21:52.430 --> 00:21:56.360
This is what it looked like
in the original version

00:21:56.360 --> 00:21:57.700
of the competition-- the trials.

00:21:57.700 --> 00:21:59.150
The robot would come up
and throw these boards out

00:21:59.150 --> 00:22:01.580
of the way, and you see the
human operators over there

00:22:01.580 --> 00:22:05.850
with her big
console of displays.

00:22:05.850 --> 00:22:08.360
This is what the laser
in the robot's head sees.

00:22:08.360 --> 00:22:09.950
We have a spinning laser.

00:22:09.950 --> 00:22:11.790
We also have stereo vision.

00:22:11.790 --> 00:22:14.000
But the laser
reconstruction of this

00:22:14.000 --> 00:22:15.621
gives you a mess of points.

00:22:15.621 --> 00:22:17.495
If you asked a vision
algorithm-- some of you

00:22:17.495 --> 00:22:19.280
are vision experts
I'm sure in the room.

00:22:19.280 --> 00:22:20.810
If you asked a vision
algorithm to figure out

00:22:20.810 --> 00:22:22.490
what's going on in
that mess of points,

00:22:22.490 --> 00:22:24.917
it's an extremely hard problem.

00:22:24.917 --> 00:22:26.250
But we have a human in the loop.

00:22:26.250 --> 00:22:29.870
So the idea is that one
or two clicks from a human

00:22:29.870 --> 00:22:31.700
can turn that from an
intractable problem

00:22:31.700 --> 00:22:33.350
to a pretty simple problem.

00:22:33.350 --> 00:22:35.420
Just say, there's
a two by four here.

00:22:35.420 --> 00:22:38.870
And then now a local search
can do sort of RANSAC type

00:22:38.870 --> 00:22:42.050
local optimizations to
find the best fit to a two

00:22:42.050 --> 00:22:45.712
by four to that local group of
points, and that works well.

00:22:45.712 --> 00:22:48.170
And so the robot didn't have
to think about the messy point

00:22:48.170 --> 00:22:49.742
clouds when it's
doing its planning.

00:22:49.742 --> 00:22:51.200
It could think
about the simplified

00:22:51.200 --> 00:22:53.480
geometry from the CAD models.

00:22:53.480 --> 00:22:56.689
And most of the planning
was just on the CAD models.

00:22:56.689 --> 00:22:58.730
So this is what it looks
like to drive the robot.

00:22:58.730 --> 00:23:01.900
So you click somewhere
saying there's a valve,

00:23:01.900 --> 00:23:03.774
then the perception
algorithm finds a valve.

00:23:03.774 --> 00:23:04.940
Then the robot starts going.

00:23:04.940 --> 00:23:08.900
It actually shows you a ghost
of what it's about to do.

00:23:08.900 --> 00:23:10.570
And then if you're
happy with it,

00:23:10.570 --> 00:23:12.939
and if all things are
going well, you just watch.

00:23:12.939 --> 00:23:15.230
But if it looks like it's
about to do something stupid,

00:23:15.230 --> 00:23:18.140
you can come in, stop,
interact, change the plans,

00:23:18.140 --> 00:23:21.380
and let it do its thing.

00:23:21.380 --> 00:23:24.950
It's kind of fun to watch the
robot view of the world, right?

00:23:24.950 --> 00:23:27.402
So this is what the robot sees.

00:23:27.402 --> 00:23:28.610
It throws down its footsteps.

00:23:28.610 --> 00:23:30.980
It's deciding how to
walk up to that valve.

00:23:30.980 --> 00:23:34.280
Again, when the
right arm was broken,

00:23:34.280 --> 00:23:35.870
this was one of
our practice runs.

00:23:35.870 --> 00:23:38.000
The right arm was
broken, it had a valve.

00:23:38.000 --> 00:23:39.230
We had to bit flip, and
now it had to walk over

00:23:39.230 --> 00:23:40.521
to the other side of the valve.

00:23:40.521 --> 00:23:42.110
And there's a lot
of things going on.

00:23:42.110 --> 00:23:44.776
A lot of pieces had to work well
together to make all this work.

00:23:47.350 --> 00:23:51.220
One of the questions that
I'll get before you ask it.

00:23:51.220 --> 00:23:53.050
If you've written it
down, OK, that's fine.

00:23:53.050 --> 00:23:54.280
Why were the robots so slow?

00:23:54.280 --> 00:23:56.350
Why were they standing still?

00:23:56.350 --> 00:24:00.190
A lot of people out there
waiting for the human, maybe,

00:24:00.190 --> 00:24:01.075
but for us it wasn't.

00:24:01.075 --> 00:24:02.560
It wasn't the planning time.

00:24:02.560 --> 00:24:04.690
The planning algorithms
were super fast.

00:24:04.690 --> 00:24:07.090
Most of the time, we were
waiting for sensor data.

00:24:07.090 --> 00:24:08.590
And that meant there
was two things.

00:24:08.590 --> 00:24:11.500
There was waiting for the
laser to spin completely around

00:24:11.500 --> 00:24:13.600
and also just being
conservative-- wanting

00:24:13.600 --> 00:24:16.120
to get that laser data
while the robot was stopped.

00:24:16.120 --> 00:24:18.400
And then there was getting
the laser data back

00:24:18.400 --> 00:24:20.230
to the computer that
had the fast planning

00:24:20.230 --> 00:24:21.320
algorithms in back.

00:24:21.320 --> 00:24:22.240
So if there was a
network blackout,

00:24:22.240 --> 00:24:24.010
we had to wait a
little bit, and that

00:24:24.010 --> 00:24:25.840
meant we were standing still.

00:24:25.840 --> 00:24:28.180
But we've actually done
a lot of work in lab

00:24:28.180 --> 00:24:30.730
to show that we don't
have to stand still.

00:24:30.730 --> 00:24:33.460
This is now the robot
walking with its laser

00:24:33.460 --> 00:24:36.820
blindfolded and using
only stereo vision

00:24:36.820 --> 00:24:40.000
using one of the capabilities
that came out of John's lab

00:24:40.000 --> 00:24:43.990
and others to do stereo fusion.

00:24:43.990 --> 00:24:47.290
The laser gives very
accurate points,

00:24:47.290 --> 00:24:49.759
but it gives them
slowly at a low rate.

00:24:49.759 --> 00:24:51.550
And you have to wait
for it to spin around.

00:24:51.550 --> 00:24:55.979
The camera is very dense, very
high rate, but very noisy.

00:24:55.979 --> 00:24:58.270
And John and others have
developed these new algorithms

00:24:58.270 --> 00:25:00.970
that can do real time
filtering of that noisy data,

00:25:00.970 --> 00:25:03.430
and we demonstrated that
they were good enough

00:25:03.430 --> 00:25:04.539
to do walking on.

00:25:04.539 --> 00:25:06.580
And so we put all the
pieces together-- real time

00:25:06.580 --> 00:25:08.680
footstep planning, real
time balancing, real time

00:25:08.680 --> 00:25:11.305
perception-- and we were able to
show we can walk continuously.

00:25:11.305 --> 00:25:12.880
This will be the future.

00:25:12.880 --> 00:25:14.972
So we had to do networking.

00:25:14.972 --> 00:25:16.180
We optimized network systems.

00:25:16.180 --> 00:25:19.360
We had to build servers, unit
test logistics, politics.

00:25:19.360 --> 00:25:22.270
It was exhausting.

00:25:22.270 --> 00:25:24.880
I think it was overall
incredibly good

00:25:24.880 --> 00:25:26.780
experience-- a huge success.

00:25:26.780 --> 00:25:28.690
I think the robots
can move faster

00:25:28.690 --> 00:25:31.915
with only small changes
mostly on the perception side.

00:25:31.915 --> 00:25:33.040
The walking was sufficient.

00:25:33.040 --> 00:25:35.677
We can definitely do better.

00:25:35.677 --> 00:25:37.010
The manipulation was very basic.

00:25:37.010 --> 00:25:38.468
I think we need to
do better there,

00:25:38.468 --> 00:25:40.750
but we didn't have
to for those tasks.

00:25:40.750 --> 00:25:44.500
The robustness
dominated everything.

00:25:44.500 --> 00:25:46.660
So I'll just end
and take questions,

00:25:46.660 --> 00:25:49.660
but I'll show this sort
of fun, again, robot view.

00:25:49.660 --> 00:25:53.430
This is the robot's God's
eye view of the world

00:25:53.430 --> 00:25:54.910
while it's doing
all these tasks.

00:25:54.910 --> 00:25:58.810
You can sort of see what the
robot labels with the geometry

00:25:58.810 --> 00:26:01.199
and what it's
leading its points.

00:26:01.199 --> 00:26:03.490
And it's just kind of fun to
have on in the background,

00:26:03.490 --> 00:26:06.190
and then I'll take
any questions.

00:26:06.190 --> 00:26:09.240
[APPLAUSE]