Lecture 11: Walking

Flash and JavaScript are required for this feature.

Download the video from iTunes U or the Internet Archive.

Topics covered: Walking

Instructors: Russell Tedrake

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

RUSS TEDRAKE: OK, welcome back. At the end of last time, we said we'd built up a pretty good arsenal of optimal control tools. Now, we're going to start pushing them in exciting directions, OK?

So many of you know I work on walking robots. It's one of my passions. I think it's one of clear unsolved problems in robotics, right?

We have so many things we can do with robotic arms. Relatively, we have almost nothing that we can do with walking robots. Walking down the street is still a challenge, OK?

So in my mind, one of the essential problems in robotics, it also happens to be the thing that animals do just incredibly well. And even the really dumb animals, you know, locomote very, very, very well. So some people might even argue it's a predecessor to higher cognitive function. We'll see.

OK, so there's lots of good reasons why walking robots are tough. There's mechanical problems, of course. Suddenly, if you're a walking robot, you have to start carrying your actuators.

So you didn't have to do that as a robotic arm. And that's created lots of interesting work in actuator technologies. How do you carry an actuator that can produce torque accurately, let's say?

If you have to carry its own weight, that's been a challenging problem. There's lots of interesting work there. But I will stand by. I think that the reason we don't have walking robots today is not actuators.

It's not power density. These are problems, but I think the fundamental challenge is in control. OK. And we are now well-suited to make progress on that control problem.

OK. So based on what we've learned so far, walking robots aren't that different than the Acrobot and the CartPole. There's two things that are different dynamically speaking about a walking robot. The first is we have to think about limit cycles instead of just trajectories. And we're going to talk for some time about that.

The first thing was limit cycles. The second thing is impacts. Walking robots, every time their feet hit the ground, we have to start dealing with impact.

So dealing with periodic motions and dealing with impacts are the two big technical hurdles. And they're not even that big necessarily, but they're the technical hurdles of taking the techniques we've already developed and applying them to walking robots.

And impacts more generally might be called sort of hybrid dynamics, where you have a mixing of discrete time and continuous time dynamics. OK. So before I launch into some of the models of walking, which we'll get to pretty quick, let's do a toy example to start thinking about limit cycles and limit cycle stability.

OK. So you can get your head around limit cycle stability with some very simple oscillator type circuits. My favorite simple oscillator is the Van der Pol oscillator.

OK. So it's a second order system. These are the standard equations for Van der Pol oscillator where mu is some parameter you can choose.

You can look at it pretty quickly and start thinking about it almost like a very simple spring mass damper system. The only complicated part is it's got state dependent nonlinear damping. So think about it almost like a spring mass system, but it's got nonlinear damping, state dependent damping.

And in particular, if you look at it, you can think of it as having, if the magnitude of x is less than 1, then is that positive damping or negative damping?

AUDIENCE: [INAUDIBLE]

RUSS TEDRAKE: There's a sign change in the word damping somewhere in there, too. It's going to add energy to the system if x is less than 1, which I think is negative damping typically. And if x is greater than 1, it's going to remove energy. It's going to dissipate energy, right?

So it's not too hard to believe that you're going to end up with something that oscillates. If it's adding energy for small x's and removing energy for large X's. It's not too surprising maybe that you get this thing going. And it starts oscillating.

I might as well show you the real trajectories instead of drawing them badly.

So if I do a phase plot of this system, which is two-dimensional-- it's got a walking robot in the middle. Sorry-- close figures. There we go. OK. And yeah, that's good. That's excellent.

OK. So these are three plots, the three trajectories from three different initial conditions, one that started off with some large negative initial conditions in both theta and theta dot-- or x and x dot, sorry. And damping took the energy out and started going to this stable oscillation. From another very large trajectory, it did roughly the same thing, went to that same basic oscillation.

And then even from very small initial conditions near the origin, it'll swing out and start going to these same characteristic oscillation in phase space. So this is exactly the picture you want to have in your head of a stable limit cycle behavior. It's a periodic motion.

When I'm away from that motion, I return to it, right? This is exactly what we want to capture by saying a stable limit cycle. It turns out it's not sort of trivial to write down what you mean by a stability in the limit cycle context. We'll see more about that in a second.

But first, if you just sort of-- let's just use our imagine and talk about some asymptotic stability to this orbit. Does anybody know it from this system whether this system is globally stable? Why is it not globally stable?

AUDIENCE: [INAUDIBLE]

RUSS TEDRAKE: Good. I think it's true that it's globally stable except for a set of measures 0 is the way you say it, right? Because, darn it, if you start at the origin, it's not going to get there. It'll never start moving from the origin.

Any other initial conditions will find their way back to that stable limit cycle. So why is it not trivial to say what I mean by a stable limit cycle? Well, let's plot that same system in the time domain, same initial conditions. Everything is the same.

Here we go. Same exact trajectories, you can see they all go to their characteristic oscillation pretty quickly, in fact, right? But you can see it's crystal clear here that the trajectories never synchronize themselves in time. And that's not too surprising.

So you're on the limit cycle. If there's two trajectories that are a different place in phase on the limit cycle, there's nothing that's forcing them to come together. So they'll continue to be apart in time.

Such a simple thing, but actually that complicates our ability to define stability. We can't talk about trajectories asymptotically converging anymore because they don't. They asymptotically converge to some manifold, but that's harder to describe. As such, we actually have relatively few good tools for describing limit cycle stability.

The standard tool-- well, actually let me say a few things more about limit cycle before I tell you the standard tool. But there's actually a few things that are interesting, just good general knowledge, about limit cycles. It turns out, if you have a closed region in state space, meaning trajectories that are inside this region never leave this region, and if that region of state space has no fixed points, then it's got a closed orbit.

There's got to be something. It doesn't have to be a stable limit cycle, but it has to repeat itself. If you stay inside a region forever and you don't go to a fixed point, then you have to have a closed orbit. That's the Poincaré and Bendixon theorem.

And it's also interesting to note that, if you have a dynamical system described by a potential function, like any system that's described by Lyapunov function, that cannot have a limit cycle. Anything that's uniformly going down a gradient can't have a limit cycle dynamic. So a lot of Lyapunov theory doesn't apply directly.

And another thing is that people don't always call any closed orbits-- these are just random factoids about limit cycle before we go on, but they're good culture here. But if you have a closed orbit in state space, if it's stable, people call it a limit cycle. If it's unstable, people call it a limit cycle. If it's marginally stable, people don't really use the word limit cycle.

So for instance, on the undamped pendulum, the marginally stable orbits, there's no stability properties on these. People would typically not use the word limit cycle to describe those closed orbits on the phase space. But anything that has the sort of converging or diverging property around a single trajectory-- so what's another good way to say it?

There's some words that I liked about the trajectories have to be isolated in space, basically. If these closed orbits are not isolated in space, if there's not something sort of separating between them, then people don't use the word limit cycle to describe it.

OK, so we can't talk about asymptotic convergence of trajectories on limit cycles. We can't use vanilla Lyapunov functions to talk about stability on limit cycles. So what do people do, right?

There's a standard trick that most people play. And that's the idea of Poincaré sections and Poincaré maps. How many people have seen Poincaré maps before? OK.

Here's the idea. So I've got some orbit in state space. This is XXed out of my Van der Pol oscillator. Instead of talking about convergent to some orbit, I'm going to define a particular surface in this state space.

Let's say I define-- that's a bad choice. I've got 40 colors here. I had to pick the one that was-- that's not a very good choice either. Green's got to work for me, yeah? They're all fairly toned down.

OK, let's pick a particular section in state space. Instead of talking about the convergence in this continuous dynamics to this orbit, let's just look at the dynamics of that limit cycle every time it crosses this section. OK. This is called the surface of section, often referred to as just some big S.

And a surface of section, if it's the case that all trajectories that leave the surface of section eventually return to the surface of section-- I'm going to write this down-- then defining a surface of section allows you to convert the continuous time stability of a limit cycle into the discrete time stability of a fixed point.

So we know a lot about defining stability for fixed points. The big idea is we're going to turn limit cycle stability into fixed point stability. And the name of the technique is the Poincaré map technique, let's say.

OK. So I've got some continuous dynamical system. Let's make it simple. Let's say there's no control to speak of, no time dependence. Let's just keep the simplest sort of analysis. I've got some continuous time dynamical system.

Let me call tc the time of crossing the surface of section. So I'll say tcn is n-th crossing of S. That allows me to define a discrete time system, which I'm going to call xp of n.

xp of n is x at tc of n. I'm just trying to be very careful about my notation. I'm going to use p for any time the vector x I'm talking about lives on the Poincaré map. I'm trying to be careful about the transition from continuous time to discrete time.

And the result is a discrete time map, which I typically call P for Poincaré map. It can be linear or non-linear, right? In general, it's non-linear for us.

Defining that surface of section defines for me a map which says, if I cross the surface of section at the n-th time at state xp, then the next time I cross it's going to be xp n plus 1, right? So if I'm on the Van der Pol oscillator idea, let's say xpn is something here.

I've defined my surface of section, in this case, to be the place where x equals 0. x dot is greater than 0 just to define only half of that line there. It's a line segment. This thing we saw is going to go around like this. It's going to get a little closer to the nominal trajectory. It's going to come back here.

So this mapping is what I'm capturing at p. If I go around again, I'll come here, yeah? The key idea is that, if that map exists for all states for any time on this map-- I go around, and I can get back to the map-- then knowing that you have stability on this map that points in xp, if xp as it evolves in time goes to a fixed point, then that must also imply that you go to a limit cycle fixed point in continuous domain.

Nothing else is allowed to happen. If I'm on this state in my continuous time world, then there's no time dependent functions here. There's no other state that I'm not looking at.

If I'm on this trajectory, I'm going to stay on that trajectory. Of course, a disturbance or something could knock me off, but we're just looking at the nominal system. So fixed point stability on the map implies limit cycle stability in continuous time, OK?

Awesome. Yeah, I don't really need that yet. If I work out the map for the Van der Pol oscillator, let me draw this map now as xp at n-th step going to xp at the n plus 1-th step.

At 0, it doesn't go anywhere. So that's an easy point to draw. I know it crosses 0.

I'm being a little loose here. So what I really care about is, since I know the position is 0, really what I'm drawing is sort of the velocity at that, let me call it, xp dot. Should I?

Well, xpn has two elements the way I've defined it here. Only one of them has anything interesting going on. I'm not going to write a dot on top of my discrete time system just because I don't like that notation. But I'm going to draw the only interesting variable of the two-dimensional object xn.

If I look at my velocity at the n-th crossing going to the velocity of the n plus 1 crossing, it turns out, if it's in here, I know from simulation there that it gets bigger every time. Getting bigger on this discrete map means that it's greater than the line of slope 1. Let me draw the line of slope 1 here.

On the Van der Pol oscillator, it starts off getting bigger, bigger, bigger. And then at some point, it crosses here and goes off like this. I think I can even tell it to generate that, so you believe me, Poincaré.

And I did it on top of my other plot again. So that's the Poincaré map now of this system. The blue line, this is xpk versus xpk plus 1. And that red line is the line of slope 1.

This is a luxury of the sort of low-dimensional systems here. If I have a map from one variable to another variable, then just like the flows on a line, this is a discrete time map. But I can actually analyze everything graphically.

So what do I know? I already told you by writing that equation that, if I look at my Poincaré map there, the place where it crosses the line of slope 1 is a special point. What is it?

AUDIENCE: Fixed point.

RUSS TEDRAKE: It's a fixed point, yeah. I've actually got another fixed point here at this, the origin. Just by inspection, I can tell you about the stability of that fixed point.

So when I'm over here, I'm getting bigger. When I'm over here, I'm getting smaller. So I can actually, just by inspection, tell you that that's going to be a stable fixed point.

We have to be a little careful about it. I'm going to show you. In fact, the right way to think about it is the staircase technique. OK. So if I want to follow the dynamics of this iterated map, how many people see the staircase pictures? Yeah.

OK, so let's say I start in my system, and xp is some small number here. And the right way to think about that is the next time I'm going to go to this new value I can sort of graphically copy that down to my next xp by drawing a line to the line of slope 1.

And then I can, from there, evaluate where the next xp is going to be. And now it goes right into the fixed point. That should have been a little flatter than I ended up making it. And then on this side, similarly, I can got to here right into the fixed point. Yes?

AUDIENCE: Could you kind of emulate a flow diagram that plotted xpn plus 1 minus xpn? And then instead of reaching the line of slope of 1 when it crosses the--

RUSS TEDRAKE: Absolutely, sure. Sure, you could subtract out xpn from there and get it in terms of 0s. Yeah. The thing that might be misleading about that, you have to be careful. Because discrete time systems can do something that continuous time systems can't do-- is they can jump, right?

So I could actually define something that then looked a little benign, but is actually unstable. If I had something that looks like this, what does that do? You go here, here. And you know, well, I guess it goes down to here.

That one might actually catch-- shoot, I didn't make my point. You could limit cycle here like this or something, right? I did a more dramatic plot to make my point.

It's pretty funny that it's all unitless. So the fact that I gave myself that problem is ridiculous. You can imagine starting off here, getting bigger and bigger, and off you go, right?

OK, so what are the conditions for stability, local stability on that fixed point? You all know what they are, but just now you can maybe see them graphically by my exaggerated example. What are the conditions on the local stability? What does the slope of that line have to be?

AUDIENCE: It has to be less than 1?

RUSS TEDRAKE: The magnitude has to be less than 1, right? It could be sloped like this, sloped like this. What matters is that it's not sloped past the line.

The other, the simpler thing, to draw for instability would be if my plot had gone like this, that it's clearly unstable, right? Because then I'd go like this, this, this. Then off I go to infinity.

But it also happens if I'm unstable like this because I oscillate myself off to infinity.

If I look at the gradient of p with respect to x at some fixed point, the eigenvalues of that had better be less than 1 for it to be stable. Yeah.

AUDIENCE: So if you're linearized, then you're basically at that point.

RUSS TEDRAKE: So that's a statement about local stability. One of the cool things about these maps, actually, is people have stronger-- certainly, graphically I did sort of a stronger thing. I said I could just look at this and know that it was stable.

People actually know quite a bit about the properties of these maps. Just again for culture, it turns out, if you have a unimodal map like this, if the gradient doesn't change same and if it's locally stable, then you can infer global stability. There's a citation in the notes where you can see the careful result of that.

Some of the Koditschek's hopping work exploited that result. So people do know some things, more strong global properties of these maps. But the trivial one to think about is this local stability.

OK, so we know a bit now about how to talk about whether a limit cycle system is stable. It's a little more subtle than fixed point stability, but not too bad. The hard part comes if you can't find a mapping where it always returns to itself. And then you have problems, right?

But in this case, you can say strong things about the stability. OK. Now, we're set up to talk about walking, so my favorite topic here. Let me actually say historically, I mean, people have obviously been interested in walking for a long, long time.

Some people say that the first sort of serious study of legged locomotion was by a photographer. What's his name?

AUDIENCE: Muybridge.

RUSS TEDRAKE: Muybridge, good. Yeah. So a photographer had a sort of a fast shutter speed camera. And he had animals run in front of graph paper, basically.

And he tried to win a bet, I think, was the story. Because people weren't sure, when horses were doing their sort of certain gait, whether they-- I think it was a gallop. They asked, when the horse gallops does, he ever actually get all four feet off the ground?

And the guy who sort of proved that was a photographer, that took a picture of all four feet coming off the ground. And you can get his books, which are these fantastic books of all sorts of animals, just pictures and pictures and pictures of all sorts of animals, sort of almost stroke photography of these animals walking in front of graph paper. You get elephants, and camels, and humans, and babies, and horses, and dogs. You name it.

So if you ever sort of are curious what the gate of a certain animal looks like, you can go to Muybridge. And he also sort of started defining this language for describing the gaits of different animals, like which feet are on the ground at which time and stuff like that. So this was in the turn of the century, the late 1800s, early 1900s. This photographer sort of started our field.

More recently, though, the first sort of serious modeling, which is the impact that sort of remained today, I think, was McMahon at Harvard. He was a biomechanicist. And he observed that EMG in your leg, if you look at the muscle activity, Electromyographic activity in your leg, so just the electrical signal sent to your muscle, that your stance leg has very little EMG-- sorry, a lot of EMG.

And your swing has very little EMG with the exception of, right when you begin swing, there's a lot of EMG. And right towards the end of swing, there's a lot of EMG activity. So your muscles are doing seemingly a lot of work at the beginning and end, and your stance leg is doing a lot of work.

But the surprising thing was that inside of phase your muscles almost turn off. So McMahon produced for the world the ballistic walker model. He was at Harvard.

And he basically did some modeling of a three-link pendulum, something like that and showed that, even with a straight leg on your stance leg, if you let your swing leg just go, that you can get very natural gaits coming out, very reasonable kinematic descriptions of what human walking looks like, hence the name ballistic walker. This is a passive phase.

So one of the dominant ideas that came out of that work was the sort of notion that people talk about now as walking by vaulting. If you want a first order approximation of what walking is, they said think about your stance leg as a rod. And you just vault over your stance leg.

And in fact, peoples' center of mass do tend to go up over the course of a stance and then back down. So the dominant theory really was that you could think about this as a stiff leg and then an almost passive swing phase. Now, I remember last year's dynamic walking meeting.

Basically, half the talks were saying walking is not vaulting anymore. It's just in the last year or so people are really, really changing their mind about this, including you guys in Media Lab are talking a lot about the role of compliance in the stance leg in energy storage during walking, which was left out of these initial models. But I think, if you want a 0 order model of walking, thinking about your stance leg as straight is not a bad way to start.

And the other thing that happened because of this work was this idea of passive dynamic walking. So this guy Tad McGeer-- so McMahon's model was just a model of the swing phase.

It didn't talk about stability. It didn't even talk about ground contact. It wasn't sort of a periodic motion. It was saying something about the kinematics of swing phase.

And this guy Tad McGeer, who's actually an aeroengineer, he works on UADs now, came in and turned the robotics world on its head by building a bunch of machines that had no motor, no controller, and walked down a small ramp basically using these ballistic walking kind of ideas. So Tad's a good guy.

Let him describe. I've got a video of him talking about passive walking here.

[VIDEO PLAYBACK]

- This familiar toy is a passive dynamic walker. [INAUDIBLE] all the way. And right at the start, it settles into a steady walking cycle sustained by an entirely passive interaction of gravity and inertia. This machine is also a passive dynamic walker.

For that matter, I may be a passive dynamic walker. As you'll see, our gaits are quite similar.

[INAUDIBLE] with the analysis, and by intuition [INAUDIBLE] exclusively [INAUDIBLE].

[LAUGHTER]

God. Once you establish [INAUDIBLE], the reason the machine failed was a problem called [INAUDIBLE]. So for later experiments, we went to a set of mechanical patches. And the next sequence shows our first trial with those. I guess that worked.

[END VIDEO PLAYBACK]

RUSS TEDRAKE: That's a big deal. That started a lot of research.

AUDIENCE: Is that 1990?

RUSS TEDRAKE: 1990, exactly.

[VIDEO PLAYBACK]

- This familiar toy--

RUSS TEDRAKE: Whoops, sorry.

- [INAUDIBLE] today is that we have all four debouncers [INAUDIBLE].

RUSS TEDRAKE: The knee latches the last time are the debouncers of today.

- [INAUDIBLE]

RUSS TEDRAKE: No motor, no controller, that's just falling down a ramp. That's beautiful. I love this video, because it shows you sort of the honest capabilities of these things. There's a few outtakes.

[LAUGHTER]

That's what it's like working with them. That was pretty benign. It's a pretty robust phenomenon, right? So this is--

[LAUGHTER]

--he's changing the ramp angle quite a bit and getting these really graceful motions out.

This guy walks sideways for some reason. It's really good, yeah?

[END VIDEO PLAYBACK]

You got to realize this was at a time when the walking robots of the world were going like this, moving like that. This guy says, I got a robot with no motor, no controller. Look what it can do, yeah?

Really amazing-- and he was an aerospace engineer, go figure. This is stealing the show. That's the next bit.

So these passive walking ideas have really had a big impact on the world of walking robots. They had it in two waves, I guess. Everybody knew that a purely passive walker could walk down a small slope.

And then people have been working for-- the second part of that video was a video. Finally, someone had put motors on it to make it walk on the flat because it took a surprisingly long time to make that leap. So now, people, I think, don't just think they're a nice party trick, but they actually may be useful for designer of real walking machines.

AUDIENCE: Why [INAUDIBLE]?

RUSS TEDRAKE: Well, because they don't know how to underactuate a control. I think, quite honestly, that's my answer. Yeah.

We don't have any good control ideas. I mean, what you want to do here is just push a little bit of energy into the system in the right way. We don't know how to formulate those control problems when the systems are so non-linear and discontinuous and everything.

Plus, I think the people that are good at building those things maybe are not the same people that are going at control. It takes a special talent to turn enough screws. So you should know that the way that the knee stayed attached-- how do you think the knee state attached in that? Why did the knee not bend during the stance phase?

AUDIENCE: Magnetic or something?

RUSS TEDRAKE: It wasn't magnetic, but that's close. You can imagine, maybe, the curvature of the foot doing it, but that's not it. They're suction cups, yeah? It goes.

And the suction cups don't just stick. You poke a hole in the back of the suction cup. And you tune the leakage of the suction, right? So that at the right time, it pops off and goes like this.

This is not for the weak of heart. There's a lot of tuning that goes into these things. So our most dramatic failure, I think, was we were doing a point foot version of McGeer's walker.

And for the point foot to work and get this nice swing without the curved feet, we had to put a lot of mass on the upper leg, not a lot on the lower leg. So we had these little fiberglass lower legs and big heavy upper legs. And our failure was not just sort of falling down. It was exploding the lower leg into pieces--

[LAUGHTER]

--right, and completely crashing. John, you remember that? Vanessa's walker just kind of-- I'm the one that pushed it, too. I had a reputation in lab for a little while. I had just broken Rick's airplane. And I went around and turned around and broke Vanessa's walker. And they're all on tape.

[LAUGHTER]

So McGeer, apart from building these beautiful machines, gave us this beautiful model to think about the essential dynamics of walking. The ballistic walker told us about the swing phase, but that's actually sort of secondary. What really matters is this passive interaction between inertia and gravity, as Tad said.

And it turns out you can understand that if you just think about this little system called the rimless wheel. So I'm going to put it on a slope of angle gamma.

The model is going to be a simple pendulum with a mass at the end, but a massless leg. We'll define theta to be this, the angle from vertical. Now, I sort of despise the fact that my thetas-- I'm not using the right-hand rule here.

But otherwise, you end up walking backwards with negative velocity. So this is the way it is. So it's almost like the simple pendulum except my coordinate system's from the top and theta's sort of reversed.

And in addition to this simple pendulum, I've got a few extra legs that are all massless. And they're separated by 2 alpha, an angle of 2 alpha.

There's actually reasonably good pictures of this, cartoons of this, in the notes. So you can think about this as a bicycle wheel where somebody took the rim off and just left the spokes. So what happens if you take this bicycle wheel and you put it at the top of a small ramp and you give it a push?

AUDIENCE: It falls over.

RUSS TEDRAKE: It falls over. OK. Let's say it's constrained to be in the plane. Yeah, that was a fair answer. So we made an additional assumption that it stays upright. Then what happens?

AUDIENCE: It starts rolling down.

RUSS TEDRAKE: Starts rolling down-- OK. Is it going to roll faster and faster forever?

AUDIENCE: No.

RUSS TEDRAKE: At some point, losses are going to catch up with it. In the bicycle wheel, you might think of it as rolling friction. In this model, we're going to do it by just a bit of impact, which are very real.

But that's actually the only thing we need. We're not going to model any damping in the system. So we can really think about this system as-- let me list all my assumptions carefully here.

We're going to assume that the foot, so to speak, the stance foot is a pin joint. So I'm going to artificially assume that that foot doesn't come off the ground. As soon as it catches the ground, it turns into a pin joint on the ground.

Instantaneous transfer for support-- which means, as soon as this leg comes around, hits the ground, it becomes the pin joint. And this guy is free. I only think about one ground contact at a time. I assume that is an instantaneous transfer support.

The limit of standing on two legs is just model that's switching back and forth really fast between those two. And it works out OK. And then I'm going to assume that the collisions with the ground are inelastic and impulsive.

Impulsive means instantaneous. Inelastic means that all energy going into the ground is lost to the ground. There's no bouncing. And I need that, so to make sure that there's always exactly one foot on the ground in this model.

As soon as the thing bounces, we built rimless wheels with two wheels. That way they don't fall down in practice, just like McGeer's biped had four legs. They call it a four-legged biped.

[LAUGHTER]

But that way you don't have to worry about falling over sideways. And on the real rimless wheel, you put it on a ramp. It bounces like crazy. So this is just a model, but it actually captures the essential dynamics.

OK. So now, you see why it was good to know everything we know about pendula. How is this thing going to work? I'll show you the answer real quick.

So here's my rimless wheel. That was a simulation. So I'm going to start it. Every time I hit Go, it's going to start it with some random initial velocity forward uphill or downhill. You name it.

That's going up hill, just enough energy, turned around and came back down the hill. Start speeding up a little bit-- but actually quickly settles into a stable forward rolling speed where the losses from collision exactly balance the energy that's gained from converting potential to kinetic as you go down the hill.

That is the essential idea in walking, in certainly passive walking, the conversion of potential energy and the balance with dissipation in the ground. So this is starting with a slow initial speed. And it speeds up a little bit, goes to that rolling speed.

You started really fast. It actually slows down pretty quick and goes to that same rolling speed. Can you see where I'm going with this?

We've got a stable limit cycle. It's periodic, clearly. It's not going to be stable in the trajectory sense, but it's going to be stable in the limit cycle since.

Just I want to see the one other thing that can happen. The cool thing about simulating rimless wheels is you get all these aliasing effects and stuff, right? So if it's going uphill too fast, it looks like it's going downhill. There you go, uphill, uphill. OK, see what happens.

Yeah, OK. So that one just ran out of energy and stayed still. That's still my instantaneous transfer of support. The simulation is going da, da, da, da, da, da, changing ground contacts infinitely fast.

There are serious simulation people that say that's the right way to handle even-- some say that you should simulate your rigid body-- you should simulate a pin joint by constantly modeling impacts at the joints. This guy Brian Mirtich, who was at MERL for a while, did some pretty convincing work in that world.

So that's the answer we're going to get. Now, can we see how that works on the phase portrait? If I look at theta and theta dot, my robot here, there's a couple key angles.

Theta is only going to live between-- so it turns out to be gamma minus alpha and gamma plus alpha. See if you can work that out for yourself. So this angle is 2 alpha.

When this leg hits the ground, that angle has to be alpha plus gamma. And in the other direction, it's gamma minus alpha. Those are the conditions where this leg is about to change into a different coordinate system.

And I forgot to say, when this foot changes and it hits the ground, I quickly redefine theta to be the theta around the new pin joint. I just reset my coordinate system. So there's a couple of key places in this plot.

Somewhere there's some small gamma, and that bisects two very important lines here, which are theta equals gamma minus alpha and theta equals gamma plus alpha. My coordinate system is 0 around the unstable fixed point. That's what happened when I flipped my coordinate system to the top.

So if I want to superimpose the phase portrait of the pendulum, I said 0 is going to be the unstable fixed point. So I get the i is going out this way. That's the homoclinic orbits if you remember.

And the smaller orbits come in and do this. And the ones up here do this. Does anybody have the foresight to sort of jump ahead and tell me where my limit cycle is going to live on my picture here?

AUDIENCE: Above.

RUSS TEDRAKE: Above, OK. What's it going to do?

AUDIENCE: It's going to withdraw from the same place in each [INAUDIBLE].

RUSS TEDRAKE: Awesome, very good. Yeah, very good.

OK. So here's everything we just saw in the rimless wheel. If I'm rolling fast, if I start with some initial velocity rolling fast, say, I'm high up on this line, then what happens? I'm going to ride this guy over here.

It turns out the collision model, I'll tell you the collision model in a second. But the inelastic collision at the base essentially takes out a fraction of your velocity at every step. It's a linear fraction of your velocity at every step.

So I take my coordinates over to here. And then it's going to reset to a velocity over here, but with some fraction of that velocity dissipated. And I get a new trajectory like this.

It comes over to something like this. And at some point, that fraction exactly balances. So because this thing is not symmetric, when I start here, I end up a little bit higher with a little bit more velocity at the end than when I started.

And I lose a little bit of velocity in my impact. And at some point, those things exactly balance. And I end up with a stable limit cycle that looks like that, right?

What else can happen? If I start with too little energy, the whole thing runs out of energy and stands still. What's the critical energy level?

AUDIENCE: Homoclinic orbit?

RUSS TEDRAKE: The homoclinic orbit, right? If I'm just above the homoclinic orbit, it actually speeds up and goes to that same nominal limit cycle. But if I'm just below the homoclinic orbit, then I never actually make it across to this thing.

I just end up going, bonk, over to here. And then it actually comes in-- if you have to think about it, trust me-- like this bonk, like this, and just ends up running out of energy and standing still.

AUDIENCE: Could you start just above it, but then the collision knocks it down [INAUDIBLE]?

RUSS TEDRAKE: It depends on the parameters. But in the parameter regime I'm in, which I think is the standard one you want to think about, in order for it to have some stable-- so basically, what you're talking about is a parameter regime where you have no stable fixed point, no stable rolling limit cycle.

AUDIENCE: Assuming your velocity--

RUSS TEDRAKE: If I'm just epsilon above here and I go down, then I'm only going to stand still.

AUDIENCE: If your collision model was that you always lose a particular fraction of your velocity?

RUSS TEDRAKE: Yes.

AUDIENCE: OK.

RUSS TEDRAKE: So in the case where we have a rolling fixed point, it has to be that epsilon above here actually gets you the other thing. The one maybe you're thinking about, too, is that there's a place over here where, if I start with this energy, I actually come out with a little less energy. And I can actually then go and get caught in either this standing fixed point, or I can transition over into this fixed point and go up and get caught.

OK. There's an easier way to see all this. I want you to see it in the phase portrait because we thought a lot about it. The easier way to see what all this stuff is by defining my surface of section and looking at the Poincaré map. So let's do that.

Now, in the rimless wheel, because it can be going in either direction, I'm going to define a little bit of odd surface of section. I'm going to define it-- it sure feels like I got a lot of colors, but, I don't know, some sort of red and purple mix.

Let me just make it really thick here. Good lord. Let me define this to be my surface of section and this also to be part of my surface of section.

So any time I have an impact with the ground, I'm going to use that as my return map.

AUDIENCE: So is it just before or just after?

RUSS TEDRAKE: I do just after, but it doesn't matter. As long as you're consistent, it doesn't matter. That's a little bit better.

Here's the analytical return map of the rimless wheel. I should say the rimless wheel is the one walking system that we actually understand completely. And I mean, when I talk about these Poincaré maps, I can write them down analytically. Because I can tell you, if I'm in this velocity, where am I going to end up because I just have to do a first integral on the pendulum dynamics. It's energy conserving. We're all happy.

So I can tell you everything about the rimless wheel. I could tell you about where it's fixed points lie. The gray is actually the basins of attraction of the fixed points. I can tell you everything.

There is no other walking model that I know of that I can say that about. This is what you should aspire to get with all of your walking machines. So the red line is the line of slope 1. The blue line is the analytical solution to the first return map with the surface of section defined the way I did.

This is post-collision, all right? So it turns out my lines just got thick enough that it's a little hard to see, but there's a blue line here. It says, if I'm going too fast, I'm slowing down.

And it'll do its little staircase right down to this black fixed point just above this green line, which is the rolling fixed point on the Poincaré map. This is theta dot at the n-th step versus theta dot at the n plus 1-th step. So there's a rolling fixed point.

And it's actually stable. The blue line goes just above the red line before it goes down. There's also a standing fixed point with a velocity at 0, which is the standing on two legs oscillating back and forth. That one's stable, too.

And then when you're rolling backwards uphill, it always says I'm speeding up. Every time when I'm rolling up the hill, this is negative initial velocities. Every time, I'm going to have a little bit more forward velocity on every step.

But because this thing jumps discontinuously in that staircase fashion, it's actually a little non-trivial about whether it's going to end up in the rolling fixed point or the standing fixed point. Those gray stripes are actually the basin of attraction of the rolling fixed point.

So it turns out, if you roll uphill and you stop with just enough energy-- you don't quite get to the top, and then you roll back down-- then you go into the other fixed point. But if you end up at the top of the hill pretty much losing energy like this, then you're going to stop. But I can tell you everything about this return map.

What's the green dashed lines? The green dashed lines are places where my return map are ill-defined. What could that possibly be?

AUDIENCE: You need balance.

RUSS TEDRAKE: Yes, very good, right? There is some initial velocity for which I never return to my return map. What's that? It's if I'm on the homoclinic orbit, right?

It never happens practice, no big deal. But there is a set of measure 0, if you start it with just the wrong velocity, it'll go up and sit like this and never come back. So those are ill-defined points in the Poincaré map, but everything else we completely know.

AUDIENCE: [INAUDIBLE]

RUSS TEDRAKE: Sorry. What's that?

AUDIENCE: We said, like, if you [INAUDIBLE] a velocity of 0, then it would stand still there.

RUSS TEDRAKE: Yes?

AUDIENCE: Why would there be that line?

RUSS TEDRAKE: Oh. Why is there a green line on 0?

AUDIENCE: It's on one stripe.

AUDIENCE: It's a green line at theta 0, not theta dot 0.

RUSS TEDRAKE: It's theta dot 0. So why do I have to draw a green line there? I don't know why I have to draw a green line there. I don't remember that green line being there.

AUDIENCE: [INAUDIBLE]

RUSS TEDRAKE: Yeah, but I don't think I should have drawn-- I think that's actually just a MATLAB typo. There's no reason why that should be there. I don't know why that's there.

It's not in the notes. There's no reason why that should be there. I don't know why that's there. Yeah, John?

AUDIENCE: Homoclinic orbit doesn't separate the rolling ones on the right side of the upper right and the base plot. Your homoclinic orbit is the boundary on the left side, but not on the right side, right? This is a--

RUSS TEDRAKE: That's not true.

AUDIENCE: The ones on the right side roll when they go back. But they're inside, but they bounce across to the left. [INAUDIBLE], right?

RUSS TEDRAKE: OK, good. So there's actually green lines every fraction of the way. Is that what you're asking about?

AUDIENCE: I was actually--

RUSS TEDRAKE: Even the first one, this one?

AUDIENCE: Because the homoclinic orbit is not the boundary or what marks the stable rolling gates in the [INAUDIBLE] plot, right? It is on left side maybe, but on the right side you can be inside the homoclinic orbit.

RUSS TEDRAKE: Oh, that's correct. That's correct. So I can end up here with enough energy to transfer over. That's right.

AUDIENCE: [INAUDIBLE]

RUSS TEDRAKE: Only here is at the--

AUDIENCE: Yeah.

RUSS TEDRAKE: That's good. Yes, that's true. This is the only place where it's the defining part. The transition across from over here is more complicated, which is why we end up with these trajectories, which get started up going the other way. Yeah.

And it's also the case that there's actually stripes of these green lines going up. At every boundary between the gray and the white, there's a place where I could have rolled 10 steps and ended up balanced perfectly and then go back. So I skipped over the impact dynamics. The impact dynamics are good to know, but we can avoid them.

But just to give you the intuition, you can define the energy loss at impact by just looking at this rimless wheel immediately before the impact here. And it's got some momentum that it's moving along this pin joint. So it's got to have some momentum like that.

Immediately after the collision, it's going to have a momentum that's orthogonal to this new pin joint. So any energy that's going into the ground is lost. So if you model an angular momentum conservation around this point, the angular momentum around this, which is the component of this, here remains. And the other part is lost.

It turns out that, in the rimless wheel, you get that as a simple mapping saying, theta dot after a collision, if I use that notation, turns out to be cosine of 2 alpha times theta dot just before the collision. It turns out to be a trivial mapping.

So you now know the essential dynamics of walking. This is actually most of what you need to know if you think about a walking robot. The dominant source of energy loss is not friction in the joints.

So that's pretty small. I mean, you could make a robot with friction in the joints. But the dominant dynamics of our walking are dominated by impact at the ground even for human walking, right?

So most people think that the places you spend the most energy to recover from the ground collision are push off with the toe. And then you're mostly passive through. Ernesto, correct me if the theories have changed. You're mostly passive through.

And then there's one other time where you spend a lot of energy, not a lot, but some energy. We actually decelerate your leg before knee strike maybe just to protect your kneecaps or maybe just to not walk like this or something. I don't know. But you actually spend a little bit of energy doing negative work to slow down that swing leg before it collides with the kneecap.

And that's it. And the collision dominates so much to the point where, if you want to make an energy efficient robot, Andy Ruina says, you can make it four times more efficient if you put this push off just before the collision. Because if you're coming down and you suddenly use your push off to redirect your energy just before that collision, you can reduce the velocity at collision. And he says that makes a four time improvement in the energy efficiency of your robot.

So they've been trying to build walking robots that sense the ground just before by having a little limit switch sticking down in front of the foot and push off right before ground contact. That collision dynamics, what the rimless wheel tells us about, is the dominant essential dynamics of walking. It's the only loss in the system, but it's enough to get us to stable limit cycle out.

The mathematicians like this. Phil Holmes like this because he was surprised initially that and that these piecewise-- this is a energy conserving system inside each cycle. It's a piecewise holonomic system they call it. And it can have stable limit cycles.

So people have gone further than this now. We can do a little bit better than a rimless wheel. If you understand the rimless wheel and now you want to make it a walking robot, all you have to do is take away the rigid joined at the hip, make it a pin joint.

To model the swing leg dynamics, you can add some smaller masses in the leg. So it's now exactly the Acrobot dynamics. You can imagine putting a torque at the hip. And that's called the compass gait model for obvious reasons.

And wouldn't you know it? If you take a compass gait, put it on top of a small hill, give it a push, only losses come from collisions with the ground. Otherwise, it's a, even if I have no torque at the hip, passive stance phase. This thing walks stably down a small hill.

AUDIENCE: Are you wanting the legs to swing free between [INAUDIBLE]?

RUSS TEDRAKE: Everything is just 0 torque. The only trick I'm playing is that I'm ignoring the foot scuff here, which, if you want to build the robot, you've got to build a little retracting toe to make your leg a little bit shorter when it goes through. But, otherwise, you can do this. And we built these.

Now, this guy has the rimless wheel actually. I showed you the basins of attraction. The two fixed points, the rolling and the standing together, are globally attractive except for those sets of measure 0 where you end up standing on your toes.

That's not at all true of the compass gait. The compass gait's pretty frail. You have to find initial conditions that put it into this walking gait. I can show you many simulations of it falling down catastrophically.

But within some initial conditions, you get a nice stable limit cycle out. And not surprisingly, you can go farther. Put one more set of links in there.

This is point feet walking with knees. We modeled a collision at the kneecap and with the ground, but the dynamics are the same. The limit cycles that you see in these guys look a little bit more appealing.

AUDIENCE: [INAUDIBLE] of the knee [INAUDIBLE].

RUSS TEDRAKE: It does not work if the knees have-- you need something that keeps it a stance leg. I mean, you might get a totally different gait out. But that could be--

AUDIENCE: Well, I meant just during the swing. But I guess you have to--

RUSS TEDRAKE: Somehow you have to end up with a straight leg. So if it went past and came back, if you were exceptionally lucky in that regard, you might do it. But kneecaps are a good thing, yeah? You'd feel a little silly walking down the street without kneecaps.

So the dynamics of this, if you sort of project it onto a single plane, you get this nice long limit cycle of the stance leg. You got an impact, which instantaneously decreases your velocity. Then you get your stance leg cycle which is actually basically a snapshot of your rimless wheel cycle.

And then you get another discrete impact which loses energy on the other side. And you get this nice limit cycle for the compass gait. This is the theta of leg one.

And it's the stance leg here, and it's the swing leg here. And trajectories the neighbor this converge to it just like you'd expect. The kneed walker has another collision that happens somewhere in the middle of your swing, but it all works out.

OK, so I've given you enough sort of, I think, to understand the basic limit cycle dynamics and the basic stability property of these things. Next time, we'll talk a bit about how you'd actually apply some of our previous methods, for instance, to this now that we've got a limit cycle and you've got hybrid dynamics with these switching if you want to do things like compute the gradients to do optimization. And so you have to be a little careful about those impacts and how they affect things, but it's not too hard.

And we'll tell you a bit about the state of the art and walking control. They're all sort of based on this model. And then Thursday's the midterm. I promise to have a practice exam for you. It might not be tonight, unfortunately. But it'll be very soon, tomorrow or early Saturday morning, I promise.

Yeah. OK. And feel free to ask me any questions. Some of you are finding me to run project ideas by. That's a great idea.

If you want to talk about your project, let me know. Let John know. See you next time.