WEBVTT 00:00:00.000 --> 00:00:02.520 The following content is provided under a Creative 00:00:02.520 --> 00:00:03.970 Commons license. 00:00:03.970 --> 00:00:06.360 Your support will help MIT OpenCourseWare 00:00:06.360 --> 00:00:10.660 continue to offer high quality educational resources for free. 00:00:10.660 --> 00:00:13.350 To make a donation or view additional materials 00:00:13.350 --> 00:00:17.190 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:17.190 --> 00:00:21.698 at ocw.mit.edu 00:00:21.698 --> 00:00:23.240 RUSS TEDRAKE: Today we're going to do 00:00:23.240 --> 00:00:26.390 sort of the second part of our discussion on walking. 00:00:29.720 --> 00:00:33.080 I want to start with what sounds like a little bit 00:00:33.080 --> 00:00:38.120 of a remedial question in some ways, 00:00:38.120 --> 00:00:40.190 in numerical analysis of these walking kids. 00:00:40.190 --> 00:00:42.770 Because it forces us to be more careful about talking 00:00:42.770 --> 00:00:45.420 about the dynamics through collisions and on Poincare 00:00:45.420 --> 00:00:45.920 maps. 00:00:45.920 --> 00:00:47.000 OK. 00:00:47.000 --> 00:00:50.450 So I actually want to start the discussion 00:00:50.450 --> 00:00:53.900 by just asking given we've defined a walking 00:00:53.900 --> 00:00:57.860 robot with some stance dynamics, and some sort of collision 00:00:57.860 --> 00:01:00.200 dynamics, like we did for the rimless wheel 00:01:00.200 --> 00:01:03.834 and the compass gait and the neat compass gait, quickly. 00:01:03.834 --> 00:01:05.459 I just want to ask the simple question. 00:01:05.459 --> 00:01:08.270 Can you find the stable limits cycles or even 00:01:08.270 --> 00:01:12.130 any periodic gaits of the system? 00:01:12.130 --> 00:01:13.640 OK. 00:01:13.640 --> 00:01:26.180 So the first question, I want to find equilibrium, 00:01:26.180 --> 00:01:28.850 fixed points of the Poincare map. 00:01:56.280 --> 00:01:56.960 OK. 00:01:56.960 --> 00:02:02.570 So remember we talked about Poincare analysis 00:02:02.570 --> 00:02:04.430 of limited cycle behaviors last time. 00:02:04.430 --> 00:02:08.720 And we said that this continuous time, stable oscillation 00:02:08.720 --> 00:02:12.590 could be nicely characterized as a discrete time, fixed point 00:02:12.590 --> 00:02:13.730 analysis. 00:02:13.730 --> 00:02:17.660 If we're able to, for instance, for the van der Pol 00:02:17.660 --> 00:02:20.180 oscillator-- 00:02:20.180 --> 00:02:24.110 and call that x the next dot on the van der Pol oscillator. 00:02:24.110 --> 00:02:26.870 If we're able to design some surface of section, 00:02:26.870 --> 00:02:29.030 and just look at the state of the system every time 00:02:29.030 --> 00:02:32.060 it goes through that surface of section, 00:02:32.060 --> 00:02:34.880 then I can build a Poincare map. 00:02:34.880 --> 00:02:38.220 On the Poincare, I've got a state x, 00:02:38.220 --> 00:02:46.280 which is just some potentially nonlinear mapping from the nth 00:02:46.280 --> 00:02:49.250 crossing of this map to the n plus 1 crossing 00:02:49.250 --> 00:02:54.640 of the mapping of the surface of section. 00:02:54.640 --> 00:03:02.440 So given I have some sort of bisecting surface here, 00:03:02.440 --> 00:03:05.740 what I'd like to do is now find the fixed points 00:03:05.740 --> 00:03:22.030 of that solution, which means xp star is just p of x p star. 00:03:22.030 --> 00:03:23.680 Simple enough. 00:03:23.680 --> 00:03:26.920 And that should tell me where I've got a fixed point, 00:03:26.920 --> 00:03:30.610 and, therefore, where I have a limit cycle. 00:03:30.610 --> 00:03:32.440 It doesn't actually imply necessarily 00:03:32.440 --> 00:03:34.883 that limit cycle is stable. 00:03:34.883 --> 00:03:36.550 If it was stable, then you could imagine 00:03:36.550 --> 00:03:37.750 finding the fixed point, for instance, 00:03:37.750 --> 00:03:40.250 of the van der Pol oscillator just by simulating the van der 00:03:40.250 --> 00:03:41.740 Pol oscillator for enough time. 00:03:41.740 --> 00:03:43.960 I'll eventually go and do the work for you 00:03:43.960 --> 00:03:46.300 and find its way to the stable fixed point 00:03:46.300 --> 00:03:47.800 if you just run along long enough. 00:03:47.800 --> 00:03:50.440 But if it's unstable, that's not as easy to do. 00:03:50.440 --> 00:03:52.660 And for systems like the compass gait, 00:03:52.660 --> 00:03:54.610 it turns out the basin of attraction 00:03:54.610 --> 00:03:58.185 is sort of small enough that you could spend a long time trying 00:03:58.185 --> 00:03:59.560 to find initial conditions, which 00:03:59.560 --> 00:04:02.480 happened to simulate their way into the stable fixed point. 00:04:02.480 --> 00:04:03.220 OK. 00:04:03.220 --> 00:04:05.345 So you want to do better and get some better tools. 00:04:08.170 --> 00:04:11.200 Given what we've talked about in other parts of the class, 00:04:11.200 --> 00:04:13.960 if you want to define the roots of this-- 00:04:13.960 --> 00:04:17.062 the fixed points of the system, how would you do it? 00:04:17.062 --> 00:04:22.200 I just slipped a little bit and gave you half the answer there. 00:04:30.610 --> 00:04:32.020 How would you do it? 00:04:32.020 --> 00:04:33.850 I've got some big, complicated function p. 00:04:33.850 --> 00:04:35.183 I want to find the fixed points. 00:04:39.170 --> 00:04:40.670 Close. 00:04:40.670 --> 00:04:45.020 So I want to find the roots of some function of xp. 00:04:45.020 --> 00:04:47.090 So it's equivalent to finding the roots-- 00:04:57.140 --> 00:05:02.600 the roots of pxp star minus xp star. 00:05:09.460 --> 00:05:12.520 So if I can evaluate p through simulation or whatever, 00:05:12.520 --> 00:05:16.202 then I can evaluate this. 00:05:16.202 --> 00:05:18.160 And I could just hand it to MATLAB or something 00:05:18.160 --> 00:05:23.890 and say, find me the roots of this nonlinear function. 00:05:23.890 --> 00:05:24.910 OK. 00:05:24.910 --> 00:05:26.170 And that's easy enough. 00:05:26.170 --> 00:05:28.150 We talked about the Newton method 00:05:28.150 --> 00:05:30.790 for finding roots quickly, and we 00:05:30.790 --> 00:05:34.300 talked about the constraint solvers that SNOPT and the like 00:05:34.300 --> 00:05:35.890 uses. 00:05:35.890 --> 00:05:38.445 OK. 00:05:38.445 --> 00:05:40.570 So in fact, we could just hand this to SNOPT as is, 00:05:40.570 --> 00:05:41.530 and ask it to do its thing. 00:05:41.530 --> 00:05:43.072 And it would do numerical derivatives 00:05:43.072 --> 00:05:48.660 and would find its way to some zero of that function. 00:05:48.660 --> 00:05:50.560 We're going to be a lot better shape, though, 00:05:50.560 --> 00:05:53.675 if we can compute the gradients of that analytically. 00:05:53.675 --> 00:05:54.550 And hand it to SNOPT. 00:05:54.550 --> 00:05:57.560 It would take a lot fewer trials. 00:05:57.560 --> 00:06:00.310 And I want to sort of do that exercise quickly, 00:06:00.310 --> 00:06:02.230 computing the gradients of the Poincare map. 00:06:02.230 --> 00:06:04.690 Because it's going to be broadly useful in control. 00:06:04.690 --> 00:06:08.440 And because it forces us to really think about how 00:06:08.440 --> 00:06:09.950 we define that Poincare map. 00:06:09.950 --> 00:06:10.450 OK. 00:06:14.860 --> 00:06:18.070 So what we'd like to do, we just say 00:06:18.070 --> 00:06:27.520 it's easier if we have partial p partial x 00:06:27.520 --> 00:06:29.260 available for the optimization. 00:06:54.920 --> 00:06:55.420 OK. 00:06:55.420 --> 00:06:56.878 Just think ahead for a second here. 00:06:56.878 --> 00:07:02.180 How would you compute if I would just ask you on the street, 00:07:02.180 --> 00:07:04.930 I've got a Poincare map, how would I compute the gradients. 00:07:04.930 --> 00:07:06.310 What would you think? 00:07:06.310 --> 00:07:08.620 What would you start trying to do? 00:07:11.830 --> 00:07:13.120 Yeah. 00:07:13.120 --> 00:07:19.810 So where we analyze the perturbations before? 00:07:19.810 --> 00:07:21.990 What kind of tools go with that idea? 00:07:29.030 --> 00:07:29.530 Yeah. 00:07:29.530 --> 00:07:34.155 So again around any given point which defines the trajectory, 00:07:34.155 --> 00:07:35.530 we could do backprop through time 00:07:35.530 --> 00:07:37.367 if we cared about a cost function. 00:07:37.367 --> 00:07:38.950 But remember the trick behind backprop 00:07:38.950 --> 00:07:40.640 through time, computing those gradients, 00:07:40.640 --> 00:07:41.710 the reason that was so efficient was 00:07:41.710 --> 00:07:44.440 because we were trying to find the gradient with respect 00:07:44.440 --> 00:07:46.040 to a scalar output. 00:07:46.040 --> 00:07:48.160 So backprop through time did a forward pass, 00:07:48.160 --> 00:07:50.620 and then connected it to that scalar cost function output. 00:07:50.620 --> 00:07:54.520 And then backed up a minimal state vector. 00:07:54.520 --> 00:07:56.140 The forward version we talked about 00:07:56.140 --> 00:08:01.720 was just computing the gradients marching forward. 00:08:01.720 --> 00:08:05.590 And I called that real time recurrent learning, RTRL. 00:08:05.590 --> 00:08:08.132 And that's actually sort of 80% of what you need 00:08:08.132 --> 00:08:09.340 to compute the gradient here. 00:08:09.340 --> 00:08:11.270 Is there something I did wrong? 00:08:11.270 --> 00:08:11.770 No. 00:08:15.350 --> 00:08:16.010 OK. 00:08:16.010 --> 00:08:18.980 So RTRL, which we've talked about before 00:08:18.980 --> 00:08:23.390 and I'll say again real quickly here, is a lot of the answer. 00:08:23.390 --> 00:08:25.760 But there's a subtlety that I want you to see. 00:08:25.760 --> 00:08:26.690 OK. 00:08:26.690 --> 00:08:34.549 So let's be a little careful about how we go from xp of m 00:08:34.549 --> 00:08:37.820 over to xp n plus 1. 00:08:42.740 --> 00:08:45.277 That mapping is sort of a complicated-- 00:08:48.260 --> 00:08:50.720 there's a transition from this discrete time system 00:08:50.720 --> 00:08:52.730 to the continuous time dynamics, a simulation 00:08:52.730 --> 00:08:54.770 of those continuous time dynamics. 00:08:54.770 --> 00:08:56.870 Potentially, an impact map at the end. 00:08:56.870 --> 00:08:58.670 We talked about the rimless wheel 00:08:58.670 --> 00:09:01.040 doing the Poincare map right at the impact. 00:09:01.040 --> 00:09:02.540 So let's do that carefully. 00:09:02.540 --> 00:09:12.740 So let me define xp at m as being equivalent to, 00:09:12.740 --> 00:09:16.880 in continuous time, x, the original vector-- 00:09:16.880 --> 00:09:21.170 I used a p to denote the discrete time, Poincare thing-- 00:09:21.170 --> 00:09:24.890 at the time of collision of the nth collision. 00:09:28.820 --> 00:09:31.130 In the walking case, we always did our Poincare map 00:09:31.130 --> 00:09:33.800 at the same time as the collision. 00:09:33.800 --> 00:09:35.630 And more generally, you could just 00:09:35.630 --> 00:09:37.130 think of this as sort of a collision 00:09:37.130 --> 00:09:40.850 with a surface of section. 00:09:40.850 --> 00:09:41.960 OK. 00:09:41.960 --> 00:09:46.550 In the walking case, because the collision has some dynamics, 00:09:46.550 --> 00:09:49.160 we want to be explicit about whether it's 00:09:49.160 --> 00:09:53.790 before or after those dynamics, before or after the collision. 00:09:53.790 --> 00:09:59.480 So I'm going to call our discrete time 00:09:59.480 --> 00:10:02.810 state of our system to be the state of the system 00:10:02.810 --> 00:10:05.290 immediately after I process that collision dynamics. 00:10:05.290 --> 00:10:06.665 So I'm going to put a plus there. 00:10:06.665 --> 00:10:08.480 Minus would be just before the collision. 00:10:08.480 --> 00:10:08.780 Yeah. 00:10:08.780 --> 00:10:10.530 They're both at the same time, but there's 00:10:10.530 --> 00:10:13.380 an instantaneous dynamics. 00:10:13.380 --> 00:10:13.950 OK. 00:10:13.950 --> 00:10:15.000 So I have this. 00:10:15.000 --> 00:10:18.370 I know how to go from xp to a continuous time thing. 00:10:18.370 --> 00:10:24.240 And then I know how to go from this to march that forward. 00:10:24.240 --> 00:10:30.690 If I know when the next collision is, 00:10:30.690 --> 00:10:33.060 then I know that had better just be 00:10:33.060 --> 00:10:45.390 x of tc plus n plus the integral from tc plus n 00:10:45.390 --> 00:10:48.970 to tc minus n plus 1. 00:10:48.970 --> 00:10:52.650 Let's just say my dynamics are x dot equals f of x. 00:10:52.650 --> 00:10:55.230 I forget about control for a second just to keep it simpler. 00:11:04.050 --> 00:11:07.020 Once I go from my discrete time thing 00:11:07.020 --> 00:11:08.550 to a continuous time thing, I just 00:11:08.550 --> 00:11:11.460 integrate my equations of motion forward 00:11:11.460 --> 00:11:19.950 until I collide with the surface of section again. 00:11:19.950 --> 00:11:22.663 And I want tc defined somehow. 00:11:22.663 --> 00:11:24.705 I want to somehow define that surface of section. 00:11:29.180 --> 00:11:36.300 And to define that surface of section and that collision 00:11:36.300 --> 00:11:38.040 time, I'm going to define-- 00:11:48.090 --> 00:11:52.560 or more generally a collision-- 00:11:52.560 --> 00:12:02.150 as some function that equals zero. 00:12:10.340 --> 00:12:14.310 There's some manifold of states that I 00:12:14.310 --> 00:12:17.220 can describe by this function, which will 00:12:17.220 --> 00:12:19.330 define my surface of section. 00:12:21.840 --> 00:12:25.751 So what was it for the van der Pol oscillator? 00:12:28.520 --> 00:12:33.960 I designed my surface of section to be this thing going forward. 00:12:33.960 --> 00:12:36.680 So what the surface of section is 00:12:36.680 --> 00:12:39.740 defined as some function which just happens to be zero here. 00:12:42.320 --> 00:12:45.710 So a perfectly good candidate would be like the distance, 00:12:45.710 --> 00:12:51.920 the signed distance let's say, between your current state 00:12:51.920 --> 00:12:52.865 and that manifold. 00:12:55.520 --> 00:12:56.460 You OK with that? 00:12:56.460 --> 00:12:57.960 I don't think I said that very well. 00:12:57.960 --> 00:12:59.512 But if I want to know-- 00:12:59.512 --> 00:13:01.220 define my surface of section, just define 00:13:01.220 --> 00:13:04.040 a function of state, which is non-zero everywhere 00:13:04.040 --> 00:13:06.890 and zero where the surface I care about is. 00:13:06.890 --> 00:13:07.790 Yeah. 00:13:07.790 --> 00:13:09.623 And that's going to allow me, as you'll see, 00:13:09.623 --> 00:13:13.400 to do things like taking the gradient of knowing 00:13:13.400 --> 00:13:18.870 when I'm going to collide with that surface of section. 00:13:18.870 --> 00:13:23.030 So this could just be for instance 00:13:23.030 --> 00:13:31.400 the signed distance between, let's say, 00:13:31.400 --> 00:13:40.780 the foot of the robot and the ground. 00:13:43.770 --> 00:13:44.270 Yeah. 00:13:49.980 --> 00:13:51.870 So when the robot's foot hits the ground, 00:13:51.870 --> 00:13:54.155 that thing's going to be zero. 00:13:54.155 --> 00:13:55.530 When it's below the ground, it'll 00:13:55.530 --> 00:13:56.490 be some negative distance. 00:13:56.490 --> 00:13:57.865 When it's above the ground, it'll 00:13:57.865 --> 00:13:59.150 be some positive distance. 00:13:59.150 --> 00:14:05.190 OK now that's going to allow me then to define this. 00:14:05.190 --> 00:14:07.538 At a time of collision, this thing 00:14:07.538 --> 00:14:08.830 is going to add better be zero. 00:14:18.098 --> 00:14:19.640 Good so I would I'd want to define it 00:14:19.640 --> 00:14:21.230 in a way that doesn't include that. 00:14:21.230 --> 00:14:23.105 So I could put a-- 00:14:23.105 --> 00:14:24.980 so the robot's foot colliding with the ground 00:14:24.980 --> 00:14:27.022 doesn't have that problem, if you think about it. 00:14:29.130 --> 00:14:33.152 But I do typically put something like a velocity term in it. 00:14:33.152 --> 00:14:34.610 Let me just write down what I would 00:14:34.610 --> 00:14:35.693 use for the rimless wheel. 00:14:35.693 --> 00:14:38.800 Actually, I've got it here. 00:14:38.800 --> 00:14:55.910 So I do phi of x is just sine, the sine function of theta 00:14:55.910 --> 00:15:02.218 dot times theta minus gamma minus alpha. 00:15:02.218 --> 00:15:04.010 So if you don't remember all those symbols, 00:15:04.010 --> 00:15:05.120 that's almost useless. 00:15:05.120 --> 00:15:08.750 But this is saying-- 00:15:08.750 --> 00:15:12.903 the sine of theta dot tells me if the robot's moving forward, 00:15:12.903 --> 00:15:14.570 then I'm going to look for the collision 00:15:14.570 --> 00:15:16.508 with the ground at this angle. 00:15:16.508 --> 00:15:18.050 And if the robot's looking backwards, 00:15:18.050 --> 00:15:19.010 I'm going to actually look at the collision 00:15:19.010 --> 00:15:20.540 with the ground at that angle. 00:15:20.540 --> 00:15:23.040 And that's how I end up getting the surface of section which 00:15:23.040 --> 00:15:25.040 doesn't go-- 00:15:25.040 --> 00:15:30.230 just goes in the top half and bottom half of the thing. 00:15:30.230 --> 00:15:31.730 So you can divide whatever function, 00:15:31.730 --> 00:15:33.350 but you just want it to be not 0 here. 00:15:40.460 --> 00:15:43.980 OK we're getting closer to having this be well defined. 00:15:43.980 --> 00:15:45.590 In fact, this is a very common way 00:15:45.590 --> 00:15:51.480 to define a collision in any hybrid system. 00:15:51.480 --> 00:15:53.957 So systems that are continuous with discrete impacts 00:15:53.957 --> 00:15:55.040 are called hybrid systems. 00:15:57.830 --> 00:15:59.990 If you wanted to simulate a hybrid system in MATLAB 00:15:59.990 --> 00:16:02.240 with ODE, then you would very much 00:16:02.240 --> 00:16:04.880 do that by defining a function, which 00:16:04.880 --> 00:16:07.790 they call a collision function or a collision event function, 00:16:07.790 --> 00:16:09.890 which has got 0's here. 00:16:09.890 --> 00:16:11.270 And you can hand, as your options 00:16:11.270 --> 00:16:14.990 to the ODE solvers in MATLAB, event functions like this. 00:16:14.990 --> 00:16:18.740 And they will very carefully watch a crossing of that event 00:16:18.740 --> 00:16:22.730 function across 0, and pinpoint the time and state at which 00:16:22.730 --> 00:16:25.340 that thing goes across 0. 00:16:25.340 --> 00:16:27.830 So this is a very common notation in hybrid systems 00:16:27.830 --> 00:16:29.720 kind of worlds. 00:16:32.365 --> 00:16:34.550 It turns out if we do care about finding 00:16:34.550 --> 00:16:36.773 this gradient of the Poincare map, 00:16:36.773 --> 00:16:39.440 then this is going to be also be the enabling thing which allows 00:16:39.440 --> 00:16:40.607 us to compute that gradient. 00:16:47.840 --> 00:16:51.180 OK So we have this x of t minus. 00:16:51.180 --> 00:16:55.410 We've gone from just after a collision. 00:16:55.410 --> 00:16:58.060 We've integrated forward over a continuous dynamics. 00:16:58.060 --> 00:17:00.390 Yeah. 00:17:00.390 --> 00:17:02.670 So now we're at the time of the new collision, tc 00:17:02.670 --> 00:17:05.099 minus n plus 1. 00:17:05.099 --> 00:17:09.030 We're going to compute the new dynamics by having 00:17:09.030 --> 00:17:10.775 some collision function. 00:17:10.775 --> 00:17:12.150 In the rimless wheel, it was just 00:17:12.150 --> 00:17:15.030 the thing that took out some energy because 00:17:15.030 --> 00:17:17.910 of the inelastic collision with the ground. 00:17:17.910 --> 00:17:24.069 So the final piece of the puzzle here is x of p n 00:17:24.069 --> 00:17:33.310 plus 1, which is equivalent to x of tc plus n plus 1, 00:17:33.310 --> 00:17:36.820 which is some impact dynamics. 00:17:36.820 --> 00:17:43.720 I'll just call it big f of x of tc minus n plus 1. 00:18:04.810 --> 00:18:08.230 In 90% of cases that I would care about, 00:18:08.230 --> 00:18:10.350 it's only a function of x. 00:18:10.350 --> 00:18:12.350 I like to write in the more general form in case 00:18:12.350 --> 00:18:14.510 you have things like moving obstacles. 00:18:14.510 --> 00:18:15.730 Let's say. 00:18:15.730 --> 00:18:18.880 In the surface of section case for limit cycles, 00:18:18.880 --> 00:18:21.430 I think it'd be pretty unusual to have 00:18:21.430 --> 00:18:22.853 a direct dependence on time. 00:18:22.853 --> 00:18:25.270 But it doesn't make an our derivation any more complicated 00:18:25.270 --> 00:18:26.320 than if you had-- 00:18:26.320 --> 00:18:28.630 you wanted to worry about collisions 00:18:28.630 --> 00:18:31.172 with moving things or something like that, you could do that. 00:18:35.720 --> 00:18:36.220 OK. 00:18:36.220 --> 00:18:40.100 So this is a sufficient recipe now for simulating the Poincare 00:18:40.100 --> 00:18:40.837 map. 00:18:40.837 --> 00:18:42.670 If I want to just evaluate in the simulation 00:18:42.670 --> 00:18:44.620 with the Poincare map, someone says 00:18:44.620 --> 00:18:48.640 I'm at state xp on the nth Poincare map, 00:18:48.640 --> 00:18:51.340 I'm going to turn that into my continuous time thing. 00:18:51.340 --> 00:18:54.310 I'm going to integrate forward my dynamics, 00:18:54.310 --> 00:18:55.600 just like we always do. 00:18:55.600 --> 00:18:59.380 But I'm going to stop at a particular time, tc minus, 00:18:59.380 --> 00:19:03.400 which is defined as the time when it causes phi 00:19:03.400 --> 00:19:06.188 to equals zero. 00:19:06.188 --> 00:19:08.230 And then at that time, in the case of the walking 00:19:08.230 --> 00:19:10.870 robots or any collisions, physical collisions, 00:19:10.870 --> 00:19:15.580 you might have to implement the discrete collision dynamics. 00:19:15.580 --> 00:19:17.800 And that takes you back to the next map. 00:19:24.060 --> 00:19:24.560 Good. 00:19:24.560 --> 00:19:26.890 So how do we take the gradients of that? 00:19:31.220 --> 00:19:35.450 It's almost trivial, almost trivial. 00:19:35.450 --> 00:19:37.840 We know how to take the gradients off of this. 00:19:40.430 --> 00:19:41.990 We'll do it again, just in one line. 00:19:41.990 --> 00:19:46.130 But we know how to just integrate forward 00:19:46.130 --> 00:19:53.330 the gradient dynamics to compute the long term gradients of x. 00:19:56.180 --> 00:19:59.820 The only subtlety is that-- 00:19:59.820 --> 00:20:01.610 so when I'm computing the gradient, 00:20:01.610 --> 00:20:02.900 I'm changing xp a little bit. 00:20:02.900 --> 00:20:05.000 I'm trying to do a sensitivity analysis between x. 00:20:05.000 --> 00:20:06.800 If I change xp a little bit, how is it 00:20:06.800 --> 00:20:09.980 going to change xp n plus 1? 00:20:09.980 --> 00:20:14.510 So the only subtlety is that when I change xp a little bit, 00:20:14.510 --> 00:20:17.563 it can change my collision time. 00:20:17.563 --> 00:20:18.980 So you have to make darn sure when 00:20:18.980 --> 00:20:20.570 you're taking the gradients, that you 00:20:20.570 --> 00:20:23.240 capture the changes in collision time 00:20:23.240 --> 00:20:27.500 that are due to your change in the initial conditions. 00:20:27.500 --> 00:20:31.100 Other than that it's almost exactly the RTRL code 00:20:31.100 --> 00:20:33.330 that we've done before. 00:20:33.330 --> 00:20:37.170 And that turns out to be not so bad, so I want to do it. 00:20:37.170 --> 00:20:37.670 All right. 00:20:37.670 --> 00:20:43.100 So we can just go forward with the chain rule here, so dxp 00:20:43.100 --> 00:20:46.850 of n plus 1 dxp of n. 00:20:51.700 --> 00:20:57.520 Well that's going to be, first of all, the gradient 00:20:57.520 --> 00:21:07.010 of this f, partial capital f partial x and then times 00:21:07.010 --> 00:21:08.135 the gradient of the inside. 00:21:13.610 --> 00:21:18.060 The inside is defined by x at a certain time. 00:21:18.060 --> 00:21:19.880 So this is where that comes in. 00:22:14.490 --> 00:22:14.990 OK. 00:22:19.915 --> 00:22:21.290 So first of all, let me say we're 00:22:21.290 --> 00:22:22.790 doing we're doing this linearization 00:22:22.790 --> 00:22:24.230 around some nominal trajectory. 00:22:24.230 --> 00:22:30.980 We have some xp0 that we're linearizing around. 00:22:30.980 --> 00:22:34.820 That implies we have an xt trajectory 00:22:34.820 --> 00:22:38.300 that we're linearizing, around which I'm calling x0. 00:22:41.030 --> 00:22:45.240 And it implies that there's some nominal time of impact-- 00:22:45.240 --> 00:22:47.240 time of collision that we're linearizing around, 00:22:47.240 --> 00:22:50.090 which I'm calling tc0. 00:22:50.090 --> 00:22:52.670 OK. 00:22:52.670 --> 00:22:55.250 So the obvious gradient here, partial f partial x, 00:22:55.250 --> 00:22:56.150 gets us past-- 00:22:56.150 --> 00:22:58.580 gets up backwards with respect to the impact dynamics. 00:22:58.580 --> 00:23:01.850 And then we have to figure out what the state of x 00:23:01.850 --> 00:23:04.430 is relative to the initial conditions, 00:23:04.430 --> 00:23:07.430 at time evaluated at the original impact time. 00:23:10.250 --> 00:23:15.620 And then the subtlety is that I can also increment or decrement 00:23:15.620 --> 00:23:19.940 the index of the integral, the limits of the integral, which 00:23:19.940 --> 00:23:26.270 has the effect of adding this term again modified 00:23:26.270 --> 00:23:31.050 by that increment in the limit of the integral. 00:23:31.050 --> 00:23:36.020 So I get f of x times that increment. 00:23:36.020 --> 00:23:37.125 OK. 00:23:37.125 --> 00:23:38.375 So let's do this in a picture. 00:23:42.590 --> 00:23:49.220 So let's say I have some thing, some manifold defined by phi 00:23:49.220 --> 00:23:49.940 equals 0. 00:23:53.300 --> 00:23:55.610 And I have some nominal trajectory 00:23:55.610 --> 00:23:57.980 that I took to get there. 00:24:02.800 --> 00:24:09.050 I got to my switching surface. 00:24:09.050 --> 00:24:10.880 I'm calling this x0 of t. 00:24:15.140 --> 00:24:26.030 And this one here is x0 of tc0 plus n, 00:24:26.030 --> 00:24:29.405 to be entirely confusing. 00:24:29.405 --> 00:24:32.450 It just takes a lot to write. 00:24:32.450 --> 00:24:36.320 I could write that equally well as xp0 of m. 00:24:40.250 --> 00:24:42.860 And what I'm trying to figure out 00:24:42.860 --> 00:24:46.042 is what's the state going to be if I'm 00:24:46.042 --> 00:24:47.750 going to simulate my system from slightly 00:24:47.750 --> 00:24:51.950 different initial conditions when it hits the map. 00:24:51.950 --> 00:24:57.350 So using my multi-colored approach here, 00:24:57.350 --> 00:25:04.850 if I take some increment in x0 and I simulate my new dynamics 00:25:04.850 --> 00:25:08.330 forward, then maybe I get something pretty similar. 00:25:08.330 --> 00:25:12.620 But at time-- what my original end of time, 00:25:12.620 --> 00:25:15.557 if I evaluated this integral for the same amount of time, 00:25:15.557 --> 00:25:17.390 there's no reason to expect I would get back 00:25:17.390 --> 00:25:18.473 to that switching surface. 00:25:21.500 --> 00:25:23.810 So this is my modified thing at tc0. 00:25:27.530 --> 00:25:29.180 But what I would figure out is where 00:25:29.180 --> 00:25:33.620 it's going to be the next time it hits to the surface. 00:25:37.200 --> 00:25:41.600 So you can do that by figuring out what this difference is. 00:25:41.600 --> 00:25:51.050 This is partial x of t given the initial conditions at time tc0. 00:25:51.050 --> 00:25:55.580 That's this term here, corresponds to this. 00:25:55.580 --> 00:25:56.750 Yeah. 00:25:56.750 --> 00:26:01.910 But I'd better also add in the dynamics of this thing pushed 00:26:01.910 --> 00:26:06.260 forward by the amount of time necessary to get me back 00:26:06.260 --> 00:26:10.640 to the surface for that collision. 00:26:10.640 --> 00:26:11.540 Yeah. 00:26:11.540 --> 00:26:14.175 STUDENT: So you never take the derivative of the p function? 00:26:14.175 --> 00:26:15.050 RUSS TEDRAKE: You do. 00:26:15.050 --> 00:26:15.660 I'll show you. 00:26:15.660 --> 00:26:17.390 So in order to get this, we're going 00:26:17.390 --> 00:26:19.223 have to take a derivative of the p function. 00:26:21.845 --> 00:26:24.470 I can't tell if people are bored by this, or intrigued by this, 00:26:24.470 --> 00:26:27.810 or don't care. 00:26:27.810 --> 00:26:28.310 OK. 00:26:28.310 --> 00:26:30.740 Does that makes sense? 00:26:30.740 --> 00:26:32.630 Good. 00:26:32.630 --> 00:26:33.170 All right. 00:26:33.170 --> 00:26:34.670 So how do we figure out the change-- 00:26:34.670 --> 00:26:37.255 the increment in time, given the initial conditions? 00:26:44.050 --> 00:26:48.010 Well it's defined based on that phi equaling 0 00:26:48.010 --> 00:26:54.238 is the thing that defines this collision surface. 00:26:54.238 --> 00:26:55.780 So it's going to be used in computing 00:26:55.780 --> 00:26:57.190 that increment in time. 00:26:57.190 --> 00:26:58.660 OK. 00:26:58.660 --> 00:27:04.840 So if phi equals 0, then it better also be the case 00:27:04.840 --> 00:27:12.220 that dphi dxp of n-- 00:27:12.220 --> 00:27:16.120 and let me just write this-- 00:27:16.120 --> 00:27:27.670 tc minus tc n plus 1, x of tc minus n plus 1. 00:27:31.840 --> 00:27:34.840 This is just the derivative of that thing 00:27:34.840 --> 00:27:36.715 based on the changes of in xp. 00:27:40.330 --> 00:27:41.740 I've defined that. 00:27:41.740 --> 00:27:44.530 I'm saying that even though I changed xp, 00:27:44.530 --> 00:27:46.690 phi had better still be 0. 00:27:46.690 --> 00:27:49.970 I'm integrating until I hit that surface. 00:27:49.970 --> 00:27:51.370 So this thing a better equal 0. 00:28:00.240 --> 00:28:08.610 And I can take that derivative with partial phi 00:28:08.610 --> 00:28:11.310 tx, partial x-- 00:29:02.250 --> 00:29:03.780 t evaluated those same places. 00:29:30.680 --> 00:29:31.620 OK. 00:29:31.620 --> 00:29:33.950 I'll save myself from writing that last equation, 00:29:33.950 --> 00:29:39.390 but obviously I can now solve this for dtc dxp. 00:29:43.830 --> 00:29:44.330 Yeah. 00:29:51.260 --> 00:29:55.760 Stepping back just a second, I'm making an increment here. 00:29:55.760 --> 00:29:58.250 I'm defining that, even though I made that increment, 00:29:58.250 --> 00:30:01.010 I still better get to phi equals 0. 00:30:04.040 --> 00:30:07.160 I can look at the change in phi that could potentially occur. 00:30:07.160 --> 00:30:10.460 It's going to also depend on the change in x and the change 00:30:10.460 --> 00:30:13.070 in final time. 00:30:13.070 --> 00:30:15.860 And that allows me to solve for the final time that 00:30:15.860 --> 00:30:17.490 must have made that happen. 00:30:17.490 --> 00:30:21.620 That must have made it so I get back to the switching surface. 00:30:25.140 --> 00:30:25.640 Yeah. 00:30:32.650 --> 00:30:33.150 OK. 00:30:36.370 --> 00:30:38.080 That's really the only thing you need 00:30:38.080 --> 00:30:39.860 to know to make all of your tools 00:30:39.860 --> 00:30:42.850 we've used for loop optimal control, 00:30:42.850 --> 00:30:46.900 for instance, for the acrobat and the cart-pole work 00:30:46.900 --> 00:30:49.030 for the walking systems. 00:30:49.030 --> 00:30:54.550 The additional advantage of thinking about this 00:30:54.550 --> 00:31:01.360 now as on the Poincare map is in some ways we 00:31:01.360 --> 00:31:02.590 could do even easier control. 00:31:06.480 --> 00:31:06.980 OK. 00:31:10.170 --> 00:31:13.260 So what did I just solve for? 00:31:13.260 --> 00:31:17.070 I solved for partial p partial x. 00:31:24.830 --> 00:31:25.470 Absolutely. 00:31:25.470 --> 00:31:25.970 Please. 00:31:29.100 --> 00:31:29.600 Yeah. 00:31:29.600 --> 00:31:31.120 Good I want questions. 00:31:46.840 --> 00:31:48.280 No, not necessarily. 00:31:48.280 --> 00:31:52.630 I mean this to say, so it happens-- 00:31:52.630 --> 00:31:55.960 it happens that oftentimes at the surface of section 00:31:55.960 --> 00:32:01.120 we want to compute a discrete collision dynamics, which 00:32:01.120 --> 00:32:04.448 is just some other function which I'm calling capital f. 00:32:04.448 --> 00:32:06.490 I don't mean that to be directly related to that. 00:32:06.490 --> 00:32:08.440 I can see that coming right after the integral that's 00:32:08.440 --> 00:32:08.980 confusing. 00:32:08.980 --> 00:32:10.647 Want to I call that something different? 00:32:10.647 --> 00:32:13.950 We can call it something different. 00:32:13.950 --> 00:32:14.650 Yeah. 00:32:14.650 --> 00:32:18.440 This is the equation-- 00:32:18.440 --> 00:32:20.008 well in the rimless wheel example, 00:32:20.008 --> 00:32:22.300 it actually says I'm going to change coordinate systems 00:32:22.300 --> 00:32:25.400 back to where my new leg is on the ground. 00:32:25.400 --> 00:32:28.478 So I'm actually going to do a discrete change in theta, 00:32:28.478 --> 00:32:30.520 and I'm going to take away some of the theta dot. 00:32:30.520 --> 00:32:35.180 Because I've lost some energy into the ground And in general, 00:32:35.180 --> 00:32:37.750 when you have collisions in mechanical systems, 00:32:37.750 --> 00:32:39.610 if you model them as impulses, you're 00:32:39.610 --> 00:32:41.335 going to have some function like f. 00:32:49.260 --> 00:32:49.760 All right. 00:33:02.720 --> 00:33:06.900 This is equivalent, right, to being the partial p partial x, 00:33:06.900 --> 00:33:09.150 Everybody sees that? 00:33:09.150 --> 00:33:09.650 OK. 00:33:18.440 --> 00:33:20.390 Not a little bit of change of phi. 00:33:20.390 --> 00:33:23.500 What we're changing, the thing we're changing, is xp. 00:33:39.910 --> 00:33:42.950 STUDENT: And x0 and all the nots are basically 00:33:42.950 --> 00:33:45.978 the initial trajectory we would have taken without changing. 00:33:45.978 --> 00:33:48.020 RUSS TEDRAKE: Yes, because we're doing everything 00:33:48.020 --> 00:33:48.690 as an increment. 00:33:48.690 --> 00:33:50.190 We're doing an incremental analysis. 00:33:50.190 --> 00:33:53.330 So I want to say that the new tc is going 00:33:53.330 --> 00:33:58.130 to be tc0 plus this increment. 00:34:13.310 --> 00:34:16.310 I've defined the problem that way, again. 00:34:16.310 --> 00:34:18.590 So even if I make a small change in xp, 00:34:18.590 --> 00:34:21.100 I still want it to be that a minute collision point-- 00:34:21.100 --> 00:34:22.850 I'm not going to call it a collision point 00:34:22.850 --> 00:34:25.842 until phi equals 0. 00:34:25.842 --> 00:34:27.800 And if phi's going to equal 0 along everything, 00:34:27.800 --> 00:34:30.679 then it certainly equals 0 on an increment in xp. 00:34:43.650 --> 00:34:44.489 Yeah. 00:34:44.489 --> 00:34:46.080 So it's tempting to say-- 00:34:46.080 --> 00:34:49.690 so we've got gradient based calculations flying around 00:34:49.690 --> 00:34:50.190 here. 00:34:50.190 --> 00:34:51.565 We've been doing a lot with that. 00:34:51.565 --> 00:34:53.429 I think they're pretty powerful. 00:34:53.429 --> 00:34:55.679 It's tempting to say that if I have a discontinuity, 00:34:55.679 --> 00:34:57.360 then it breaks all my gradient calculations 00:34:57.360 --> 00:34:58.590 it doesn't break them, you just have 00:34:58.590 --> 00:34:59.840 to be more careful about them. 00:34:59.840 --> 00:35:00.430 Yeah. 00:35:00.430 --> 00:35:00.930 OK. 00:35:00.930 --> 00:35:03.910 So we can take a gradient through the impact of a walking 00:35:03.910 --> 00:35:04.410 robot. 00:35:04.410 --> 00:35:05.950 No problem. 00:35:05.950 --> 00:35:07.920 You just have to use this. 00:35:07.920 --> 00:35:08.820 OK. 00:35:08.820 --> 00:35:12.822 This will be the same if you're doing a ping pong 00:35:12.822 --> 00:35:14.030 robot or something like that. 00:35:14.030 --> 00:35:16.830 Anything with collisions, a mobile robot 00:35:16.830 --> 00:35:21.480 that runs into a museum visitors or something like this, 00:35:21.480 --> 00:35:25.150 would also have an impact map. 00:35:25.150 --> 00:35:25.650 OK. 00:35:25.650 --> 00:35:30.660 So that's all you need to use all those methods we 00:35:30.660 --> 00:35:31.590 did before. 00:35:31.590 --> 00:35:33.540 OK. 00:35:33.540 --> 00:35:36.690 So if I wanted to now optimize the trajectory, 00:35:36.690 --> 00:35:40.170 let's say, of my periodic system-- 00:35:40.170 --> 00:35:42.480 if I had my dynamics and now we're back. 00:35:42.480 --> 00:35:45.270 I did this just in the simple case of f of x, 00:35:45.270 --> 00:35:50.110 but if I had f of x, u and I wanted to optimize u in order 00:35:50.110 --> 00:35:52.480 to do something good on the Poincare map that 00:35:52.480 --> 00:35:54.800 minimizes the cost function, I can take the gradients. 00:35:54.800 --> 00:35:56.090 I can do my optimization. 00:35:59.500 --> 00:36:02.140 But this idea of actually changing to a discrete time 00:36:02.140 --> 00:36:04.780 system is a bit empowering too. 00:36:04.780 --> 00:36:05.950 You can always do it. 00:36:05.950 --> 00:36:08.890 It's not that you only do it for walking robots. 00:36:08.890 --> 00:36:10.720 Any continuous time system, instead 00:36:10.720 --> 00:36:14.110 of looking at it at every t, I could look at it 00:36:14.110 --> 00:36:17.020 at discrete intervals of time. 00:36:17.020 --> 00:36:20.110 I could take a ball that I throw across the room, 00:36:20.110 --> 00:36:23.530 has no impact whatsoever well at the beginning, 00:36:23.530 --> 00:36:27.010 and then I could look at the system at x of time 0, 00:36:27.010 --> 00:36:30.550 then x at t equals 1 second, x of t equals 2 second. 00:36:30.550 --> 00:36:32.380 And I could build a discrete time system 00:36:32.380 --> 00:36:35.920 for any of these continuous time systems. 00:36:35.920 --> 00:36:40.420 It's particularly nice to do it on these walking robots, 00:36:40.420 --> 00:36:42.730 because, well, let's do it. 00:36:42.730 --> 00:36:45.190 So what have I just done? 00:36:45.190 --> 00:36:55.720 I've computed partial p partial x, which is telling me 00:36:55.720 --> 00:37:00.580 that I'm going to have-- 00:37:00.580 --> 00:37:03.220 I'm approximating, it's a Taylor expansion, 00:37:03.220 --> 00:37:13.390 of my dynamics around some nominal point. 00:37:25.390 --> 00:37:31.450 If x nominal is a fixed point, or if I just change 00:37:31.450 --> 00:37:38.630 my coordinate systems to x bar is xn minus x nominal n . 00:37:38.630 --> 00:37:41.200 And I could do the time varying thing if I want, 00:37:41.200 --> 00:37:46.540 or I could just do it simply if xbar0 is a fixed point, 00:37:46.540 --> 00:37:53.000 then I'm left with about a model like this. 00:38:00.630 --> 00:38:01.710 Careful here. 00:38:01.710 --> 00:38:03.120 p's everywhere. 00:38:03.120 --> 00:38:09.948 p n plus 1 is a xbar p of n. 00:38:16.430 --> 00:38:18.050 OK. 00:38:18.050 --> 00:38:20.300 I can immediately look for the stability of that model 00:38:20.300 --> 00:38:24.830 by taking the eigenvectors, eigenvalues. 00:38:24.830 --> 00:38:26.570 Stability conditions are discrete time 00:38:26.570 --> 00:38:30.020 so those eigenvalues had better be bounded by 1 and negative 1. 00:38:30.020 --> 00:38:33.800 It's not the same condition but perfectly easy 00:38:33.800 --> 00:38:36.680 to analyze the stability of the fixed points. 00:38:40.400 --> 00:38:42.650 So now for the-- 00:38:42.650 --> 00:38:46.790 if I just cared about analyzing the passive walker, 00:38:46.790 --> 00:38:49.910 I've got ways now by computing partial p partial x, 00:38:49.910 --> 00:38:53.090 handing it to SNOPT or something to solve to find the roots. 00:38:53.090 --> 00:38:55.325 I can find the fixed points and it also 00:38:55.325 --> 00:38:57.200 happens that by computing partial p partial x 00:38:57.200 --> 00:38:59.730 I can evaluate the stability of those fixed points. 00:38:59.730 --> 00:39:00.230 Yeah. 00:39:02.760 --> 00:39:05.850 The rimless wheels fixed points, if you remember, 00:39:05.850 --> 00:39:08.080 there was two of them. 00:39:08.080 --> 00:39:10.230 One was a rolling fixed point. 00:39:10.230 --> 00:39:13.320 One was a standing still fixed point. 00:39:13.320 --> 00:39:16.920 Both of them locally stable. 00:39:16.920 --> 00:39:20.740 The compass gait, if you remember the compass gait, 00:39:20.740 --> 00:39:24.600 It also can have multiple fixed points. 00:39:24.600 --> 00:39:26.940 It has one fixed point for a nominal slope. 00:39:26.940 --> 00:39:28.980 It has one fixed point that's walking, 00:39:28.980 --> 00:39:30.910 and it actually has an unstable fixed point. 00:39:33.890 --> 00:39:36.290 Now the cool thing is-- so Ambarish Goswami, 00:39:36.290 --> 00:39:39.620 who's a friend and does a lot of the initial confiscated 00:39:39.620 --> 00:39:40.640 analysis. 00:39:40.640 --> 00:39:45.290 If you start inclining the ramp steeper and steeper, 00:39:45.290 --> 00:39:48.470 then those fixed points change as a function of the dynamics. 00:39:48.470 --> 00:39:50.150 Something really interesting happens. 00:39:50.150 --> 00:39:52.670 It's not surprising that these are complicated dynamics. 00:39:52.670 --> 00:39:54.990 But you actually at some point, at some critical angle, 00:39:54.990 --> 00:39:58.640 you get a period doubling bifurcation. 00:39:58.640 --> 00:40:01.430 So you get an extra point that corresponds 00:40:01.430 --> 00:40:06.730 not to a one step fixed point, but to a two step fixed point. 00:40:06.730 --> 00:40:07.990 So what does that mean? 00:40:07.990 --> 00:40:11.500 What does that correspond to physically? 00:40:11.500 --> 00:40:12.812 What is a two step? 00:40:12.812 --> 00:40:14.770 What do I mean by two step on the Poincare map? 00:40:19.240 --> 00:40:25.120 So at x of n plus 2 is going to equal x of n. 00:40:25.120 --> 00:40:26.740 But x of n plus 1 doesn't necessarily. 00:40:26.740 --> 00:40:29.300 But every other thing is going to be the same. 00:40:29.300 --> 00:40:32.710 So what does that physically correspond to in the walking? 00:40:32.710 --> 00:40:34.600 It's more of a limp I'd say. 00:40:34.600 --> 00:40:36.490 Yeah it's some asymmetric gait. 00:40:36.490 --> 00:40:39.220 It's like this. 00:40:39.220 --> 00:40:42.100 These robots are doing potentially asymmetric thing. 00:40:42.100 --> 00:40:46.180 Every other footstep ends up landing in the same place 00:40:46.180 --> 00:40:48.460 but not every footstep. 00:40:48.460 --> 00:40:49.025 OK. 00:40:49.025 --> 00:40:51.400 So it's actually-- there's a lot of interesting work just 00:40:51.400 --> 00:40:53.400 looking at these passive models and just looking 00:40:53.400 --> 00:40:56.170 at the stability of these passive gaits. 00:40:56.170 --> 00:40:58.350 OK. 00:40:58.350 --> 00:41:02.060 I do want to say make one note here, 00:41:02.060 --> 00:41:04.180 If you're looking at the eigenvalues of A, 00:41:04.180 --> 00:41:05.230 the way I've defined it. 00:41:08.120 --> 00:41:11.680 So the trivial condition here is that the eigenvalues of A 00:41:11.680 --> 00:41:13.420 had better be less than-- 00:41:13.420 --> 00:41:15.640 the magnitude had better be less than 1. 00:41:19.808 --> 00:41:22.350 There's different notations for working on the Poincare maps. 00:41:22.350 --> 00:41:26.400 My notation is to denote x of p where 00:41:26.400 --> 00:41:29.972 x is the same size as the original vector. 00:41:29.972 --> 00:41:32.430 Because that's more useful in simulation that's more useful 00:41:32.430 --> 00:41:34.050 most of our computations. 00:41:34.050 --> 00:41:37.200 But remember the Poincare map effectively 00:41:37.200 --> 00:41:41.135 reduces dimensionality by 1 And in the way 00:41:41.135 --> 00:41:42.510 I've written it here, if you just 00:41:42.510 --> 00:41:46.590 look at A that comes out of a stable periodic oscillation, 00:41:46.590 --> 00:41:49.950 there's actually, the way I've done it, 00:41:49.950 --> 00:41:52.590 there's always a trivial eigenvalue 00:41:52.590 --> 00:42:00.900 of 1, which doesn't degrade the stability of the system. 00:42:00.900 --> 00:42:03.000 What does that eigenvalue correspond to? 00:42:08.010 --> 00:42:12.120 If I take my stable rimless wheel, 00:42:12.120 --> 00:42:16.122 I compute A. I take the eigenvalues. 00:42:16.122 --> 00:42:17.330 I see that the system stable. 00:42:17.330 --> 00:42:19.980 It goes to my nominal thing. 00:42:19.980 --> 00:42:22.200 This is the standard stability criteria 00:42:22.200 --> 00:42:23.500 for discrete time system. 00:42:23.500 --> 00:42:25.848 But I notice that there's actually always a trivial-- 00:42:25.848 --> 00:42:27.390 there's one eigenvalue that equals 1. 00:42:31.150 --> 00:42:33.330 Good. 00:42:33.330 --> 00:42:34.620 People see that? 00:42:34.620 --> 00:42:37.440 There's one direction that I can push it 00:42:37.440 --> 00:42:43.912 in which it does not reject that disturbance, 00:42:43.912 --> 00:42:45.620 and that's the direction-- if you push it 00:42:45.620 --> 00:42:46.537 along the limit cycle. 00:42:50.110 --> 00:42:51.420 So there's two ways to do it. 00:42:51.420 --> 00:42:52.795 I'm sorry to make it complicated, 00:42:52.795 --> 00:42:54.080 but there's two ways to do it. 00:42:54.080 --> 00:42:56.925 If I were to have redefined my coordinate system on the map, 00:42:56.925 --> 00:42:59.050 I wouldn't want to call it x anymore, but something 00:42:59.050 --> 00:43:02.830 that reduce the dimensionality of the map by 1. 00:43:02.830 --> 00:43:08.080 And then I could use this condition alone. 00:43:08.080 --> 00:43:11.290 But in the way I've done it in the notes and in life, 00:43:11.290 --> 00:43:15.280 I like to keep this sort of as the same dimension of x. 00:43:15.280 --> 00:43:18.700 And then because I want limit cycle stability, 00:43:18.700 --> 00:43:21.647 it's absolutely the case that one of the eigenvalue-- 00:43:21.647 --> 00:43:23.980 there is a direction in which those disturbances are not 00:43:23.980 --> 00:43:24.910 rejected. 00:43:24.910 --> 00:43:25.810 That's what I want. 00:43:25.810 --> 00:43:27.690 Yeah. 00:43:27.690 --> 00:43:29.860 OK. 00:43:29.860 --> 00:43:31.510 So the way I describe it in the notes 00:43:31.510 --> 00:43:35.590 is that what you want for stability of the periodic gait 00:43:35.590 --> 00:43:39.250 is you want to ignore the first eigenvalue of one. 00:43:39.250 --> 00:43:40.595 The rest of them better be-- 00:43:40.595 --> 00:43:42.220 I want it to be only one direction that 00:43:42.220 --> 00:43:45.310 doesn't reject disturbances. 00:43:45.310 --> 00:43:46.910 You with me? 00:43:46.910 --> 00:43:47.410 OK. 00:43:52.210 --> 00:43:53.740 Then there's some other direction 00:43:53.740 --> 00:43:55.448 that doesn't reject disturbances and then 00:43:55.448 --> 00:43:58.750 I start questioning whether it's stable. 00:43:58.750 --> 00:44:01.090 I wouldn't call it stable in the sense of a limit cycle 00:44:01.090 --> 00:44:01.972 stability. 00:44:04.510 --> 00:44:08.590 If I could push in some other direction in state space 00:44:08.590 --> 00:44:10.540 and doesn't get rejected, then it's 00:44:10.540 --> 00:44:14.347 not going to return to that orbit. 00:44:14.347 --> 00:44:16.430 There's a special direction along the orbit, which 00:44:16.430 --> 00:44:18.230 I'm allowed to push, and that's what 00:44:18.230 --> 00:44:20.840 defines limits of stability. 00:44:20.840 --> 00:44:22.340 But there's only one direction. 00:44:25.700 --> 00:44:29.210 Even in high dimensional space, a 50 dimensional robot 00:44:29.210 --> 00:44:31.430 going around, there's only one place 00:44:31.430 --> 00:44:35.418 that trajectory is going forward that I'm allowed to push 00:44:35.418 --> 00:44:36.710 and not reject the disturbance. 00:44:36.710 --> 00:44:42.200 Every other one better converge for me to call it stable. 00:44:47.420 --> 00:44:47.920 OK. 00:44:47.920 --> 00:44:51.820 We did the case for the non-controlled system. 00:44:51.820 --> 00:44:53.590 But we're only a stone's throw away 00:44:53.590 --> 00:44:56.673 from doing the actuated version. 00:44:56.673 --> 00:44:58.840 I'm not going to write it out again, but let's say-- 00:45:15.840 --> 00:45:17.340 let me do it a little carefully just 00:45:17.340 --> 00:45:20.952 to say what I mean carefully. 00:45:36.230 --> 00:45:41.250 So if I want to do the controlled case, 00:45:41.250 --> 00:45:45.370 get back to a system like this. 00:45:45.370 --> 00:45:48.250 We talked about in the policy gradient-- 00:45:48.250 --> 00:45:50.920 in the policy research world-- 00:45:50.920 --> 00:45:56.273 that I could then define u to be a tape of u's, or come out 00:45:56.273 --> 00:45:57.940 of a linear feedback control or whatever 00:45:57.940 --> 00:45:59.740 sort of parameterization. 00:45:59.740 --> 00:46:06.040 But I like to think of it as being some function which 00:46:06.040 --> 00:46:09.580 depends on a parameter vector, alpha, and can generally 00:46:09.580 --> 00:46:11.080 depend on x and u. 00:46:31.292 --> 00:46:32.750 So there's a different way to think 00:46:32.750 --> 00:46:39.390 about control in the discrete time Poincare sense. 00:46:43.960 --> 00:46:46.510 One way to think about it is, let's 00:46:46.510 --> 00:46:50.110 say every time I hit a surface of section, 00:46:50.110 --> 00:46:53.440 or my foot hits the ground in the walking case, 00:46:53.440 --> 00:46:59.230 why don't I change the parameter vector alpha OK. 00:46:59.230 --> 00:47:02.580 But I'm going to make decisions only once per step. 00:47:02.580 --> 00:47:05.850 And then I'm going to execute this policy 00:47:05.850 --> 00:47:11.022 for the duration of that cycle. 00:47:11.022 --> 00:47:12.480 And then the next time my foot hits 00:47:12.480 --> 00:47:15.210 the ground I'll make a different decision about alpha. 00:47:17.850 --> 00:47:19.200 You can imagine. 00:47:19.200 --> 00:47:22.650 So let's say that pi of alpha and my compass gait walker, 00:47:22.650 --> 00:47:25.943 we'll use this example in simulation in a second. 00:47:25.943 --> 00:47:27.360 Let's say I have a controller that 00:47:27.360 --> 00:47:29.790 runs during the limit cycle which 00:47:29.790 --> 00:47:35.550 tries to set my interleg angle to be some desired value. 00:47:35.550 --> 00:47:37.770 Every step I take the only decision I make 00:47:37.770 --> 00:47:40.440 is what should that desired interleg angle 00:47:40.440 --> 00:47:42.840 be for my compass gait. 00:47:42.840 --> 00:47:44.850 But over the course of the cycle, 00:47:44.850 --> 00:47:46.440 I'll simulate a PD controller that 00:47:46.440 --> 00:47:49.260 tries to make that interleg angle happen. 00:47:49.260 --> 00:47:51.900 But my discrete time decisions are just 00:47:51.900 --> 00:47:55.830 what interleg angle should I be. 00:47:55.830 --> 00:48:00.810 Fumi has got a compass gait that works with open loop 00:48:00.810 --> 00:48:03.810 trajectories that play out, where his major parameters 00:48:03.810 --> 00:48:08.430 are the frequency and phase of the periodic input. 00:48:08.430 --> 00:48:09.840 It's amazingly stable coming out. 00:48:09.840 --> 00:48:13.140 It's like a beautiful example of open loop stability. 00:48:13.140 --> 00:48:14.760 OK. 00:48:14.760 --> 00:48:18.000 So Fumi and I have been talking about changing 00:48:18.000 --> 00:48:22.830 the parameters of his open loop controller once per step, 00:48:22.830 --> 00:48:28.960 in order to try to regulate the behavior of the system. 00:48:28.960 --> 00:48:29.670 OK. 00:48:29.670 --> 00:48:31.470 There's lots of ways you could do it. 00:48:31.470 --> 00:48:35.730 But, generally, what that gives you-- 00:48:35.730 --> 00:48:39.880 if you just try to compute these exact same gradients again, 00:48:39.880 --> 00:48:43.412 we've computed the gradient with respect to the initial state. 00:48:43.412 --> 00:48:45.870 But we could also compute a gradient with respect to alpha. 00:48:48.980 --> 00:48:52.970 And what that could give me here is a model like this. 00:48:52.970 --> 00:48:56.180 I call it B because I'm thinking of alpha like a control 00:48:56.180 --> 00:48:58.100 decision. 00:48:58.100 --> 00:49:07.760 Where alpha bar of n is the difference between my control 00:49:07.760 --> 00:49:11.690 decision at n minus whatever the nominal alpha is. 00:49:21.610 --> 00:49:22.350 OK. 00:49:22.350 --> 00:49:27.420 So let's say I find a stable limit cycle where I just 00:49:27.420 --> 00:49:30.240 command the interleg angle to be, I don't know, 00:49:30.240 --> 00:49:33.450 pi/4 every step. 00:49:33.450 --> 00:49:37.230 And I can find a stable limit cycle behavior. 00:49:37.230 --> 00:49:41.200 Well if someone pushes my robot, computing these A and B 00:49:41.200 --> 00:49:44.353 matrices give me a nice, simple way 00:49:44.353 --> 00:49:46.770 to make discrete decisions to try to stabilize that robot. 00:49:46.770 --> 00:49:48.900 If I'm away from my limit cycle, I 00:49:48.900 --> 00:49:52.800 can correct for those differences by just saying, OK, 00:49:52.800 --> 00:49:55.740 next time take a bigger step and then take a smaller step. 00:49:58.470 --> 00:50:02.070 And how would I design that rule to change alpha? 00:50:08.000 --> 00:50:11.210 How would you design it if you wanted to make a feedback 00:50:11.210 --> 00:50:12.020 law an alpha here? 00:50:17.540 --> 00:50:19.640 This is a discrete time linear system. 00:50:19.640 --> 00:50:23.160 I could just hand it to MATLAB and call LQR, discrete time 00:50:23.160 --> 00:50:25.250 LQR. 00:50:25.250 --> 00:50:31.250 Give it A, B, Q, and R. It'll give me back a negative k 00:50:31.250 --> 00:50:33.800 alpha, a kx to find alpha. 00:50:37.850 --> 00:50:39.630 Are you with me on that? 00:50:39.630 --> 00:50:44.210 So the discrete time summarization 00:50:44.210 --> 00:50:47.720 of the dynamics in this way can actually 00:50:47.720 --> 00:50:51.740 make it very natural and very simple to compute controllers, 00:50:51.740 --> 00:50:54.332 which stabilize a walking cycle for instance, where you 00:50:54.332 --> 00:50:55.540 make decisions once per step. 00:51:01.870 --> 00:51:07.260 I want to show you some work that Katie Byl did in my lab, 00:51:07.260 --> 00:51:10.080 where she did it on the nonlinear case 00:51:10.080 --> 00:51:13.200 for the compass gait. 00:51:13.200 --> 00:51:16.200 Instead of linearizing this, she went right 00:51:16.200 --> 00:51:21.900 with the full nonlinear Poincare map. 00:51:21.900 --> 00:51:28.830 She just did x of n plus 1 is some Poincare map 00:51:28.830 --> 00:51:34.110 of x of n parameterized by alpha, or alpha n. 00:51:38.540 --> 00:51:40.790 And the compass gait, how many dimensions? 00:51:40.790 --> 00:51:43.800 What it is it's four basic dimensions. 00:51:43.800 --> 00:51:47.962 There's two angles and two velocities. 00:51:51.410 --> 00:51:54.260 And it's just about the right size 00:51:54.260 --> 00:51:58.820 that you can try discretizing and doing value iteration. 00:51:58.820 --> 00:52:00.380 So Katie had some nice work showing 00:52:00.380 --> 00:52:01.970 that on the compass gait we can really 00:52:01.970 --> 00:52:04.670 get a pretty good sense of the nonlinear optimal solution 00:52:04.670 --> 00:52:09.140 by just popping this discrete time control 00:52:09.140 --> 00:52:15.050 problem into a value iteration, doing some work, 00:52:15.050 --> 00:52:17.390 and computing back an optical feedback policy, which 00:52:17.390 --> 00:52:21.290 tells you where you should step every time in order 00:52:21.290 --> 00:52:23.210 to stabilize your gait. 00:52:23.210 --> 00:52:27.290 One better, if you add one more dimension to your compass gait 00:52:27.290 --> 00:52:31.540 robot, you can actually think about where it is on terrain, 00:52:31.540 --> 00:52:33.040 and make it walk over rough terrain. 00:52:35.760 --> 00:52:38.490 So maybe I didn't spend enough time 00:52:38.490 --> 00:52:40.500 in these two lectures convincing you 00:52:40.500 --> 00:52:43.620 that nobody knows how to make robots walk on rough terrain, 00:52:43.620 --> 00:52:45.208 but-- 00:52:45.208 --> 00:52:47.000 oh shoot, it's going to take a second here. 00:52:52.530 --> 00:52:54.197 But you might know for yourself that you 00:52:54.197 --> 00:52:56.613 don't see a lot of walking robots walking on rough terrain 00:52:56.613 --> 00:52:57.150 yet. 00:52:57.150 --> 00:53:00.537 Big Dog's sort of the exception. 00:53:00.537 --> 00:53:01.870 Big Dog seems to be pretty good. 00:53:07.450 --> 00:53:09.900 We've got a little dog upstairs in our lab, which 00:53:09.900 --> 00:53:11.850 is also pretty good I think. 00:53:11.850 --> 00:53:12.840 They're different. 00:53:12.840 --> 00:53:14.490 Little dog got can see the terrain 00:53:14.490 --> 00:53:16.290 and Big Dog can't, so we're supposed 00:53:16.290 --> 00:53:23.040 to be doing the long term research for Big Dog's idea. 00:53:34.550 --> 00:53:38.105 Now we've had that entry music here. 00:53:38.105 --> 00:53:39.980 It's a very dramatic simulation that follows. 00:53:39.980 --> 00:53:40.480 OK. 00:53:40.480 --> 00:53:44.170 Here's Katie's work on compass gaits on rough terrain. 00:53:44.170 --> 00:53:46.130 A little compass gait robot. 00:53:46.130 --> 00:53:49.160 Its making control decisions every footstep. 00:53:49.160 --> 00:53:51.530 It knows where the terrain is. 00:53:51.530 --> 00:53:54.230 Pop it into to a big value iteration solver. 00:53:54.230 --> 00:53:59.660 And this thing can walk over almost anything, 00:53:59.660 --> 00:54:01.130 It applies to talk at the hip width 00:54:01.130 --> 00:54:03.290 the PD controller I described, and it also 00:54:03.290 --> 00:54:09.180 puts in an impulse at the foot, one time every post impact. 00:54:09.180 --> 00:54:12.050 So it can push itself forward a little bit. 00:54:12.050 --> 00:54:14.180 So this thing it's pretty good. 00:54:19.660 --> 00:54:21.280 And it's just converting this sort 00:54:21.280 --> 00:54:25.570 of complicated looking problem into a valuation problem. 00:54:30.970 --> 00:54:32.970 It doesn't think about the slope of the terrain. 00:54:32.970 --> 00:54:34.240 It could. 00:54:34.240 --> 00:54:37.540 But the reason it's tractable, actually, 00:54:37.540 --> 00:54:41.303 if you can tell by the footprints, it wraps around. 00:54:41.303 --> 00:54:42.970 So we've got a limited state space in x. 00:54:47.318 --> 00:54:47.860 We ignore it. 00:54:47.860 --> 00:54:49.810 If you look real carefully, I think 00:54:49.810 --> 00:54:52.210 the leg gets a little shorter every time it goes. 00:54:52.210 --> 00:54:54.520 But the mass looks like there's no dynamics like that. 00:54:54.520 --> 00:54:57.190 We built the robot the same way to have a little tweak 00:54:57.190 --> 00:54:58.200 toe that pulls up. 00:55:03.850 --> 00:55:05.930 It's trivial in simulation. 00:55:05.930 --> 00:55:06.640 Yeah. 00:55:06.640 --> 00:55:08.020 How do we do it in real life? 00:55:08.020 --> 00:55:09.820 Not as well as we'd like. 00:55:09.820 --> 00:55:12.160 Fumi has got a design that's got a servo that 00:55:12.160 --> 00:55:14.290 pushes as fast as we can. 00:55:14.290 --> 00:55:18.190 We've talked about pneumatics, that would be a little faster. 00:55:18.190 --> 00:55:21.250 But we haven't run that, run them yet. 00:55:24.390 --> 00:55:27.300 It's interesting. 00:55:27.300 --> 00:55:30.910 So she actually did the case of just hip, just foot. 00:55:30.910 --> 00:55:31.690 They both work. 00:55:31.690 --> 00:55:34.960 They both stabilize the walking on moderate terrain. 00:55:34.960 --> 00:55:37.700 But together they're much more stable than the other. 00:55:37.700 --> 00:55:39.550 The thing with the hip one, it always gets 00:55:39.550 --> 00:55:43.060 in these configurations like this and falls backwards. 00:55:43.060 --> 00:55:45.557 You need a little bit of foot energy to get over. 00:55:45.557 --> 00:55:47.140 And the one that pushes off of the toe 00:55:47.140 --> 00:55:49.098 keeps putting its foot down in the wrong place. 00:55:51.580 --> 00:55:55.420 There's one thing that Katie's simulations had problems with. 00:55:55.420 --> 00:55:57.730 You guys in my lab know. 00:55:57.730 --> 00:56:01.280 But there's a pretty sort of very visual-- 00:56:01.280 --> 00:56:02.530 I should find the video maybe. 00:56:02.530 --> 00:56:05.850 But there's a very visual thing that we had problems. 00:56:05.850 --> 00:56:07.600 What was the problem I always talked about 00:56:07.600 --> 00:56:08.260 with valuation iteration? 00:56:08.260 --> 00:56:09.670 Even if you could pile the space, 00:56:09.670 --> 00:56:13.540 you got to watch out for something. 00:56:13.540 --> 00:56:14.950 Yeah. 00:56:14.950 --> 00:56:18.370 Your discretization, if there's hard discontinuities, 00:56:18.370 --> 00:56:21.670 you can do things wrong in your discretization. 00:56:21.670 --> 00:56:24.010 Your discrimination can be a poor approximation 00:56:24.010 --> 00:56:25.390 of your continuous thing. 00:56:25.390 --> 00:56:28.780 So Katie stimulated a bunch of terrain that had holes in it 00:56:28.780 --> 00:56:29.830 like cliffs. 00:56:29.830 --> 00:56:32.320 And had these beautiful policies that 00:56:32.320 --> 00:56:36.280 would choose their step across Karate Kid 00:56:36.280 --> 00:56:39.130 sort of style terrain. 00:56:39.130 --> 00:56:40.840 But every once in a while, the mesh 00:56:40.840 --> 00:56:43.205 points would land in the wrong place 00:56:43.205 --> 00:56:44.830 and the stupid thing would put its foot 00:56:44.830 --> 00:56:49.840 right in the middle of the hole, right down into the ether. 00:56:49.840 --> 00:56:52.180 This is sort of the textbook case of value iteration 00:56:52.180 --> 00:56:53.170 resolution problems. 00:56:57.260 --> 00:56:57.760 OK. 00:56:57.760 --> 00:57:02.960 So walking isn't any harder than any other robot really. 00:57:02.960 --> 00:57:04.210 There's mechanical challenges. 00:57:04.210 --> 00:57:05.780 You have to carry your actuators. 00:57:05.780 --> 00:57:08.590 You're typically dealing with actuator saturations a lot. 00:57:08.590 --> 00:57:10.513 So it's definitely underactuated. 00:57:10.513 --> 00:57:12.430 I don't mean to say it's easy, because I think 00:57:12.430 --> 00:57:14.080 the acrobats are rich and hard. 00:57:14.080 --> 00:57:15.760 But it's not significantly harder 00:57:15.760 --> 00:57:18.302 than the acrobat and this is, remember, the acrobat dynamics. 00:57:20.472 --> 00:57:22.930 The only two differences are that you have to worry about-- 00:57:22.930 --> 00:57:25.602 you tend to think about Poincare maps to define stability, 00:57:25.602 --> 00:57:27.810 and you have to worry about these collision dynamics. 00:57:30.460 --> 00:57:31.450 You get 100 degree-- 00:57:31.450 --> 00:57:34.090 I'm not showing you 100 degree of freedom robot here walking 00:57:34.090 --> 00:57:36.220 along but you could. 00:57:36.220 --> 00:57:37.570 I mean we're getting there. 00:57:37.570 --> 00:57:40.840 So the dork hall type methods that we're using 00:57:40.840 --> 00:57:42.270 with the LTV linearization. 00:57:42.270 --> 00:57:44.770 We think that's going to work pretty nicely for a little dog 00:57:44.770 --> 00:57:48.760 and that's 36 dimensional, walking on very rough terrain. 00:57:53.170 --> 00:57:56.380 So there's been a lot of work in the control 00:57:56.380 --> 00:57:59.020 of this, sort of outside the optimal control 00:57:59.020 --> 00:58:01.360 view of the world. 00:58:01.360 --> 00:58:04.000 I'd be doing you a disservice if I didn't tell you 00:58:04.000 --> 00:58:05.020 a little bit of it. 00:58:18.347 --> 00:58:20.180 Make sure I said everything I wanted to say. 00:58:25.528 --> 00:58:27.320 There was one other point I wanted to make. 00:58:30.320 --> 00:58:34.940 There's something you can do in discrete time 00:58:34.940 --> 00:58:38.960 that you can't really do in continuous time 00:58:38.960 --> 00:58:41.070 when it comes to stabilizing controllers. 00:58:44.380 --> 00:58:45.630 Anybody get it from just that? 00:58:45.630 --> 00:58:48.850 I said it very obtusely, but yeah 00:58:48.850 --> 00:58:49.350 Yeah. 00:58:49.350 --> 00:58:51.240 Good. 00:58:51.240 --> 00:58:54.030 So in discrete time, I can actually 00:58:54.030 --> 00:58:55.710 find an action, potentially, depends 00:58:55.710 --> 00:58:59.370 on B. I could find an action which would actually 00:58:59.370 --> 00:59:05.940 drive me to zero, in my aerodynamics, in a single step. 00:59:05.940 --> 00:59:08.640 In continuous time, getting there arbitrarily 00:59:08.640 --> 00:59:12.090 fast means setting your gains arbitrarily high. 00:59:12.090 --> 00:59:13.810 In discrete time, that's not the case. 00:59:13.810 --> 00:59:17.970 I can set a finite magnitude gain 00:59:17.970 --> 00:59:23.063 that will get me to zero in a single step. 00:59:23.063 --> 00:59:24.480 So that's called deadbeat control. 00:59:32.740 --> 00:59:36.910 It's a very beautiful goal to have for a walking robot. 00:59:36.910 --> 00:59:40.390 if I get perturbed by the terrain or by someone hitting 00:59:40.390 --> 00:59:41.950 me with a baseball bat or whatever-- 00:59:41.950 --> 00:59:44.260 you should see the videos that the robotics people make 00:59:44.260 --> 00:59:45.010 of walking robot-- 00:59:47.860 --> 00:59:50.170 then a very beautiful goal is to say 00:59:50.170 --> 00:59:53.590 that before my next foot hits the ground, 00:59:53.590 --> 00:59:57.090 I'm going to cancel out all the error for my disturbance. 01:00:00.190 --> 01:00:04.840 Certainly you can do that here if you have no limits on alpha 01:00:04.840 --> 01:00:08.920 and if B is full row rank. 01:00:08.920 --> 01:00:12.040 Then it's algebraic equation to compute 01:00:12.040 --> 01:00:16.720 what alpha n had better be. 01:00:16.720 --> 01:00:20.115 I could just say, if you want something that could basically 01:00:20.115 --> 01:00:21.490 look like feedback linearization, 01:00:21.490 --> 01:00:22.698 I could just cancel that out. 01:00:26.400 --> 01:00:28.500 Sometimes you can't do it, but I want 01:00:28.500 --> 01:00:32.040 you to know that's a beautiful goal for control. 01:00:32.040 --> 01:00:34.770 We'll see in the running bit, we'll 01:00:34.770 --> 01:00:38.580 see some running models where you can do that B control. 01:00:38.580 --> 01:00:41.790 The compass gait, if you had the right parameterization, 01:00:41.790 --> 01:00:43.207 you could do compass gait control. 01:00:43.207 --> 01:00:44.915 John and I were debating this last night. 01:00:44.915 --> 01:00:47.340 I don't think that the PD controller is probably 01:00:47.340 --> 01:00:50.400 going to be enough to give you that B control. 01:00:50.400 --> 01:00:53.160 Because it won't simultaneously take out your energy 01:00:53.160 --> 01:00:56.250 and get your foot in the right place. 01:00:56.250 --> 01:00:57.000 But it's possible. 01:00:57.000 --> 01:00:58.417 You could actually do the analysis 01:00:58.417 --> 01:01:00.540 and answer that question. 01:01:03.050 --> 01:01:03.738 OK. 01:01:03.738 --> 01:01:06.030 So thinking about these things as discrete time control 01:01:06.030 --> 01:01:09.240 problems is pretty beautiful. 01:01:09.240 --> 01:01:15.060 More generally, though, we can do control through the swing 01:01:15.060 --> 01:01:16.130 phase, in this case. 01:01:16.130 --> 01:01:19.140 Or between the surfaces of section 01:01:19.140 --> 01:01:22.380 in a general limited cycle case. 01:01:22.380 --> 01:01:24.510 And if you want to do that, you could do 01:01:24.510 --> 01:01:26.010 you could do a shooting method. 01:01:26.010 --> 01:01:27.060 You could do deer call. 01:01:27.060 --> 01:01:28.200 Optimize some trajectory. 01:01:28.200 --> 01:01:30.450 You have to be careful because the time can change 01:01:30.450 --> 01:01:32.700 during your optimization, the duration of your trajectory, 01:01:32.700 --> 01:01:33.617 but you could do that. 01:01:36.690 --> 01:01:38.490 And then optimize it. 01:01:38.490 --> 01:01:42.360 But just like we showed in the acrobat and cart-pole case, 01:01:42.360 --> 01:01:45.090 people have come up with more-- 01:01:45.090 --> 01:01:47.550 I don't want to say problem specific-- but more sort 01:01:47.550 --> 01:01:52.650 of problem specific solutions. 01:01:52.650 --> 01:01:58.800 So Goswami, Ambarish Goswami did some nice work. 01:02:03.190 --> 01:02:04.650 Can anybody guess? 01:02:04.650 --> 01:02:07.620 What's one of the dominant nonlinear control ideas 01:02:07.620 --> 01:02:12.060 we talked about for the acrobat and cart-pole. 01:02:12.060 --> 01:02:16.110 PFL, but that led to energy shaping. 01:02:16.110 --> 01:02:20.040 So Goswami did a nice controller based on energy shaping. 01:02:29.950 --> 01:02:34.390 And he showed that you take your nominal confiscate 01:02:34.390 --> 01:02:36.310 with its fragile little basin of attraction, 01:02:36.310 --> 01:02:39.250 which he computed by sampling. 01:02:39.250 --> 01:02:41.782 Pushing the system until it fell down. 01:02:41.782 --> 01:02:43.990 And he showed that with an energy shaping controller, 01:02:43.990 --> 01:02:45.532 which is derived exactly the same way 01:02:45.532 --> 01:02:48.080 we derive the other one, but I can 01:02:48.080 --> 01:02:50.080 regulate the energy of the system to put me back 01:02:50.080 --> 01:02:51.400 on the energetic orbit. 01:02:51.400 --> 01:02:54.880 Because this thing is with zero torque. 01:02:54.880 --> 01:02:58.550 It's passive in the swing phase. 01:02:58.550 --> 01:03:00.790 So I can just drive myself up to the place 01:03:00.790 --> 01:03:05.380 where I will passively fall down and hit the right place 01:03:05.380 --> 01:03:07.510 on my Poincare map. 01:03:07.510 --> 01:03:08.710 And that worked locally. 01:03:08.710 --> 01:03:09.510 That's a good idea. 01:03:16.050 --> 01:03:18.680 There's another bunch of work that's 01:03:18.680 --> 01:03:21.720 become popular in our world. 01:03:21.720 --> 01:03:34.970 Eric Westervelt and Jesse Grizzelle and these guys 01:03:34.970 --> 01:03:36.770 they talk about hybrid zero dynamics. 01:03:50.830 --> 01:03:54.850 And more recently, our friend who's 01:03:54.850 --> 01:04:02.230 going to join the lab, the advisor in Manchester. 01:04:05.200 --> 01:04:08.080 These guys have been doing sort of similar work using more 01:04:08.080 --> 01:04:11.890 optimal control derivations and similar hybrid 01:04:11.890 --> 01:04:15.490 zero dynamic kind of methods. 01:04:15.490 --> 01:04:17.230 Let me just cartoon the idea. 01:04:19.840 --> 01:04:23.170 It's sort of similar to the PFL idea. 01:04:23.170 --> 01:04:25.270 But these hybrid zero dynamic methods 01:04:25.270 --> 01:04:27.040 are one of the places where we actually 01:04:27.040 --> 01:04:30.490 have proofs of convergence. 01:04:36.090 --> 01:04:41.190 And the way they do it, like in partial feedback linearization, 01:04:41.190 --> 01:04:45.600 is that when the system is-- 01:04:45.600 --> 01:04:48.750 let me say it carefully here. 01:04:48.750 --> 01:04:50.280 They find some desired trajectory 01:04:50.280 --> 01:04:54.450 of the robot, which has a periodic gait that they like. 01:04:54.450 --> 01:04:55.975 It has actuated joints that follow 01:04:55.975 --> 01:04:57.600 this trajectory and passive joints that 01:04:57.600 --> 01:04:59.640 follow this trajectory. 01:04:59.640 --> 01:05:03.780 If you do a collocated PFL, if you just 01:05:03.780 --> 01:05:06.480 regulate the dynamics of your system, 01:05:06.480 --> 01:05:10.020 you're actuated joints to follow that trajectory, 01:05:10.020 --> 01:05:12.420 then the passive one pretty much has to do the same, 01:05:12.420 --> 01:05:15.160 has to do the right thing. 01:05:15.160 --> 01:05:16.440 OK. 01:05:16.440 --> 01:05:19.770 So the results that these guys on the hybrid zero dynamics-- 01:05:19.770 --> 01:05:22.020 they talk at the zero dynamics of the resulting output 01:05:22.020 --> 01:05:24.660 dynamics you drive to drive to zero. 01:05:24.660 --> 01:05:25.710 Yeah. 01:05:25.710 --> 01:05:28.890 When your PFL controller has done its job-- 01:05:28.890 --> 01:05:30.690 and has it doesn't have to be a PFL. 01:05:30.690 --> 01:05:32.293 It could be a speed controller. 01:05:32.293 --> 01:05:34.710 When you've driven all your actuated joints to the desired 01:05:34.710 --> 01:05:38.820 trajectory, if you parameterized that trajectory off 01:05:38.820 --> 01:05:42.750 let's say the ankle angle, then your whole big complicated 01:05:42.750 --> 01:05:46.380 bipedal robot, five link ten link whatever. 01:05:46.380 --> 01:05:52.770 Looks like a remote wheel rolling around the stance foot. 01:05:52.770 --> 01:05:54.160 And that's a beautiful idea. 01:05:54.160 --> 01:05:56.820 So maybe the bigger idea here is actually. 01:06:13.302 --> 01:06:14.760 A lot of the walking robots you can 01:06:14.760 --> 01:06:30.290 think of as having a lot of actuators at all the joints 01:06:30.290 --> 01:06:35.160 and just having one passive joint. 01:06:35.160 --> 01:06:38.570 So a pretty good idea for a walking robot 01:06:38.570 --> 01:06:42.620 is to just drive all of the actuated joints as a function 01:06:42.620 --> 01:06:44.990 of the passive joint. 01:06:44.990 --> 01:06:49.503 And that gives you a dynamical system in one variable. 01:06:49.503 --> 01:06:51.170 It's a more complicated dynamical system 01:06:51.170 --> 01:06:52.128 than the rimless wheel. 01:06:52.128 --> 01:06:55.310 It's one that you can design but the analysis 01:06:55.310 --> 01:06:59.510 reverts to basically the rimless wheel type analysis. 01:06:59.510 --> 01:07:00.607 OK. 01:07:00.607 --> 01:07:02.690 So this hybrid zero dynamics is roughly that idea. 01:07:34.270 --> 01:07:36.864 And then you can sort of do a rimless wheel. 01:08:01.070 --> 01:08:05.250 I have had this debate with Eric Westervelt many times. 01:08:05.250 --> 01:08:05.750 Yeah. 01:08:05.750 --> 01:08:06.980 I think it is. 01:08:06.980 --> 01:08:07.940 Yeah. 01:08:07.940 --> 01:08:08.750 That's a good idea. 01:08:08.750 --> 01:08:10.320 Just like PFL is great. 01:08:10.320 --> 01:08:11.998 We should use it for a little while 01:08:11.998 --> 01:08:13.790 until we figure out something better to do. 01:08:13.790 --> 01:08:15.470 I think they'd admit that too. 01:08:15.470 --> 01:08:17.029 Their stability guarantees are when 01:08:17.029 --> 01:08:19.700 you've driven the aerodynamics to zero. 01:08:19.700 --> 01:08:21.229 You squash the aerodynamics. 01:08:21.229 --> 01:08:24.200 And you have to have gains that are high enough to squash those 01:08:24.200 --> 01:08:26.420 arbitrarily fast. 01:08:26.420 --> 01:08:31.729 Now the bigger idea I think is that I 01:08:31.729 --> 01:08:34.850 mean you can do sort of softer feedback, 01:08:34.850 --> 01:08:36.770 and get some of the same type of performance, 01:08:36.770 --> 01:08:38.556 but the theory does depend on squashing. 01:08:41.913 --> 01:08:43.580 So all this stuff I talked about I think 01:08:43.580 --> 01:08:47.029 is sort of the right way to think about walking control. 01:08:47.029 --> 01:08:49.880 It's a small fraction of what people actually 01:08:49.880 --> 01:08:52.430 do in the walking robot world. 01:08:52.430 --> 01:08:56.370 By far, the dominant approach is these Honda ASIMO type robots. 01:08:56.370 --> 01:09:03.352 Anybody see HRP4C yesterday. 01:09:03.352 --> 01:09:04.310 What does it look like? 01:09:09.830 --> 01:09:11.660 It looks like a model. 01:09:11.660 --> 01:09:14.569 They actually videotaped a Japanese model 01:09:14.569 --> 01:09:17.990 walking in high heels, and they built 01:09:17.990 --> 01:09:23.240 one of their newest that's the AIST robot by Kawada 01:09:23.240 --> 01:09:23.870 Industries. 01:09:23.870 --> 01:09:28.107 It's got now sort of a female Android head on it. 01:09:28.107 --> 01:09:28.649 And it works. 01:09:28.649 --> 01:09:29.982 I mean, I just saw a few videos. 01:09:29.982 --> 01:09:31.560 But it looks pretty good. 01:09:31.560 --> 01:09:37.464 The shapes are more feminine and the gait 01:09:37.464 --> 01:09:38.464 was a little bit looser. 01:09:41.340 --> 01:09:43.850 But most of the walking robots out there today, 01:09:43.850 --> 01:09:48.740 most of the successful ones, are all pretty similar to ASIMO. 01:09:48.740 --> 01:09:52.729 And ASIMO doesn't do any of this stuff really. 01:09:52.729 --> 01:09:54.050 ASIMO plays one trick. 01:09:54.050 --> 01:09:56.810 I can say it is sort of in a single line. 01:09:56.810 --> 01:10:02.310 They try to keep one foot flat on the ground, 01:10:02.310 --> 01:10:05.513 and assume that foot is bolted to the ground. 01:10:05.513 --> 01:10:07.430 And they pretend they're fully actuated system 01:10:07.430 --> 01:10:10.258 and do their trajectory tracking. 01:10:10.258 --> 01:10:12.300 And when they're doing their trajectory tracking, 01:10:12.300 --> 01:10:14.550 they have to make darn sure that foot doesn't roll off 01:10:14.550 --> 01:10:15.408 the ground. 01:10:15.408 --> 01:10:16.950 But if you watch the videos of ASIMO, 01:10:16.950 --> 01:10:18.408 you're going to see it's always got 01:10:18.408 --> 01:10:19.950 one foot flat on the ground. 01:10:19.950 --> 01:10:21.510 Running is a small excursion. 01:10:21.510 --> 01:10:24.798 They just give up on stability for a little bit, 01:10:24.798 --> 01:10:26.340 until they go back to the place where 01:10:26.340 --> 01:10:28.560 they can catch on the ground. 01:10:28.560 --> 01:10:29.317 OK. 01:10:29.317 --> 01:10:31.650 That's relatively unappealing compared to these I think, 01:10:31.650 --> 01:10:33.435 because it uses a ton of energy. 01:10:33.435 --> 01:10:36.060 I told you in the first lecture it uses 20 times as much energy 01:10:36.060 --> 01:10:37.477 as a human when it walks. 01:10:37.477 --> 01:10:39.060 People have used these type of methods 01:10:39.060 --> 01:10:42.030 to make a robot that uses the same kind of energy economy 01:10:42.030 --> 01:10:43.618 as a human. 01:10:43.618 --> 01:10:45.910 And it's just walking with its foot flat on the ground. 01:10:45.910 --> 01:10:47.460 It doesn't work on rough terrain It's 01:10:47.460 --> 01:10:49.440 not walking as fast as we are. 01:10:49.440 --> 01:10:51.240 All the energetics of heel strike, 01:10:51.240 --> 01:10:53.460 you have to do all that with active control 01:10:53.460 --> 01:10:55.890 because you can't sort of reduce your collision if you're 01:10:55.890 --> 01:10:59.310 going to land with a flat foot, so this is a better way 01:10:59.310 --> 01:11:02.190 to do things. 01:11:02.190 --> 01:11:05.400 Excellent That's a quick preview into the world of walking. 01:11:05.400 --> 01:11:07.320 And really the key message here is 01:11:07.320 --> 01:11:11.040 it only takes one or two more tools to turn it back 01:11:11.040 --> 01:11:12.200 into the acrobat problem. 01:11:12.200 --> 01:11:12.700 OK. 01:11:15.720 --> 01:11:16.680 Midterm on Thursday. 01:11:16.680 --> 01:11:18.472 If you have questions about that, ask John. 01:11:18.472 --> 01:11:19.410 Ask me. 01:11:19.410 --> 01:11:22.140 If I do disappear, I apologize. 01:11:22.140 --> 01:11:24.480 But it's for a good reason. 01:11:24.480 --> 01:11:27.690 And we'll have a good spring break. 01:11:27.690 --> 01:11:29.500 And I'll see you soon.