1 00:00:01,585 --> 00:00:03,910 The following content is provided under a Creative 2 00:00:03,910 --> 00:00:05,300 Commons license. 3 00:00:05,300 --> 00:00:07,510 Your support will help MIT OpenCourseWare 4 00:00:07,510 --> 00:00:11,600 continue to offer high quality educational resources for free. 5 00:00:11,600 --> 00:00:14,140 To make a donation or to view additional materials 6 00:00:14,140 --> 00:00:18,100 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:18,100 --> 00:00:18,980 at ocw.mit.edu. 8 00:00:24,220 --> 00:00:29,810 PROFESSOR: OK, let's go ahead and get started again. 9 00:00:29,810 --> 00:00:31,760 So we had a nice review on Monday 10 00:00:31,760 --> 00:00:33,350 of some of the different concepts 11 00:00:33,350 --> 00:00:35,930 that we've talked about up to this point in the course. 12 00:00:35,930 --> 00:00:38,990 We've covered linear algebra, solutions of systems 13 00:00:38,990 --> 00:00:41,554 of nonlinear equations. 14 00:00:41,554 --> 00:00:43,720 There's been a lot of fundamentals involved in that. 15 00:00:43,720 --> 00:00:46,629 Things are going to get more applied 16 00:00:46,629 --> 00:00:48,170 as the sorts of problems we're trying 17 00:00:48,170 --> 00:00:50,510 to solve get more complicated. 18 00:00:50,510 --> 00:00:53,420 Complicated problems require very sophisticated methods 19 00:00:53,420 --> 00:00:55,280 to solve them. 20 00:00:55,280 --> 00:00:57,320 It's not going to be our expectation that you're 21 00:00:57,320 --> 00:00:59,240 necessarily able to implement all 22 00:00:59,240 --> 00:01:00,690 of these sophisticated methods. 23 00:01:00,690 --> 00:01:03,466 So for example, you implemented Newton's method 24 00:01:03,466 --> 00:01:04,840 on your last homework assignment. 25 00:01:04,840 --> 00:01:07,800 That'll be the last time we ask you to do that by hand. 26 00:01:07,800 --> 00:01:10,400 You'll utilize tools built into Matlab 27 00:01:10,400 --> 00:01:13,340 to solve other systems of nonlinear equations going 28 00:01:13,340 --> 00:01:14,340 forward. 29 00:01:14,340 --> 00:01:17,150 You understand how it works and what the limitations are. 30 00:01:17,150 --> 00:01:19,250 You understand how to build different tools that 31 00:01:19,250 --> 00:01:21,560 will be inputs to these functions 32 00:01:21,560 --> 00:01:24,740 like the function you're trying to find the root for, 33 00:01:24,740 --> 00:01:28,640 or an explicit expression for the Jacobian. 34 00:01:28,640 --> 00:01:32,000 You even understand how to set certain parameters associated 35 00:01:32,000 --> 00:01:34,910 with these methods, and what those particular parameter 36 00:01:34,910 --> 00:01:35,660 values mean. 37 00:01:35,660 --> 00:01:37,760 Like setting the function norm tolerance, 38 00:01:37,760 --> 00:01:40,040 or setting the step norm tolerance 39 00:01:40,040 --> 00:01:41,565 associated with these methods. 40 00:01:41,565 --> 00:01:43,190 Setting the number of iterations you're 41 00:01:43,190 --> 00:01:44,300 going to let the method go. 42 00:01:44,300 --> 00:01:45,966 You've implemented it, so you understand 43 00:01:45,966 --> 00:01:48,740 what these things mean, what the consequences of choosing them 44 00:01:48,740 --> 00:01:50,780 are, how they might increase the time associated 45 00:01:50,780 --> 00:01:55,280 with computations that you're going to conduct. 46 00:01:55,280 --> 00:01:56,974 So we've done a lot of fundamentals. 47 00:01:56,974 --> 00:01:58,640 Maybe that was difficult for some of you 48 00:01:58,640 --> 00:02:00,680 who haven't seen this before. 49 00:02:00,680 --> 00:02:03,740 Now things are becoming more about the practice 50 00:02:03,740 --> 00:02:04,690 of numerical methods. 51 00:02:04,690 --> 00:02:06,770 How do we actually solve things reliably, 52 00:02:06,770 --> 00:02:09,680 and that's what today's lecture will talk about. 53 00:02:09,680 --> 00:02:12,350 We discussed solving systems of nonlinear equations 54 00:02:12,350 --> 00:02:16,140 and needing good initial guesses for our methods. 55 00:02:16,140 --> 00:02:19,340 Bad initial guesses can lead to unpredictable results. 56 00:02:19,340 --> 00:02:22,460 Good initial guesses, we know the Newton-Raphson method 57 00:02:22,460 --> 00:02:25,640 is going to double the number of accurate digits each iteration, 58 00:02:25,640 --> 00:02:27,870 and we should converge very rapidly to a solution. 59 00:02:27,870 --> 00:02:30,680 So it seems like having good initial guesses is 60 00:02:30,680 --> 00:02:31,950 really ideal. 61 00:02:31,950 --> 00:02:35,870 And this topic, homotopy and bifurcation, 62 00:02:35,870 --> 00:02:39,140 will cover precisely those issues, 63 00:02:39,140 --> 00:02:41,540 and the practice of getting these good initial guesses. 64 00:02:41,540 --> 00:02:42,860 I think is one of the most useful things you'll 65 00:02:42,860 --> 00:02:44,510 learn in this class, actually. 66 00:02:44,510 --> 00:02:46,790 It's a really nice way of going about solving 67 00:02:46,790 --> 00:02:48,950 complicated problems. 68 00:02:48,950 --> 00:02:53,000 Before we begin, I want to address a question 69 00:02:53,000 --> 00:02:54,909 I got via email last night. 70 00:02:54,909 --> 00:02:56,450 That's always good if these questions 71 00:02:56,450 --> 00:02:59,600 go on [? Piaza ?] so everyone can read them, 72 00:02:59,600 --> 00:03:03,044 but I thought it was a very nice question about how 73 00:03:03,044 --> 00:03:04,460 do you find-- the question was how 74 00:03:04,460 --> 00:03:07,680 do you find multiple roots using the Newton-Raphson method. 75 00:03:07,680 --> 00:03:11,750 So if I have a polynomial, where the function 76 00:03:11,750 --> 00:03:13,369 doesn't cross through the origin, 77 00:03:13,369 --> 00:03:15,410 but it just touches the origin, and goes back up. 78 00:03:15,410 --> 00:03:18,077 That's a root with multiplicity in the polynomial. 79 00:03:18,077 --> 00:03:19,910 And can the Newton-Raphson method find that? 80 00:03:19,910 --> 00:03:23,820 Because the derivative of the function is 0 at the root. 81 00:03:23,820 --> 00:03:25,760 So who says the Newton-Raphson method will 82 00:03:25,760 --> 00:03:27,231 be able to find those roots? 83 00:03:27,231 --> 00:03:27,980 What do you think? 84 00:03:27,980 --> 00:03:29,590 Yes? 85 00:03:29,590 --> 00:03:32,730 And who says it won't be able to find those roots? 86 00:03:32,730 --> 00:03:34,265 Or who doesn't know? 87 00:03:34,265 --> 00:03:34,890 You don't know. 88 00:03:34,890 --> 00:03:35,831 That's OK. 89 00:03:35,831 --> 00:03:37,080 That's a good question, right. 90 00:03:37,080 --> 00:03:38,820 Do you know or don't you know? 91 00:03:38,820 --> 00:03:40,980 So we know the Newton-Raphson method 92 00:03:40,980 --> 00:03:44,160 relies on knowing the derivative of the function 93 00:03:44,160 --> 00:03:47,190 at different points in order to take 94 00:03:47,190 --> 00:03:51,280 our next Newton-Raphson step towards that solution. 95 00:03:51,280 --> 00:03:54,540 It may be the case, the derivative is 0 at the root, 96 00:03:54,540 --> 00:03:59,109 but nearby the root, the derivative may not be 0. 97 00:03:59,109 --> 00:04:01,400 And then the Newton-Raphson steps are going to be fine. 98 00:04:01,400 --> 00:04:04,690 They're are always going to keep stepping down towards the root. 99 00:04:04,690 --> 00:04:08,940 If you go through an analysis of how the algorithm converges, 100 00:04:08,940 --> 00:04:11,970 you'll find out that you only get linear convergence 101 00:04:11,970 --> 00:04:14,350 in that case, instead of quadratic convergence. 102 00:04:14,350 --> 00:04:17,640 So there will be some penalty to pay for the fact 103 00:04:17,640 --> 00:04:20,399 that the derivative is 0 at the root 104 00:04:20,399 --> 00:04:22,890 itself, but it's not going to be so bad. 105 00:04:22,890 --> 00:04:26,865 The method can still converge to those sorts of solutions. 106 00:04:26,865 --> 00:04:29,610 Not all is lost there, but the rate of convergence 107 00:04:29,610 --> 00:04:32,650 may be not as good as you expected it to be. 108 00:04:32,650 --> 00:04:35,600 And the same is true in the multi-dimensional case as well. 109 00:04:35,600 --> 00:04:38,569 It's a little hard to imagine what a multiple root is 110 00:04:38,569 --> 00:04:39,360 going to look like. 111 00:04:39,360 --> 00:04:41,460 It's something like tangent curves 112 00:04:41,460 --> 00:04:42,660 in the solution of plane. 113 00:04:42,660 --> 00:04:46,680 So if I have f1 of x1 and x2 equal 0, 114 00:04:46,680 --> 00:04:48,720 that's a curve in the x1, x2 plane. 115 00:04:48,720 --> 00:04:52,140 And I have f2 in that same plane equal to 0. 116 00:04:52,140 --> 00:04:54,850 The place where they touch might be one of these multiple roots. 117 00:04:54,850 --> 00:04:58,450 That's a place where the determinant of the Jacobian 118 00:04:58,450 --> 00:04:59,751 may be equal to 0. 119 00:04:59,751 --> 00:05:01,500 You'll have the same sort of circumstance. 120 00:05:01,500 --> 00:05:04,260 Nearby, the Jacobian can be non singular, 121 00:05:04,260 --> 00:05:06,244 and you can take steps towards the root, 122 00:05:06,244 --> 00:05:08,160 but there's a worst case scenario where you're 123 00:05:08,160 --> 00:05:11,900 coming in on a direction that's parallel 124 00:05:11,900 --> 00:05:15,232 that belongs to the null space of the Jacobian at the root, 125 00:05:15,232 --> 00:05:16,940 and then everything's going to slow down. 126 00:05:16,940 --> 00:05:18,660 You can still converge, but it may 127 00:05:18,660 --> 00:05:20,780 be very slow as a consequence. 128 00:05:20,780 --> 00:05:22,530 I thought that was a really nice question. 129 00:05:22,530 --> 00:05:25,180 It's a detailed question about how Newton-Raphson works. 130 00:05:25,180 --> 00:05:27,330 It's not necessarily going to fail 131 00:05:27,330 --> 00:05:28,860 under these weird circumstances. 132 00:05:28,860 --> 00:05:30,210 They don't come up too often. 133 00:05:30,210 --> 00:05:33,480 They're going to come up in our discussion today, actually. 134 00:05:33,480 --> 00:05:36,720 It's not going to fail, but it may converge very slowly 135 00:05:36,720 --> 00:05:38,430 to the solution. 136 00:05:38,430 --> 00:05:39,390 OK? 137 00:05:39,390 --> 00:05:40,480 Good. 138 00:05:40,480 --> 00:05:42,230 I'm going to remind you, the last thing we 139 00:05:42,230 --> 00:05:44,180 did with Newton-Raphson methods, we 140 00:05:44,180 --> 00:05:45,770 talked about one way of fixing them. 141 00:05:45,770 --> 00:05:47,960 In lieu of having good initial guesses, 142 00:05:47,960 --> 00:05:50,600 we may not want to take the full Newton-Raphson step. 143 00:05:50,600 --> 00:05:53,870 We may want to do these quasi Newton-Raphson methods instead. 144 00:05:53,870 --> 00:05:55,970 One of those was this damp Newton-Raphson 145 00:05:55,970 --> 00:05:58,740 where we introduced some damping parameter. 146 00:05:58,740 --> 00:06:01,640 We don't take the full step, we take some shorter step, 147 00:06:01,640 --> 00:06:04,490 and we try to make the absolute value over function 148 00:06:04,490 --> 00:06:06,409 become as small as possible. 149 00:06:06,409 --> 00:06:07,700 That's an optimization problem. 150 00:06:07,700 --> 00:06:11,390 It's as hard as finding the root of the system of equations 151 00:06:11,390 --> 00:06:12,810 that we're looking at. 152 00:06:12,810 --> 00:06:14,990 So we don't solve that optimization problem exactly. 153 00:06:14,990 --> 00:06:16,070 That's crazy. 154 00:06:16,070 --> 00:06:19,100 Instead we use this backtracking line search method 155 00:06:19,100 --> 00:06:22,400 where we change our damping parameter 156 00:06:22,400 --> 00:06:25,220 in some systematic way until we get to a point 157 00:06:25,220 --> 00:06:26,570 where we're satisfied. 158 00:06:26,570 --> 00:06:30,410 Where we say OK, in the process of changing this, 159 00:06:30,410 --> 00:06:33,290 we've actually found a smaller value of our function, 160 00:06:33,290 --> 00:06:36,740 and we accept that as our best guess 161 00:06:36,740 --> 00:06:39,890 for where the next iterate is supposed to go. 162 00:06:39,890 --> 00:06:42,050 And this sort of approach can give you 163 00:06:42,050 --> 00:06:44,570 global convergence for the algorithm, which is great, 164 00:06:44,570 --> 00:06:47,570 but it's not globally convergent to roots. 165 00:06:47,570 --> 00:06:53,330 It's globally convergent to minima, asymptotes, and roots 166 00:06:53,330 --> 00:06:54,052 as well. 167 00:06:54,052 --> 00:06:55,260 But it's globally convergent. 168 00:06:55,260 --> 00:06:56,390 So it'll terminate. 169 00:06:56,390 --> 00:06:59,000 The algorithm will stop, and when it stops, 170 00:06:59,000 --> 00:07:02,190 you can ask is this a root or not? 171 00:07:02,190 --> 00:07:04,860 Do I think this is sufficiently close to a root or not? 172 00:07:04,860 --> 00:07:06,632 If it's not sufficiently close to a root, 173 00:07:06,632 --> 00:07:08,840 then you know you terminated somewhere like a minima, 174 00:07:08,840 --> 00:07:10,535 or you rushed out towards an asymptote, 175 00:07:10,535 --> 00:07:13,220 and maybe you need to start with a new initial guess 176 00:07:13,220 --> 00:07:14,706 for your solution. 177 00:07:14,706 --> 00:07:17,070 These quasi Newton-Raphson methods are very useful. 178 00:07:17,070 --> 00:07:19,100 This is what Matlab uses in order to-- 179 00:07:19,100 --> 00:07:20,600 it uses a more sophisticated version 180 00:07:20,600 --> 00:07:22,100 of this called the Dogleg method, 181 00:07:22,100 --> 00:07:23,920 but it's the same principle. 182 00:07:23,920 --> 00:07:25,430 It's trying to make sure you're not 183 00:07:25,430 --> 00:07:27,386 taking steps that are too big. 184 00:07:27,386 --> 00:07:28,760 Trying to make sure you're taking 185 00:07:28,760 --> 00:07:33,890 steps in the right direction so that you're approaching 186 00:07:33,890 --> 00:07:35,340 the roots of your function. 187 00:07:35,340 --> 00:07:37,910 Or at least reducing the absolute value of your function 188 00:07:37,910 --> 00:07:39,752 as close to zero as you can get it. 189 00:07:39,752 --> 00:07:41,210 Are there any questions about this? 190 00:07:41,210 --> 00:07:43,584 Anything we need to follow up on in the previous lecture? 191 00:07:46,545 --> 00:07:49,340 No. 192 00:07:49,340 --> 00:07:50,900 OK, good initial guesses. 193 00:07:50,900 --> 00:07:52,460 This is the key, really, for solving 194 00:07:52,460 --> 00:07:54,470 systems of nonlinear equations. 195 00:07:54,470 --> 00:07:56,860 It's the key for doing optimization problems too. 196 00:07:56,860 --> 00:08:01,380 We need to have some idea of what our solution looks like, 197 00:08:01,380 --> 00:08:05,280 and we want guesses that are close to those solutions 198 00:08:05,280 --> 00:08:09,186 so that maybe we are in this region of local convergence 199 00:08:09,186 --> 00:08:11,060 for the Newton-Raphson method, and everything 200 00:08:11,060 --> 00:08:12,410 converges very quickly. 201 00:08:12,410 --> 00:08:15,170 So where do they come from? 202 00:08:15,170 --> 00:08:17,780 This is an important question, and I'll 203 00:08:17,780 --> 00:08:19,409 show you how you can get them. 204 00:08:19,409 --> 00:08:21,200 There's another issue too, which is related 205 00:08:21,200 --> 00:08:23,420 to this, which is the non-linear equations, 206 00:08:23,420 --> 00:08:25,317 they can have multiple roots. 207 00:08:25,317 --> 00:08:27,650 If we do optimization problems-- that's our next topic-- 208 00:08:27,650 --> 00:08:29,930 they can have multiple minima or maxima 209 00:08:29,930 --> 00:08:31,850 that we're trying to seek out. 210 00:08:31,850 --> 00:08:34,429 Depending on our initial guess, our numerical method 211 00:08:34,429 --> 00:08:36,940 may find one of these roots or another root. 212 00:08:36,940 --> 00:08:39,049 It may find one of these minima or another minima. 213 00:08:39,049 --> 00:08:41,130 Maybe we're looking for the global minimum, 214 00:08:41,130 --> 00:08:43,130 so we need to look at them all. 215 00:08:43,130 --> 00:08:46,480 Are there methods that we can use to find them all? 216 00:08:46,480 --> 00:08:48,920 So we'll talk about continuation in homotopy, 217 00:08:48,920 --> 00:08:50,354 and we'll talk about bifurcation. 218 00:08:50,354 --> 00:08:51,770 And it turns out there are methods 219 00:08:51,770 --> 00:08:54,860 one can use to find all the roots for a system 220 00:08:54,860 --> 00:08:55,980 of non-linear equations. 221 00:08:55,980 --> 00:08:58,490 In fact, the homework assignment you have this week 222 00:08:58,490 --> 00:09:02,600 will ask you to do ternary phase equilibrium problem. 223 00:09:02,600 --> 00:09:06,140 Vapor liquid phase equilibrium with three components. 224 00:09:06,140 --> 00:09:07,760 There are five parts to this problem. 225 00:09:07,760 --> 00:09:10,090 Four of them, I think, are relatively straightforward. 226 00:09:10,090 --> 00:09:12,080 They're about finding the pure component roots 227 00:09:12,080 --> 00:09:15,470 or randomly guessing some initial conditions 228 00:09:15,470 --> 00:09:18,290 for your nonlinear solver in order 229 00:09:18,290 --> 00:09:22,640 to find all of the different azeotropes in the system. 230 00:09:22,640 --> 00:09:25,430 There's a fifth part that asks you to actually implement 231 00:09:25,430 --> 00:09:29,150 this homotopy method, and try to find all the roots in one 232 00:09:29,150 --> 00:09:30,109 go with your algorithm. 233 00:09:30,109 --> 00:09:31,608 Like the other homeworks you've had, 234 00:09:31,608 --> 00:09:33,110 that part's a little more tricky. 235 00:09:33,110 --> 00:09:34,520 Usually these homeworks have one thing 236 00:09:34,520 --> 00:09:36,110 that's a little bit more tricky than everything else. 237 00:09:36,110 --> 00:09:37,940 That part is a little bit more tricky, 238 00:09:37,940 --> 00:09:40,630 but it's really instructive to try to do it. 239 00:09:40,630 --> 00:09:42,230 We'll learn how to do it today. 240 00:09:42,230 --> 00:09:45,950 You can find all of the coexisting compositions 241 00:09:45,950 --> 00:09:49,400 of this ternary phase diagram, simultaneously, 242 00:09:49,400 --> 00:09:51,484 with one go through this algorithm. 243 00:09:54,270 --> 00:09:56,490 So let's talk about continuation first. 244 00:09:56,490 --> 00:09:59,270 This is an interesting idea, and it 245 00:09:59,270 --> 00:10:01,910 relates to trying to find roots of these equations, 246 00:10:01,910 --> 00:10:03,200 having good initial guesses. 247 00:10:03,200 --> 00:10:04,850 Here's a simple example. 248 00:10:04,850 --> 00:10:06,990 We have a third degree polynomial, 249 00:10:06,990 --> 00:10:09,290 and it has three roots. 250 00:10:09,290 --> 00:10:12,530 We can look, graphically, because this is one dimension, 251 00:10:12,530 --> 00:10:16,040 and see that one of these roots is near minus 1 and 1/2, 252 00:10:16,040 --> 00:10:19,460 another root is near a 1/2, and another root is near 1. 253 00:10:19,460 --> 00:10:21,830 And so I can go to my non-linear equation solver, 254 00:10:21,830 --> 00:10:25,130 and give it initial guesses, minus 1/2, 1/2, and 1, 255 00:10:25,130 --> 00:10:28,240 and I'm probably going to find these roots pretty reliably. 256 00:10:28,240 --> 00:10:31,220 But in many dimensions, we can't do these sorts 257 00:10:31,220 --> 00:10:32,990 of graphical approaches. 258 00:10:32,990 --> 00:10:35,160 You can't look at this function. 259 00:10:35,160 --> 00:10:36,830 It may be difficult to actually get 260 00:10:36,830 --> 00:10:40,040 enough data to even look at it if it's a two-dimensional set 261 00:10:40,040 --> 00:10:41,420 of functions. 262 00:10:41,420 --> 00:10:44,420 It's hard to see where these roots are, 263 00:10:44,420 --> 00:10:47,570 and so continuation is a method of transforming the problem 264 00:10:47,570 --> 00:10:50,780 into an easier one to solve. 265 00:10:50,780 --> 00:10:54,952 So let's take this problem, this third order polynomial, 266 00:10:54,952 --> 00:10:55,910 and let's transform it. 267 00:10:55,910 --> 00:11:00,545 Let's replace the 2 with a 2 times the lambda, 268 00:11:00,545 --> 00:11:02,670 and let's try to find the roots of this polynomials 269 00:11:02,670 --> 00:11:05,230 as lambda grows from 0 to 1. 270 00:11:08,310 --> 00:11:10,650 This is sort of a weird idea, but we actually 271 00:11:10,650 --> 00:11:15,720 know when lambda is equal to 0, this goes away, 272 00:11:15,720 --> 00:11:18,030 and there's only one root to this polynomial-- one 273 00:11:18,030 --> 00:11:19,860 real root-- which is minus 1. 274 00:11:19,860 --> 00:11:24,150 We actually know the answer when lambda is equal to 0. 275 00:11:24,150 --> 00:11:27,990 If I make lambda a little bit bigger than 0, 276 00:11:27,990 --> 00:11:31,380 there will be a root which is close to minus 1. 277 00:11:31,380 --> 00:11:34,390 In fact, minus 1 is a good initial guess for that root, 278 00:11:34,390 --> 00:11:37,220 and my non-linear equation solver 279 00:11:37,220 --> 00:11:39,890 ought to converge very quickly to that value. 280 00:11:39,890 --> 00:11:41,910 If I choose lambda to be small, then it 281 00:11:41,910 --> 00:11:44,580 should be in that region of local convergence 282 00:11:44,580 --> 00:11:46,860 of my Newton-Raphson method, and I very quickly 283 00:11:46,860 --> 00:11:48,340 should find the next root. 284 00:11:48,340 --> 00:11:50,670 So I can step lambda along. 285 00:11:50,670 --> 00:11:55,140 I use the lambda equals 0 root as an initial guess 286 00:11:55,140 --> 00:11:57,870 for the next root, and I keep increasing lambda 287 00:11:57,870 --> 00:12:01,896 from 0 all the way up to 1, and when lambda is equal to 1, 288 00:12:01,896 --> 00:12:04,020 I'm going to find the root to the initial problem I 289 00:12:04,020 --> 00:12:05,250 was looking at. 290 00:12:05,250 --> 00:12:06,630 Does that makes sense? 291 00:12:06,630 --> 00:12:10,590 So I'm continuing from one set of solutions to the next. 292 00:12:10,590 --> 00:12:13,290 I vary this parameter continuously, 293 00:12:13,290 --> 00:12:15,510 and for previous values of that parameter, 294 00:12:15,510 --> 00:12:19,750 they serve as initial guesses for the next solution. 295 00:12:19,750 --> 00:12:22,370 Here's what a code that does that would look like. 296 00:12:22,370 --> 00:12:24,440 I define my lambdas. 297 00:12:24,440 --> 00:12:27,225 We don't know good ways to define the lambda. 298 00:12:27,225 --> 00:12:28,850 I've got to figure that out for myself, 299 00:12:28,850 --> 00:12:31,550 but here lambda is a vector that goes from 0 to 1, 300 00:12:31,550 --> 00:12:33,230 in 1/100 increments. 301 00:12:33,230 --> 00:12:35,690 Maybe that's too fine, but that's OK. 302 00:12:35,690 --> 00:12:38,540 And my initial guess for the solution when lambda equals 0 303 00:12:38,540 --> 00:12:42,920 will be minus 1, and I loop over all the values of lambda, 304 00:12:42,920 --> 00:12:46,640 and I use f0, which is the one-dimensional non-linear 305 00:12:46,640 --> 00:12:48,290 equation solver in Matlab, to find 306 00:12:48,290 --> 00:12:50,780 the roots of this polynomial function-- 307 00:12:50,780 --> 00:12:54,740 x cubed minus 2 lambda x plus 1. 308 00:12:54,740 --> 00:12:57,170 Here's the solution and that becomes my initial guess 309 00:12:57,170 --> 00:12:58,130 for my next solution. 310 00:12:58,130 --> 00:13:00,255 So this loop is going to go round and round until I 311 00:13:00,255 --> 00:13:02,570 get lambda equals 1, and the solution that 312 00:13:02,570 --> 00:13:05,030 results there should be the solution to the initial problem 313 00:13:05,030 --> 00:13:07,594 I was interested in. 314 00:13:07,594 --> 00:13:09,260 Here's what that looks like graphically. 315 00:13:09,260 --> 00:13:11,840 I started with lambda equals 0. 316 00:13:11,840 --> 00:13:14,420 The root there serves as an initial guess for lambda 317 00:13:14,420 --> 00:13:17,434 equals 1/100, and so on, and these roots 318 00:13:17,434 --> 00:13:19,100 converge eventually, to something that's 319 00:13:19,100 --> 00:13:22,780 close to minus 1 and 1/2. 320 00:13:22,780 --> 00:13:25,960 So the actual value is a little bit bigger or a little bit 321 00:13:25,960 --> 00:13:28,900 smaller than minus 1.6. 322 00:13:28,900 --> 00:13:30,890 Here's one of the roots. 323 00:13:30,890 --> 00:13:33,640 So rather than having to have a precise initial guess, 324 00:13:33,640 --> 00:13:36,280 I transformed my problem from an easy to solve one 325 00:13:36,280 --> 00:13:38,320 to a more difficult to solve one, 326 00:13:38,320 --> 00:13:42,190 and I used the path along that transformation 327 00:13:42,190 --> 00:13:44,560 to give me rapid convergence to better and better 328 00:13:44,560 --> 00:13:47,290 initial guesses to the problem that I was interested in. 329 00:13:47,290 --> 00:13:47,860 Make sense? 330 00:13:50,760 --> 00:13:53,880 There's another way of doing this too. 331 00:13:53,880 --> 00:13:56,630 You don't have to go from 0 to 1. 332 00:13:56,630 --> 00:13:59,970 Let's go from lambda equals something big back down to 1 333 00:13:59,970 --> 00:14:01,830 instead. 334 00:14:01,830 --> 00:14:05,820 So if lambda's big, this term dominates 335 00:14:05,820 --> 00:14:07,704 the polynomial equation. 336 00:14:07,704 --> 00:14:09,870 This is the biggest term in the polynomial equation, 337 00:14:09,870 --> 00:14:13,920 and it can either balance x cubed, or it can balance 1. 338 00:14:13,920 --> 00:14:16,820 It can't balance against both of them, it turns out. 339 00:14:16,820 --> 00:14:18,420 That 340 00:14:18,420 --> 00:14:23,850 So if we have this term large, and it balances against 1, then 341 00:14:23,850 --> 00:14:26,170 when lambda's large, an approximation for the root 342 00:14:26,170 --> 00:14:27,750 will be 1 over 2 lambda. 343 00:14:27,750 --> 00:14:30,540 If x equals 1 over 2 lambda, and these are the two largest 344 00:14:30,540 --> 00:14:33,100 terms in the equation, then these two will cancel, 345 00:14:33,100 --> 00:14:34,730 and f will be very close to 0. 346 00:14:34,730 --> 00:14:37,400 So that's one root when lambda is very large. 347 00:14:37,400 --> 00:14:39,650 If I balance these other two terms against each other, 348 00:14:39,650 --> 00:14:41,358 I'll see there are two other roots, which 349 00:14:41,358 --> 00:14:43,890 are plus or minus the square root of 2 lambda. 350 00:14:43,890 --> 00:14:47,910 So let's start with a large lambda instead, and these 351 00:14:47,910 --> 00:14:49,830 as initial guesses for the roots-- 352 00:14:49,830 --> 00:14:52,737 and let's trace them back to lambda equals 1. 353 00:14:52,737 --> 00:14:54,570 When lambda equals 1, we recover the problem 354 00:14:54,570 --> 00:14:57,090 we were interested in before. 355 00:14:57,090 --> 00:14:59,340 So here, I start with three different initial guesses, 356 00:14:59,340 --> 00:15:02,340 and I trace them back, and I'm actually 357 00:15:02,340 --> 00:15:05,494 able to find all three roots of the initial equation. 358 00:15:05,494 --> 00:15:07,410 So there's something close to minus 1 and 1/2, 359 00:15:07,410 --> 00:15:10,410 something close to 1/2, and something close to 1. 360 00:15:10,410 --> 00:15:16,125 So in one go, I found all three roots using continuation. 361 00:15:16,125 --> 00:15:17,000 Is this clear so far? 362 00:15:17,000 --> 00:15:17,499 Sam. 363 00:15:17,499 --> 00:15:22,484 STUDENT: Is that one go or is it three separate [INAUDIBLE].. 364 00:15:22,484 --> 00:15:24,150 PROFESSOR: That's an excellent question. 365 00:15:24,150 --> 00:15:26,310 So I had to initiate this process with three 366 00:15:26,310 --> 00:15:29,950 separate guesses, but it turned out in this case, 367 00:15:29,950 --> 00:15:33,420 all three guesses converged to three distinct roots 368 00:15:33,420 --> 00:15:35,800 in the end. 369 00:15:35,800 --> 00:15:37,530 it's one go in the sense that there's 370 00:15:37,530 --> 00:15:40,350 one trajectory that I had to follow, 371 00:15:40,350 --> 00:15:42,250 one passive lambda as I had to go through, 372 00:15:42,250 --> 00:15:44,487 but I had to solve three times to follow 373 00:15:44,487 --> 00:15:45,570 each of these three paths. 374 00:15:50,480 --> 00:15:53,642 There's actually no reason to stop at lambda equals 1. 375 00:15:53,642 --> 00:15:56,100 I stopped there because that's the solution to the problem. 376 00:15:56,100 --> 00:15:58,490 I'm interested in, but there's actually something richer 377 00:15:58,490 --> 00:15:59,060 to see here. 378 00:15:59,060 --> 00:16:03,530 If I keep decreasing lambda, eventually these two branches 379 00:16:03,530 --> 00:16:06,800 combine with each other, and then they jump down, 380 00:16:06,800 --> 00:16:09,230 and they follow the original branch that I 381 00:16:09,230 --> 00:16:13,370 had that went between minus 1 at lambda equals 0, 382 00:16:13,370 --> 00:16:18,180 and this root about minus 1.6 at lambda equals 1. 383 00:16:18,180 --> 00:16:21,140 So the paths that these trajectories follow 384 00:16:21,140 --> 00:16:22,820 can be very, very complicated, actually. 385 00:16:22,820 --> 00:16:25,710 A little bit unwieldy or unpredictable. 386 00:16:25,710 --> 00:16:30,050 But oftentimes, they can result in predictions of all three 387 00:16:30,050 --> 00:16:31,692 roots or multiple roots associated 388 00:16:31,692 --> 00:16:32,900 with the system of equations. 389 00:16:32,900 --> 00:16:33,400 Yeah. 390 00:16:33,400 --> 00:16:36,394 STUDENT: Going back to the first [INAUDIBLE] solve 391 00:16:36,394 --> 00:16:41,130 from lambda equals [INAUDIBLE] 392 00:16:41,130 --> 00:16:42,231 PROFESSOR: Zero to one? 393 00:16:42,231 --> 00:16:43,133 STUDENT: Zero to one. 394 00:16:43,133 --> 00:16:46,025 So then would you find only one solution there? 395 00:16:46,025 --> 00:16:46,650 PROFESSOR: Yes. 396 00:16:46,650 --> 00:16:49,350 So going from 0 to 1, I found one solution. 397 00:16:49,350 --> 00:16:51,570 Going from infinity-- 398 00:16:51,570 --> 00:16:52,560 10 is big enough. 399 00:16:52,560 --> 00:16:54,905 10 to 1, I was able to find three. 400 00:16:54,905 --> 00:16:56,280 I didn't know beforehand that I'm 401 00:16:56,280 --> 00:16:57,910 going to be able to do that. 402 00:16:57,910 --> 00:17:00,210 It happened to be the case that we could 403 00:17:00,210 --> 00:17:03,040 with this by construction. 404 00:17:03,040 --> 00:17:06,000 This is a particular problem where it works out that way. 405 00:17:06,000 --> 00:17:06,800 Other questions? 406 00:17:09,720 --> 00:17:10,594 So this is neat. 407 00:17:10,594 --> 00:17:12,510 Like we've done before with iterative methods, 408 00:17:12,510 --> 00:17:16,589 we turn hard to solve problems into a sequence of easier 409 00:17:16,589 --> 00:17:17,710 to solve problems. 410 00:17:17,710 --> 00:17:21,060 Good initial guesses converge fast with Newton-Raphson. 411 00:17:21,060 --> 00:17:24,690 We have a sequence of better and better initial guesses, which 412 00:17:24,690 --> 00:17:29,160 all converge very quickly to eventually, the solution 413 00:17:29,160 --> 00:17:32,196 that we're looking for evaluated at lambda equals 1. 414 00:17:32,196 --> 00:17:34,320 So this is one way of getting good initial guesses. 415 00:17:34,320 --> 00:17:37,230 You have a sequence of good initial guesses. 416 00:17:37,230 --> 00:17:40,100 Make sense? 417 00:17:40,100 --> 00:17:41,080 So we're just here. 418 00:17:41,080 --> 00:17:43,810 We had this polynomial problem and we injected lambda 419 00:17:43,810 --> 00:17:44,930 in a particular place. 420 00:17:44,930 --> 00:17:47,530 We injected it there because I knew it was going to work. 421 00:17:47,530 --> 00:17:49,154 I knew the problem could be transformed 422 00:17:49,154 --> 00:17:51,790 in this way in order to work out and distinguish this. 423 00:17:51,790 --> 00:17:56,200 It's not always obvious how to transform a problem 424 00:17:56,200 --> 00:17:59,410 to make it look this way, but oftentimes we 425 00:17:59,410 --> 00:18:02,380 have physical problems in which there's a parameter 426 00:18:02,380 --> 00:18:04,960 that we can do continuation with that is obvious. 427 00:18:04,960 --> 00:18:08,380 So think about fluid mechanics problems. 428 00:18:08,380 --> 00:18:11,770 Those are nonlinear partial differential equations. 429 00:18:11,770 --> 00:18:14,590 So we're trying to solve say, the Navier-Stokes equations. 430 00:18:14,590 --> 00:18:18,980 Rather than jump in and try to solve that right away, 431 00:18:18,980 --> 00:18:20,050 we might do continuation. 432 00:18:20,050 --> 00:18:23,470 We might find a solution at very small Reynolds numbers, 433 00:18:23,470 --> 00:18:25,390 where the equations are almost linear, 434 00:18:25,390 --> 00:18:27,760 exact solutions are known in that case, 435 00:18:27,760 --> 00:18:31,870 and we might progressively step the Reynolds number up, and use 436 00:18:31,870 --> 00:18:36,535 our solution set small Reynolds numbers as initial guesses 437 00:18:36,535 --> 00:18:39,160 for increasingly large Reynolds numbers until we get to the one 438 00:18:39,160 --> 00:18:40,690 that we're interested in. 439 00:18:40,690 --> 00:18:43,300 You can imagine doing that in mass transfer problems 440 00:18:43,300 --> 00:18:45,520 where you vary the Peclet number. 441 00:18:45,520 --> 00:18:47,390 In multicomponent phase equilibria problems, 442 00:18:47,390 --> 00:18:49,060 this might happen by varying temperature 443 00:18:49,060 --> 00:18:51,360 or pressure until we get to the temperature or pressure 444 00:18:51,360 --> 00:18:53,020 that we're interested in. 445 00:18:53,020 --> 00:18:55,300 If we have reaction equilibria problems, 446 00:18:55,300 --> 00:18:57,610 we might vary the different reaction rates 447 00:18:57,610 --> 00:18:59,890 until we get to the right balance of reaction rates 448 00:18:59,890 --> 00:19:02,830 to determine the equilibrium compositions. 449 00:19:02,830 --> 00:19:05,380 So they're often natural parameters in the problem 450 00:19:05,380 --> 00:19:07,630 that we're free to adjust. 451 00:19:07,630 --> 00:19:10,320 We can transform from easy to solve problems 452 00:19:10,320 --> 00:19:12,850 to hard to solve problems in a continuous fashion. 453 00:19:15,620 --> 00:19:19,770 It's not always going to work out the way that I showed. 454 00:19:19,770 --> 00:19:23,220 It's not always going to be so nice and neat. 455 00:19:23,220 --> 00:19:25,441 It may be quite a bit more complicated, 456 00:19:25,441 --> 00:19:27,190 and sometimes there are problems for which 457 00:19:27,190 --> 00:19:32,080 there isn't a natural parameter available to vary. 458 00:19:32,080 --> 00:19:35,770 In those cases, there's another strategy one can employ. 459 00:19:35,770 --> 00:19:38,440 It's called homotopy. 460 00:19:38,440 --> 00:19:41,270 This is the transformation from one problem 461 00:19:41,270 --> 00:19:45,310 into another problem in a continuous fashion. 462 00:19:45,310 --> 00:19:48,790 So oftentimes, we're seeking a set of roots. 463 00:19:48,790 --> 00:19:52,044 x star, which are a function of a parameter lambda, 464 00:19:52,044 --> 00:19:54,460 and they're the roots of an equation that looks like this. 465 00:19:54,460 --> 00:19:57,340 So they're the roots of this function, h, 466 00:19:57,340 --> 00:19:59,590 which is a linear superposition of a function 467 00:19:59,590 --> 00:20:02,770 f and a function g. 468 00:20:02,770 --> 00:20:07,870 When lambda equals 0, h is equal to g, and so the roots of h 469 00:20:07,870 --> 00:20:09,760 are the roots of g. 470 00:20:09,760 --> 00:20:13,450 And when lambda equals 1, h is equal 471 00:20:13,450 --> 00:20:19,260 to f, and so the roots of h are the roots of f. 472 00:20:19,260 --> 00:20:22,470 So we might transform-- we make very lambda continuously 473 00:20:22,470 --> 00:20:25,740 between 0 and 1 in a construction like this. 474 00:20:25,740 --> 00:20:28,530 So we might transform the roots from the roots of g 475 00:20:28,530 --> 00:20:30,570 into the roots of f. 476 00:20:30,570 --> 00:20:34,290 Maybe the roots of g are easy to find, but the roots of f 477 00:20:34,290 --> 00:20:35,610 are hard for us to find. 478 00:20:38,300 --> 00:20:41,790 There's a smooth transformation though, from g to f. 479 00:20:41,790 --> 00:20:45,740 When we have those sorts of smooth transformations, 480 00:20:45,740 --> 00:20:47,900 there's a chance things will be well-behaved, 481 00:20:47,900 --> 00:20:50,390 or at least where we encounter bad behavior-- 482 00:20:50,390 --> 00:20:53,000 and I'll show you what I mean by bad behavior in a minute-- 483 00:20:53,000 --> 00:20:54,830 there's a reliable way to interpret 484 00:20:54,830 --> 00:20:56,550 what's going on there. 485 00:20:56,550 --> 00:20:59,630 So we vary lambda in small increments from 0 to 1, 486 00:20:59,630 --> 00:21:04,190 and the solution x star for some lambda i 487 00:21:04,190 --> 00:21:06,140 is used as the initial guess to find 488 00:21:06,140 --> 00:21:08,750 the roots at some lambda, which is a little bit bigger 489 00:21:08,750 --> 00:21:09,830 than lambda i-- 490 00:21:09,830 --> 00:21:11,420 our next iterate in lambda. 491 00:21:14,130 --> 00:21:15,860 Does this idea make sense? 492 00:21:15,860 --> 00:21:17,120 Where do f and g come from? 493 00:21:20,127 --> 00:21:21,960 So this is a big question-- f is the problem 494 00:21:21,960 --> 00:21:24,240 you want to solve, typically, and g 495 00:21:24,240 --> 00:21:26,610 is some auxiliary problem. 496 00:21:26,610 --> 00:21:29,440 It can be arbitrarily chosen. 497 00:21:29,440 --> 00:21:31,590 There's nothing to tell you why one g should 498 00:21:31,590 --> 00:21:34,290 be better than another one, but oftentimes, we 499 00:21:34,290 --> 00:21:37,620 choose them in a physical way. 500 00:21:37,620 --> 00:21:39,930 So there will be a physical connection 501 00:21:39,930 --> 00:21:41,880 between the f problem and the g problem, 502 00:21:41,880 --> 00:21:44,730 and I'll show you that right now. 503 00:21:44,730 --> 00:21:46,110 So here's an example. 504 00:21:46,110 --> 00:21:47,550 We looked at this example before. 505 00:21:47,550 --> 00:21:49,466 We want to find the roots of the van der Waals 506 00:21:49,466 --> 00:21:50,550 equation of state. 507 00:21:50,550 --> 00:21:53,190 It's written in terms of the reduced pressure, the reduced 508 00:21:53,190 --> 00:21:55,430 molar volume, and the reduced temperature, 509 00:21:55,430 --> 00:22:00,040 and let's find those roots for a given pressure and temperature. 510 00:22:00,040 --> 00:22:04,620 So this is a function f of the molar volume equals 0, 511 00:22:04,620 --> 00:22:07,410 and we're looking for these three roots here. 512 00:22:07,410 --> 00:22:13,441 There's a root down here, which is the molar volume is low. 513 00:22:13,441 --> 00:22:15,690 Not a lot of volume per number of moles, so maybe this 514 00:22:15,690 --> 00:22:17,280 is a liquid like root. 515 00:22:17,280 --> 00:22:18,870 Out here, the molar volume is high. 516 00:22:18,870 --> 00:22:21,519 We have a lot of volume, the number of moles and materials. 517 00:22:21,519 --> 00:22:23,310 This is a vapor like root, and then there's 518 00:22:23,310 --> 00:22:26,390 an unstable root in between them. 519 00:22:26,390 --> 00:22:27,770 So three roots to find. 520 00:22:27,770 --> 00:22:30,105 We'd like to be able to find them in one go, 521 00:22:30,105 --> 00:22:32,480 or we'd like to have some reliable mechanisms for finding 522 00:22:32,480 --> 00:22:33,440 them. 523 00:22:33,440 --> 00:22:35,250 Homotopy is one way to do this. 524 00:22:35,250 --> 00:22:38,150 So let's construct a new function h of the molar 525 00:22:38,150 --> 00:22:41,360 volume, which is lambda times f. 526 00:22:41,360 --> 00:22:46,400 We want to find the roots of f plus 1 minus lambda times g. 527 00:22:46,400 --> 00:22:49,020 And as g, let's use something physical. 528 00:22:49,020 --> 00:22:52,700 So let's let g be a function that represents the ideal gas 529 00:22:52,700 --> 00:22:54,320 equation of state. 530 00:22:54,320 --> 00:22:57,650 So PV is nRT, but I've written everything 531 00:22:57,650 --> 00:23:00,560 in terms of the reduced pressure, molar volume, 532 00:23:00,560 --> 00:23:03,040 and temperature of the van der Waals equation of states. 533 00:23:03,040 --> 00:23:05,690 So you pick up this factor of 8/3 there. 534 00:23:05,690 --> 00:23:08,630 But this is the ideal gas equation of state. 535 00:23:08,630 --> 00:23:10,609 Its roots will be the roots of the ideal gas 536 00:23:10,609 --> 00:23:11,400 equation of states. 537 00:23:11,400 --> 00:23:14,600 I'm going to transform from something that's easy to solve. 538 00:23:14,600 --> 00:23:18,080 I know how to find the molar volume here, there's just one, 539 00:23:18,080 --> 00:23:20,360 to something that's hard for us to solve. 540 00:23:20,360 --> 00:23:22,160 It has three roots-- 541 00:23:22,160 --> 00:23:24,560 the roots of them van der Waals equation of state. 542 00:23:24,560 --> 00:23:26,450 So when lambda equals 0, I'll get the roots associated 543 00:23:26,450 --> 00:23:28,220 with the ideal gas, and then lambda equals 1, 544 00:23:28,220 --> 00:23:30,386 I'll get the roots associated with the van der Waals 545 00:23:30,386 --> 00:23:31,864 equation. 546 00:23:31,864 --> 00:23:33,292 Yes? 547 00:23:33,292 --> 00:23:36,148 STUDENT: If you're starting with lambda as 0 548 00:23:36,148 --> 00:23:39,836 and you get the roots of g, you only get one root, 549 00:23:39,836 --> 00:23:42,180 but you're trying to find two roots [INAUDIBLE].. 550 00:23:42,180 --> 00:23:43,560 So how does that work? 551 00:23:43,560 --> 00:23:44,601 PROFESSOR: I'll show you. 552 00:23:49,890 --> 00:23:51,620 Here's what the process might look like. 553 00:23:51,620 --> 00:23:54,846 I've got to start with a reduced temperature and pressure. 554 00:23:54,846 --> 00:23:56,720 Here's my initial guess for the molar volume. 555 00:23:59,280 --> 00:24:01,840 It's the root of our ideal gas equation. 556 00:24:01,840 --> 00:24:04,940 Here's the definition of f, and the definition of g, 557 00:24:04,940 --> 00:24:06,960 and the definition of h. 558 00:24:06,960 --> 00:24:10,160 l times f plus 1 minus l times g, 559 00:24:10,160 --> 00:24:12,510 and I'm going to loop over all of my lambdas, 560 00:24:12,510 --> 00:24:15,030 solve the equation subject to some initial guess, 561 00:24:15,030 --> 00:24:18,090 and update my initial guess every time I update lambda. 562 00:24:20,631 --> 00:24:21,130 That's fine. 563 00:24:21,130 --> 00:24:21,963 That's the code you. 564 00:24:21,963 --> 00:24:24,529 Can implement the code if you want or play around with it. 565 00:24:24,529 --> 00:24:26,320 So what happens if I try to carry this out? 566 00:24:26,320 --> 00:24:28,750 So I'm going to vary lambda between 0 and 1. 567 00:24:28,750 --> 00:24:32,350 I start with my initial ideal gas guess for the root, 568 00:24:32,350 --> 00:24:34,780 and as lambda grows to 1, the molar volume 569 00:24:34,780 --> 00:24:37,540 shrinks until I get to lambda equals 1, 570 00:24:37,540 --> 00:24:40,680 and I've found one root of my equation. 571 00:24:40,680 --> 00:24:42,730 One root, good. 572 00:24:42,730 --> 00:24:45,250 There's not necessarily any reason to stop at lambda 573 00:24:45,250 --> 00:24:46,940 equals 1. 574 00:24:46,940 --> 00:24:48,580 That's the root I wanted, but there's 575 00:24:48,580 --> 00:24:50,950 no reason I have to stop there. 576 00:24:50,950 --> 00:24:52,030 So I could continue. 577 00:24:52,030 --> 00:24:54,424 If I keep pushing lambda up higher and higher, 578 00:24:54,424 --> 00:24:56,590 eventually I'll get to a point where all of a sudden 579 00:24:56,590 --> 00:24:59,770 the molar volume jumps down to a different place. 580 00:24:59,770 --> 00:25:01,270 This is the result of the algorithm. 581 00:25:01,270 --> 00:25:03,970 I just told lambda to go bigger. 582 00:25:03,970 --> 00:25:06,190 But there will be a discontinuity or a cusp 583 00:25:06,190 --> 00:25:08,860 here that I jump off of, and I find myself 584 00:25:08,860 --> 00:25:14,800 on a different solution branch, which is sort of crazy. 585 00:25:14,800 --> 00:25:17,330 So now I can do something clever. 586 00:25:17,330 --> 00:25:19,850 I found myself on a different solution branch. 587 00:25:19,850 --> 00:25:24,260 Why not try decreasing lambda along that solution branch 588 00:25:24,260 --> 00:25:24,909 instead? 589 00:25:24,909 --> 00:25:26,450 Maybe I'll find a different solution, 590 00:25:26,450 --> 00:25:28,250 and it turns out that's what happens. 591 00:25:28,250 --> 00:25:30,583 All right, so i jump down to this other solution branch, 592 00:25:30,583 --> 00:25:34,490 and now I try decreasing lambda from 2 back down to 1, 593 00:25:34,490 --> 00:25:38,670 and sure enough, I found the vapor molar volume, 594 00:25:38,670 --> 00:25:40,400 and now I found the liquid molar volume. 595 00:25:42,894 --> 00:25:44,810 There's no reason to stop and lambda equals 1. 596 00:25:44,810 --> 00:25:48,856 So if I keep decreasing lambda, eventually I'll 597 00:25:48,856 --> 00:25:50,480 hit another cusp, and I'll jump back up 598 00:25:50,480 --> 00:25:53,030 to the original solution branch. 599 00:25:53,030 --> 00:25:55,320 So I do my homotopy. 600 00:25:55,320 --> 00:25:58,100 I vary lambda from 0 to 1, and I find one root, 601 00:25:58,100 --> 00:26:01,220 but if I keep going, I might identify a cusp, 602 00:26:01,220 --> 00:26:03,290 and jump off that cross. 603 00:26:03,290 --> 00:26:06,980 I might reverse my direction with the homotopy, 604 00:26:06,980 --> 00:26:08,490 and I could find another root. 605 00:26:08,490 --> 00:26:11,894 I might hit another cusp as well. 606 00:26:11,894 --> 00:26:13,310 Now, when I hit a cusp, maybe it's 607 00:26:13,310 --> 00:26:15,740 best not to jump off the cusp, but instead 608 00:26:15,740 --> 00:26:17,930 to reverse direction at the cusp, 609 00:26:17,930 --> 00:26:19,830 and try to trace it out the other direction. 610 00:26:19,830 --> 00:26:22,360 And if I do that, if I get right to this last point, 611 00:26:22,360 --> 00:26:24,050 and I try to reverse direction again, 612 00:26:24,050 --> 00:26:27,020 I can trace out one more solution path, 613 00:26:27,020 --> 00:26:28,940 and I can find the third root to the equation. 614 00:26:28,940 --> 00:26:32,300 So in one go, if I'm paying attention 615 00:26:32,300 --> 00:26:35,930 to how my roots are changing-- if they change by too much, 616 00:26:35,930 --> 00:26:40,390 I might switch the direction with which I change lambda, 617 00:26:40,390 --> 00:26:45,050 and I could find all of these roots at once. 618 00:26:45,050 --> 00:26:47,370 So three roots in one go. 619 00:26:49,901 --> 00:26:50,400 Yes. 620 00:26:50,400 --> 00:26:51,288 STUDENT: Is this like trial and error? 621 00:26:51,288 --> 00:26:53,160 How do you know when to change? 622 00:26:53,160 --> 00:26:55,770 PROFESSOR: This is a wonderful question. 623 00:26:55,770 --> 00:26:59,100 So in this case, you can do it by trial and error. 624 00:26:59,100 --> 00:27:01,110 You can detect these jumps. 625 00:27:01,110 --> 00:27:02,280 That's even better. 626 00:27:02,280 --> 00:27:04,470 You look for jumps in the value of the solution that 627 00:27:04,470 --> 00:27:07,500 are of sufficient magnitude, and when you detect them, 628 00:27:07,500 --> 00:27:09,042 you can reverse the direction. 629 00:27:09,042 --> 00:27:10,500 There's an even more systematic way 630 00:27:10,500 --> 00:27:13,109 to do this, which is called arclength continuation. 631 00:27:13,109 --> 00:27:14,650 Give me one second, and I'll give you 632 00:27:14,650 --> 00:27:16,440 slide on arclength continuation, which 633 00:27:16,440 --> 00:27:20,640 is a systematic way of doing exactly this process of tracing 634 00:27:20,640 --> 00:27:24,480 out this path in the solution plane. 635 00:27:24,480 --> 00:27:27,810 Roots versus homotopy parameter. 636 00:27:27,810 --> 00:27:32,300 These cusps here, are sometimes called turning points, 637 00:27:32,300 --> 00:27:34,190 and they have a special property, 638 00:27:34,190 --> 00:27:39,810 which is the determinant of the Jacobian of the homotopy-- 639 00:27:39,810 --> 00:27:44,170 the homotopy function-- is going to be equal to 0. 640 00:27:44,170 --> 00:27:47,440 The Jacobian is going to be singular right at these cusps. 641 00:27:47,440 --> 00:27:50,740 So you can detect them by tracking what 642 00:27:50,740 --> 00:27:52,580 the Jacobian is doing right. 643 00:27:52,580 --> 00:27:54,580 The Jacobian is going to be non-singular at most 644 00:27:54,580 --> 00:27:56,038 of these other points, but when you 645 00:27:56,038 --> 00:28:00,070 hit one of these turning points, the Jacobian will be singular. 646 00:28:00,070 --> 00:28:03,770 There isn't a well-defined direction for the homotopy 647 00:28:03,770 --> 00:28:05,450 to proceed along. 648 00:28:05,450 --> 00:28:07,120 It hits this cusp where the slope 649 00:28:07,120 --> 00:28:09,640 is infinite in this plane. 650 00:28:09,640 --> 00:28:12,340 There isn't a well-defined direction to step along here. 651 00:28:15,290 --> 00:28:16,980 The question was-- 652 00:28:16,980 --> 00:28:19,670 OK, it seems a little ad-hoc. 653 00:28:19,670 --> 00:28:21,470 You've got this 2D plane, and it's 654 00:28:21,470 --> 00:28:24,500 easy to guess when you jump from one solution to another, 655 00:28:24,500 --> 00:28:26,930 and trace the path back. 656 00:28:26,930 --> 00:28:27,716 That's fine. 657 00:28:27,716 --> 00:28:29,090 There's a systematic way of doing 658 00:28:29,090 --> 00:28:31,790 this, which is called arclength continuation, which 659 00:28:31,790 --> 00:28:36,890 says I know that there's some curve in this solution plane. 660 00:28:36,890 --> 00:28:38,330 There is some curve here. 661 00:28:38,330 --> 00:28:40,260 I've shown it to you graphically. 662 00:28:40,260 --> 00:28:43,160 Let's try to parametrize that curve in terms 663 00:28:43,160 --> 00:28:45,380 of a measure of the arclength. 664 00:28:45,380 --> 00:28:49,100 Call it s, the distance along the curve. 665 00:28:49,100 --> 00:28:52,740 So the roots are a function of lambda, 666 00:28:52,740 --> 00:28:55,160 which can be a function of the distance along the curve, 667 00:28:55,160 --> 00:28:58,610 and lambda is a function of the distance along the curve, 668 00:28:58,610 --> 00:29:01,700 and there is a constraint which says 669 00:29:01,700 --> 00:29:04,370 s has to satisfy an arclength formula. 670 00:29:04,370 --> 00:29:08,030 This is the formula for the arclength in this plane 671 00:29:08,030 --> 00:29:10,200 of roots versus lambda. 672 00:29:10,200 --> 00:29:12,360 The derivative of this function with respect to s 673 00:29:12,360 --> 00:29:15,575 squared plus the derivative of lambda with respect to s 674 00:29:15,575 --> 00:29:16,565 squared equals 1. 675 00:29:16,565 --> 00:29:20,570 That defines s as a measure of length along this curve. 676 00:29:20,570 --> 00:29:24,050 You learned arclength formulas in the context of integration 677 00:29:24,050 --> 00:29:24,680 at one point. 678 00:29:24,680 --> 00:29:29,490 This is the differential version of that same formula. 679 00:29:29,490 --> 00:29:31,260 So I've got a differential equation. 680 00:29:31,260 --> 00:29:33,480 Well, it's actually a differential algebraic 681 00:29:33,480 --> 00:29:35,130 equation. 682 00:29:35,130 --> 00:29:37,380 This is an algebraic constraint that 683 00:29:37,380 --> 00:29:40,620 needs to be satisfied for all values of s, 684 00:29:40,620 --> 00:29:43,789 and it depends on how the roots change with s. 685 00:29:43,789 --> 00:29:45,330 We'll learn how to solve differential 686 00:29:45,330 --> 00:29:49,500 algebraic equations in the two sections from now. 687 00:29:49,500 --> 00:29:52,290 So we don't quite know how to solve something like this yet, 688 00:29:52,290 --> 00:29:54,498 but if we know how to solve those sorts of equations, 689 00:29:54,498 --> 00:29:57,150 then it's clear that we can trace out 690 00:29:57,150 --> 00:29:58,920 this entire curve in the solution plane, 691 00:29:58,920 --> 00:30:01,378 and then go back and try to find the places where the curve 692 00:30:01,378 --> 00:30:04,060 intersects lambda equals 1. 693 00:30:04,060 --> 00:30:07,650 So if we follow this path by solving this equation from s 694 00:30:07,650 --> 00:30:11,364 equals 0 out to some very large s, 695 00:30:11,364 --> 00:30:13,530 then we should be able to find all of the roots that 696 00:30:13,530 --> 00:30:15,175 are connected to the path. 697 00:30:15,175 --> 00:30:17,155 Yes. 698 00:30:17,155 --> 00:30:20,620 STUDENT: If there's a turning point that [INAUDIBLE] then 699 00:30:20,620 --> 00:30:23,590 does that imply multiplicity? 700 00:30:23,590 --> 00:30:25,090 PROFESSOR: Yes. 701 00:30:25,090 --> 00:30:28,730 So it's a real problem. 702 00:30:28,730 --> 00:30:30,590 That turning point singularity is actually 703 00:30:30,590 --> 00:30:33,470 built into this arclength formula as well. 704 00:30:33,470 --> 00:30:37,340 This is the derivative of x with respect 705 00:30:37,340 --> 00:30:41,240 to s squared, which takes the direction you're 706 00:30:41,240 --> 00:30:43,070 moving in out of the problem. 707 00:30:43,070 --> 00:30:45,750 This is a length, not a direction. 708 00:30:45,750 --> 00:30:48,050 So when we get to these turning points, 709 00:30:48,050 --> 00:30:49,769 the solution methods for these equations 710 00:30:49,769 --> 00:30:51,560 are going to have to know that I was headed 711 00:30:51,560 --> 00:30:54,500 in a downward direction first. 712 00:30:54,500 --> 00:30:56,750 I can get to this turning point, and then the question 713 00:30:56,750 --> 00:30:59,490 is which direction do I move in from the turning point? 714 00:30:59,490 --> 00:31:01,480 Do I go up or do I go down? 715 00:31:01,480 --> 00:31:03,590 Which way does the solution path change? 716 00:31:03,590 --> 00:31:06,920 The slope isn't defined there, so it's quite challenging 717 00:31:06,920 --> 00:31:09,470 to figure out exactly how to proceed. 718 00:31:09,470 --> 00:31:10,730 But it is what you say. 719 00:31:10,730 --> 00:31:15,620 It's like the solution has multiplicity at that point. 720 00:31:15,620 --> 00:31:20,110 Curves are tangent in the solution plane. 721 00:31:20,110 --> 00:31:21,680 It's a great question. 722 00:31:21,680 --> 00:31:25,606 OK, so that's the notion of arclength continuation. 723 00:31:25,606 --> 00:31:26,980 We may talk about this later when 724 00:31:26,980 --> 00:31:28,938 we talk about differential algebraic equations, 725 00:31:28,938 --> 00:31:31,780 and we'll talk about some equivalent sets of equations 726 00:31:31,780 --> 00:31:33,370 that we know how to solve, but they 727 00:31:33,370 --> 00:31:35,380 have to satisfy this constraint that s 728 00:31:35,380 --> 00:31:38,302 is a measure of arclength along the curve. 729 00:31:38,302 --> 00:31:39,010 But there you go. 730 00:31:39,010 --> 00:31:42,220 You can find all the solutions, at least all of the solutions 731 00:31:42,220 --> 00:31:46,270 connected to this path defined by the homotopy equation. 732 00:31:46,270 --> 00:31:48,340 You can find all of them in one go using 733 00:31:48,340 --> 00:31:50,262 some clever methodologies. 734 00:31:55,090 --> 00:31:58,580 Oftentimes, we're not just worried about these 735 00:31:58,580 --> 00:32:02,660 turning points or cusps, we encounter bifurcations 736 00:32:02,660 --> 00:32:03,480 in the solution. 737 00:32:03,480 --> 00:32:06,620 So we may have a path that has one root along it, 738 00:32:06,620 --> 00:32:08,450 and then that path may split, and there 739 00:32:08,450 --> 00:32:10,310 may be more than one root. 740 00:32:10,310 --> 00:32:12,380 Here's a simple example. 741 00:32:12,380 --> 00:32:16,880 Find the real roots of x cubed minus rx. 742 00:32:16,880 --> 00:32:21,360 If r is negative, there will always be one real root, 743 00:32:21,360 --> 00:32:24,230 and it'll be x equals 0. 744 00:32:24,230 --> 00:32:28,730 If r is equal to 0, there will still be one real root, it's 0. 745 00:32:28,730 --> 00:32:32,530 That root will have multiplicity 3 in that case. 746 00:32:32,530 --> 00:32:35,920 And if r becomes positive, I'll suddenly 747 00:32:35,920 --> 00:32:38,860 have three real roots instead of one. 748 00:32:38,860 --> 00:32:41,170 So as I vary the parameter r from a negative number 749 00:32:41,170 --> 00:32:45,250 to a positive number, I'll go from one real root to three. 750 00:32:45,250 --> 00:32:47,250 In the solution plane, that will look like this. 751 00:32:47,250 --> 00:32:49,670 Here are the roots as a function of r. 752 00:32:49,670 --> 00:32:50,900 r is less than 0. 753 00:32:50,900 --> 00:32:53,540 All the roots are equal to 0. 754 00:32:53,540 --> 00:32:56,630 When I hit r equals 0, I have multiplicity 755 00:32:56,630 --> 00:33:01,080 associated with the root, and the solution bifurcates, 756 00:33:01,080 --> 00:33:04,240 and there will be three possible solutions now. 757 00:33:04,240 --> 00:33:06,560 If I was trying to do a homotopy problem where I very r 758 00:33:06,560 --> 00:33:09,620 continuously from some negative number to some positive number, 759 00:33:09,620 --> 00:33:11,450 I'll hit this point, and I'll want 760 00:33:11,450 --> 00:33:16,170 to follow all three paths to find solutions out here 761 00:33:16,170 --> 00:33:18,340 at positive r. 762 00:33:18,340 --> 00:33:19,870 But I can detect those points. 763 00:33:19,870 --> 00:33:23,590 Just like I was said before, at these weird turning points 764 00:33:23,590 --> 00:33:27,670 or bifurcations where we have multiplicity in the roots, 765 00:33:27,670 --> 00:33:29,540 the Jacobian in is going to be singular. 766 00:33:29,540 --> 00:33:32,050 The Jacobian of the homotopy is going to be singular. 767 00:33:32,050 --> 00:33:33,620 Its determinant will be equal to 0. 768 00:33:33,620 --> 00:33:37,294 So I can detect these points by checking the determinant 769 00:33:37,294 --> 00:33:37,960 of the Jacobian. 770 00:33:37,960 --> 00:33:40,000 And when I find one, I want to do 771 00:33:40,000 --> 00:33:43,900 something smart to try to follow these paths. 772 00:33:43,900 --> 00:33:46,780 Your homework this week has a problem like that. 773 00:33:46,780 --> 00:33:51,430 You're trying to find all the azeotropes of this ternary 774 00:33:51,430 --> 00:33:55,120 vapor liquid equilibrium system, and you'll 775 00:33:55,120 --> 00:33:57,940 have a homotopy parameter that you change, 776 00:33:57,940 --> 00:33:59,320 and it'll bifurcate. 777 00:33:59,320 --> 00:34:01,240 OK, so as you change the parameter, 778 00:34:01,240 --> 00:34:04,090 you'll go from one path to two, and then those two paths 779 00:34:04,090 --> 00:34:05,410 will split again. 780 00:34:05,410 --> 00:34:09,280 You'll be splitting from a single component solution 781 00:34:09,280 --> 00:34:14,770 into a two component azeotrope, into a three component 782 00:34:14,770 --> 00:34:15,880 azeotrope. 783 00:34:15,880 --> 00:34:19,300 And you'll want to detect where these bifurcations occur, 784 00:34:19,300 --> 00:34:21,800 and try to follow solution branches off 785 00:34:21,800 --> 00:34:23,320 of those bifurcations, and there's 786 00:34:23,320 --> 00:34:25,403 some good hints in the problem for how to do that. 787 00:34:28,261 --> 00:34:29,969 So occasionally, we'll encounter problems 788 00:34:29,969 --> 00:34:32,880 that switch from having one solution to having 789 00:34:32,880 --> 00:34:35,269 many solutions. 790 00:34:35,269 --> 00:34:37,560 You've seen how this happens in the discontinuous sense 791 00:34:37,560 --> 00:34:40,300 with these turning points. 792 00:34:40,300 --> 00:34:44,219 Here, for a given value of lambda, 793 00:34:44,219 --> 00:34:47,520 all of a sudden it's clear that not only is there one solution 794 00:34:47,520 --> 00:34:49,440 here, but there's another solution down there. 795 00:34:49,440 --> 00:34:50,670 My path has to choose. 796 00:34:50,670 --> 00:34:53,580 Do I want this solution or the one below? 797 00:34:53,580 --> 00:34:55,170 I jump off this turning point. 798 00:34:55,170 --> 00:34:58,170 That's a sort of bifurcation. 799 00:34:58,170 --> 00:35:00,439 These are continuous bifurcations. 800 00:35:00,439 --> 00:35:01,980 They're often easier to take care of. 801 00:35:05,370 --> 00:35:06,870 So the bifurcations in a homotopy 802 00:35:06,870 --> 00:35:08,400 let us find multiple solutions. 803 00:35:08,400 --> 00:35:10,050 Every time we detect a bifurcation, 804 00:35:10,050 --> 00:35:13,230 if we just track those solution branches, 805 00:35:13,230 --> 00:35:15,750 we split our algorithm up to track each of the branches 806 00:35:15,750 --> 00:35:19,200 separately, they'll terminate at different roots 807 00:35:19,200 --> 00:35:21,450 to the homotopy equation. 808 00:35:21,450 --> 00:35:23,730 And when we get to the value of lambda equals 1 809 00:35:23,730 --> 00:35:25,364 in our homotopy, we're there. 810 00:35:25,364 --> 00:35:27,405 We found multiple roots to the original equation. 811 00:35:32,380 --> 00:35:34,990 These are often of great physical interest. 812 00:35:37,780 --> 00:35:40,628 Do you know what sort of things these bifurcations represent? 813 00:35:43,980 --> 00:35:47,160 One occurs in the van der Waals equation. 814 00:35:47,160 --> 00:35:49,470 So if I start at very high temperatures and a given 815 00:35:49,470 --> 00:35:51,600 pressure, the van der Waals equation 816 00:35:51,600 --> 00:35:56,030 will have how many real roots? 817 00:35:56,030 --> 00:35:58,430 One-- a gas. 818 00:35:58,430 --> 00:36:00,960 And then as I turn the temperature down, 819 00:36:00,960 --> 00:36:07,330 there is a propensity for liquefaction of that gas phase. 820 00:36:07,330 --> 00:36:10,120 The formation of a liquid in coexistence with the vapor. 821 00:36:10,120 --> 00:36:13,486 So I'll go from having one root to having three roots 822 00:36:13,486 --> 00:36:15,610 all of a sudden, and there will be this bifurcation 823 00:36:15,610 --> 00:36:17,830 point where that transition happens 824 00:36:17,830 --> 00:36:18,880 in a continuous fashion. 825 00:36:22,044 --> 00:36:23,460 STUDENT: Can I ask you a question? 826 00:36:23,460 --> 00:36:23,820 PROFESSOR: Yes. 827 00:36:23,820 --> 00:36:24,760 STUDENT: [INAUDIBLE] 828 00:36:24,760 --> 00:36:25,760 PROFESSOR: Yeah. 829 00:36:25,760 --> 00:36:28,236 STUDENT: So as you early put it out that the turning point, 830 00:36:28,236 --> 00:36:31,091 the determinant of the Jacobian [INAUDIBLE] including lambda 831 00:36:31,091 --> 00:36:32,350 is singular. 832 00:36:32,350 --> 00:36:34,050 But here you're writing j of x. 833 00:36:34,050 --> 00:36:36,450 Is that [INAUDIBLE] 834 00:36:36,450 --> 00:36:39,120 PROFESSOR: Excuse me. 835 00:36:39,120 --> 00:36:39,942 This is a typo. 836 00:36:39,942 --> 00:36:41,400 So there should be a little h here. 837 00:36:41,400 --> 00:36:44,940 This is the determinant of the Jacobian 838 00:36:44,940 --> 00:36:49,230 of the homotopy equation at that root. 839 00:36:49,230 --> 00:36:53,010 Where there's a bifurcation, the Jacobian will be singular. 840 00:36:53,010 --> 00:36:53,700 I apologize. 841 00:36:53,700 --> 00:36:54,360 That's a little bit-- 842 00:36:54,360 --> 00:36:55,360 STUDENT: And that lambda. 843 00:36:55,360 --> 00:36:56,484 PROFESSOR: And that lambda. 844 00:36:56,484 --> 00:37:02,042 STUDENT: Just to make it clear, you have n state variables. 845 00:37:02,042 --> 00:37:04,831 Like x is how many unknowns you have, 846 00:37:04,831 --> 00:37:07,250 and then you're adding another unknown to lambda, 847 00:37:07,250 --> 00:37:08,882 another parameter. 848 00:37:08,882 --> 00:37:12,630 This Jacobian has n plus 1 dimension. 849 00:37:12,630 --> 00:37:18,987 This is also [INAUDIBLE] Is this correct? 850 00:37:18,987 --> 00:37:19,820 PROFESSOR: Well, no. 851 00:37:19,820 --> 00:37:25,220 Actually, the Jacobian with respect to x only 852 00:37:25,220 --> 00:37:28,902 is also singular at these turning and bifurcation points. 853 00:37:28,902 --> 00:37:30,110 STUDENT: [INAUDIBLE] as well. 854 00:37:30,110 --> 00:37:31,534 PROFESSOR: Yes. 855 00:37:31,534 --> 00:37:33,950 It's also true what you said, that if I have the Jacobian, 856 00:37:33,950 --> 00:37:35,180 and I take the derivatives with respect 857 00:37:35,180 --> 00:37:36,650 to lambda, that will also be singular, 858 00:37:36,650 --> 00:37:38,775 but it will be singular because the subspace, which 859 00:37:38,775 --> 00:37:41,400 is the derivatives with respect to x, 860 00:37:41,400 --> 00:37:43,610 gives you a singular component. 861 00:37:46,460 --> 00:37:48,200 The condition is the same as here. 862 00:37:48,200 --> 00:37:52,220 So it's the Jacobian of h, taking the derivatives 863 00:37:52,220 --> 00:37:53,960 with respect to x-- 864 00:37:53,960 --> 00:37:56,660 all the different x's of h. 865 00:37:56,660 --> 00:37:58,970 The determinant of that matrix at a particular lambda 866 00:37:58,970 --> 00:38:01,250 will be equal to 0, and the value of lambda 867 00:38:01,250 --> 00:38:04,140 will be the lambda at which you have this turning point, 868 00:38:04,140 --> 00:38:05,540 or which you have a bifurcation. 869 00:38:09,500 --> 00:38:12,030 Here's what it looks like in a two-dimensional sense. 870 00:38:12,030 --> 00:38:14,680 So I'll have my homotopy equation 871 00:38:14,680 --> 00:38:17,770 where I'm trying to find the roots of h. 872 00:38:17,770 --> 00:38:20,100 Say it's a two-dimensional-- x is dimensions. 873 00:38:20,100 --> 00:38:22,990 So it has x1 and x2, and these curves 874 00:38:22,990 --> 00:38:25,720 represent h1 equals 0 and h2 equals 0, 875 00:38:25,720 --> 00:38:30,580 and they intersect at one point for a given value of lambda. 876 00:38:30,580 --> 00:38:32,780 There's the root. 877 00:38:32,780 --> 00:38:35,400 Now I increase the value of lambda. 878 00:38:35,400 --> 00:38:37,340 The homotopy equations change. 879 00:38:37,340 --> 00:38:39,930 So these curves change shape, and all of a sudden I 880 00:38:39,930 --> 00:38:43,500 go from crossing curves to tangent curves, 881 00:38:43,500 --> 00:38:46,380 and we know tangent curves in the plane 882 00:38:46,380 --> 00:38:50,304 will have a Jacobian which is singular. 883 00:38:50,304 --> 00:38:51,720 And this is the bifurcation point. 884 00:38:51,720 --> 00:38:53,580 These two curves become tangent, and as I 885 00:38:53,580 --> 00:38:59,120 go to a bigger value of lambda, they'll change shape again, 886 00:38:59,120 --> 00:39:01,250 and I'll now have multiple roots. 887 00:39:01,250 --> 00:39:05,060 I'll bifurcate from one solution to two, 888 00:39:05,060 --> 00:39:07,310 or one solution to three, but the bifurcation 889 00:39:07,310 --> 00:39:09,620 will occur through this transition 890 00:39:09,620 --> 00:39:11,810 where I have tangent curves, and we know there, 891 00:39:11,810 --> 00:39:14,030 the determinant of the Jacobian is 0. 892 00:39:14,030 --> 00:39:16,144 There's a singularity under those conditions. 893 00:39:16,144 --> 00:39:17,060 Does that makes sense? 894 00:39:24,571 --> 00:39:26,820 OK, so a couple of things about the practice, and I'll 895 00:39:26,820 --> 00:39:29,220 give you a simple example you can try out, which I think 896 00:39:29,220 --> 00:39:32,530 is an interesting one, if pathological. 897 00:39:32,530 --> 00:39:37,212 So in practice, it's hard to hit this bifurcation point exactly. 898 00:39:37,212 --> 00:39:38,670 And I'm telling you that you should 899 00:39:38,670 --> 00:39:40,025 be detecting these branches. 900 00:39:40,025 --> 00:39:41,400 You should follow these branches, 901 00:39:41,400 --> 00:39:43,920 but clearly, I've got to know where this point is in order 902 00:39:43,920 --> 00:39:46,590 to follow the branches. 903 00:39:46,590 --> 00:39:48,384 That's a problem. 904 00:39:48,384 --> 00:39:49,800 If the determinant of the Jacobian 905 00:39:49,800 --> 00:39:52,530 is 0 at the bifurcation point, then it's 906 00:39:52,530 --> 00:39:56,150 going to change from positive to negative as I 907 00:39:56,150 --> 00:39:58,560 cross the bifurcation point with lambda. 908 00:39:58,560 --> 00:40:00,109 Actually, I don't necessarily have 909 00:40:00,109 --> 00:40:02,400 to find the point where the determinant of the Jacobian 910 00:40:02,400 --> 00:40:03,190 is equal to 0. 911 00:40:03,190 --> 00:40:05,430 What I really should do is try to track 912 00:40:05,430 --> 00:40:07,690 the sign of the determinant of the Jacobian, 913 00:40:07,690 --> 00:40:09,150 and see when it goes from positive 914 00:40:09,150 --> 00:40:11,230 signed to negative signed. 915 00:40:11,230 --> 00:40:13,320 And when it does that, it will have 916 00:40:13,320 --> 00:40:15,690 crossed through one of these bifurcation points 917 00:40:15,690 --> 00:40:18,590 or one of these cusps. 918 00:40:18,590 --> 00:40:20,610 That's the point at which I should stop, 919 00:40:20,610 --> 00:40:23,790 and say, OK, there's possibly other solution branches 920 00:40:23,790 --> 00:40:25,530 to follow. 921 00:40:25,530 --> 00:40:28,320 So as I step from my smaller lambda to my bigger lambda, 922 00:40:28,320 --> 00:40:29,790 the sign changes. 923 00:40:29,790 --> 00:40:31,794 I want to step in some different directions 924 00:40:31,794 --> 00:40:33,460 than the direction I was heading before, 925 00:40:33,460 --> 00:40:35,940 and those different directions will trace out 926 00:40:35,940 --> 00:40:38,010 the other solution branches. 927 00:40:38,010 --> 00:40:40,560 It turns out those different directions 928 00:40:40,560 --> 00:40:45,090 belong to the null space of the Jacobian. 929 00:40:45,090 --> 00:40:46,590 So if you find vectors that are very 930 00:40:46,590 --> 00:40:49,590 close to the null space of the Jacobian, 931 00:40:49,590 --> 00:40:51,600 those directions are the directions 932 00:40:51,600 --> 00:40:54,439 you should go to find these other solution branches. 933 00:40:54,439 --> 00:40:55,480 So it's very interesting. 934 00:40:55,480 --> 00:40:55,660 Yeah. 935 00:40:55,660 --> 00:40:58,201 STUDENT: Can you have a Jacobian that goes from positive down 936 00:40:58,201 --> 00:41:00,590 to 0 and [INAUDIBLE]? 937 00:41:00,590 --> 00:41:03,150 PROFESSOR: That may be possible. 938 00:41:03,150 --> 00:41:05,880 You won't encounter that in the problem that you're doing. 939 00:41:05,880 --> 00:41:08,460 That would be a very difficult problem 940 00:41:08,460 --> 00:41:09,955 to do branch detection on. 941 00:41:09,955 --> 00:41:11,580 Let's Suppose that was the case and you 942 00:41:11,580 --> 00:41:14,910 want to find exactly where that bifurcation point is. 943 00:41:14,910 --> 00:41:17,350 These bifurcation points are of physical significance. 944 00:41:17,350 --> 00:41:18,974 In the van der Waals equation of state, 945 00:41:18,974 --> 00:41:21,500 that bifurcation point is the critical point. 946 00:41:21,500 --> 00:41:24,000 It's the critical temperature at which that phase separates. 947 00:41:24,000 --> 00:41:26,480 Maybe I want to know that exactly. 948 00:41:26,480 --> 00:41:29,610 Well, all I need to do is solve an augmented system 949 00:41:29,610 --> 00:41:30,910 of equations. 950 00:41:30,910 --> 00:41:33,790 So let's solve a system of equations, 951 00:41:33,790 --> 00:41:37,530 which is our original hematology equation equal to 0, 952 00:41:37,530 --> 00:41:40,860 and determinant of the Jacobian of this homotopy equation 953 00:41:40,860 --> 00:41:42,420 equal to 0. 954 00:41:42,420 --> 00:41:45,450 This is if h had dimension n. 955 00:41:45,450 --> 00:41:47,700 This is now n plus 1 equations, and let's 956 00:41:47,700 --> 00:41:49,350 solve it for n plus 1 unknowns. 957 00:41:49,350 --> 00:41:54,000 Let's solve it for x and for lambda. x has dimension n, 958 00:41:54,000 --> 00:41:56,070 and we add one more unknown-- 959 00:41:56,070 --> 00:41:58,410 the value of the homotopy parameter at which 960 00:41:58,410 --> 00:42:00,144 this bifurcation occurred. 961 00:42:00,144 --> 00:42:01,560 So we can find that point exactly. 962 00:42:01,560 --> 00:42:03,630 So there may be cases where this happens 963 00:42:03,630 --> 00:42:05,520 where it doesn't continuously transition 964 00:42:05,520 --> 00:42:06,910 from positive to negative. 965 00:42:06,910 --> 00:42:10,920 It may hit 0 and go back up, but that's 966 00:42:10,920 --> 00:42:15,130 a really challenging case to handle, but you can handle it. 967 00:42:15,130 --> 00:42:19,350 You can find precisely the point at which 968 00:42:19,350 --> 00:42:21,570 this bifurcation occurs by solving 969 00:42:21,570 --> 00:42:23,992 these augmented equations. 970 00:42:23,992 --> 00:42:25,950 It's just another non-linear equation to solve. 971 00:42:25,950 --> 00:42:26,450 Yes. 972 00:42:26,450 --> 00:42:30,416 STUDENT: I'm not sure about the previous slide. 973 00:42:30,416 --> 00:42:35,550 So in the previous slide, why is the [INAUDIBLE] 974 00:42:35,550 --> 00:42:37,630 PROFESSOR: Let me go back to here. 975 00:42:37,630 --> 00:42:40,900 So let's let the blue line here, be 976 00:42:40,900 --> 00:42:43,990 the place where the first element of h is equal to 0. 977 00:42:43,990 --> 00:42:46,690 So this is all the values of x at which 978 00:42:46,690 --> 00:42:48,970 the first element of our homotopy top equation 979 00:42:48,970 --> 00:42:51,110 is equal to 0. 980 00:42:51,110 --> 00:42:53,150 And the red curve here, let's let 981 00:42:53,150 --> 00:42:57,230 that be all the values of x at which the second element of h 982 00:42:57,230 --> 00:42:58,820 is equal to 0. 983 00:42:58,820 --> 00:43:03,170 And so h itself, is only equal in 0 where both of these curves 984 00:43:03,170 --> 00:43:06,730 are equal to 0, and that's their intersection. 985 00:43:06,730 --> 00:43:10,690 So they intersect here, and this star is the solution. 986 00:43:10,690 --> 00:43:14,530 It is the root of this non-linear equation up here. 987 00:43:14,530 --> 00:43:17,380 Now, I increase lambda. 988 00:43:17,380 --> 00:43:19,400 I make lambda a little bit bigger. 989 00:43:19,400 --> 00:43:21,070 That changes h. 990 00:43:21,070 --> 00:43:22,170 h is a function of lambda. 991 00:43:22,170 --> 00:43:27,260 So h changes, and these curves change shape in the plane. 992 00:43:27,260 --> 00:43:29,300 And at that bifurcation point, these two curves 993 00:43:29,300 --> 00:43:31,800 will become tangent to each other. 994 00:43:31,800 --> 00:43:34,970 There will be one root, but in this solution plane, 995 00:43:34,970 --> 00:43:37,220 the two curves will be tangent. 996 00:43:37,220 --> 00:43:38,660 And we saw before that when we get 997 00:43:38,660 --> 00:43:40,720 these sort of tangency conditions 998 00:43:40,720 --> 00:43:43,160 or the equivalent in higher dimensions, 999 00:43:43,160 --> 00:43:44,750 the Jacobian of our function we're 1000 00:43:44,750 --> 00:43:47,850 trying to find the roots of will be singular. 1001 00:43:47,850 --> 00:43:50,740 It's like a multiple root. 1002 00:43:50,740 --> 00:43:53,750 And when we increase lambda further, 1003 00:43:53,750 --> 00:43:56,397 curves change shape again. 1004 00:43:56,397 --> 00:43:58,730 I'm not changing the red one because that's complicated. 1005 00:43:58,730 --> 00:44:00,370 I just changed the blue one for you 1006 00:44:00,370 --> 00:44:01,760 as a graphical illustration. 1007 00:44:01,760 --> 00:44:03,680 So the blue one changes shape, and now I've 1008 00:44:03,680 --> 00:44:06,980 gone from having one solution to having three. 1009 00:44:06,980 --> 00:44:11,000 So in order to go from one solution to three, 1010 00:44:11,000 --> 00:44:12,560 in a continuous fashion, I have to go 1011 00:44:12,560 --> 00:44:16,730 through this tangency state, and that defines the bifurcation 1012 00:44:16,730 --> 00:44:17,685 point. 1013 00:44:17,685 --> 00:44:18,560 Does that make sense? 1014 00:44:23,220 --> 00:44:26,070 Here's a simple example you should try out. 1015 00:44:26,070 --> 00:44:27,750 So I'll present it to you, and then you 1016 00:44:27,750 --> 00:44:30,060 should try to solve it in Matlab. 1017 00:44:30,060 --> 00:44:32,920 You actually know the solution, geometrically, 1018 00:44:32,920 --> 00:44:34,140 so it's not too hard. 1019 00:44:34,140 --> 00:44:36,712 So here's a function. 1020 00:44:36,712 --> 00:44:38,170 The first element to that function, 1021 00:44:38,170 --> 00:44:41,304 it's a function of two unknowns-- x1 and x2. 1022 00:44:41,304 --> 00:44:42,720 The first element to that function 1023 00:44:42,720 --> 00:44:47,550 is an equation for a circle of radius r with center at minus 3 1024 00:44:47,550 --> 00:44:49,830 minus 1, and the second element is 1025 00:44:49,830 --> 00:44:56,310 an equation for a circle of radius r with center at 2, 2. 1026 00:44:56,310 --> 00:44:59,290 So there's two circles. 1027 00:44:59,290 --> 00:45:04,090 We want to find the radius where the circles just touch, 1028 00:45:04,090 --> 00:45:06,730 and that point is a bifurcation point. 1029 00:45:06,730 --> 00:45:11,050 If r is perfect, these two just touch. 1030 00:45:11,050 --> 00:45:14,650 If r is a little too big, the two circles overlap, 1031 00:45:14,650 --> 00:45:16,930 and there are two solutions to this problem. 1032 00:45:16,930 --> 00:45:19,240 And if r is a little too small, there 1033 00:45:19,240 --> 00:45:22,210 are no solutions to this problem. 1034 00:45:22,210 --> 00:45:25,180 So the perfect r for touching is a bifurcation point. 1035 00:45:25,180 --> 00:45:27,640 It has all the properties of a bifurcation point, 1036 00:45:27,640 --> 00:45:28,750 just like we discussed. 1037 00:45:28,750 --> 00:45:31,012 It's a multidimensional equation and it bifurcates. 1038 00:45:31,012 --> 00:45:33,220 You know the solution to this problem, geometrically. 1039 00:45:33,220 --> 00:45:35,780 You could work it out in a couple of minutes, 1040 00:45:35,780 --> 00:45:39,100 but you could also solve it using 1041 00:45:39,100 --> 00:45:40,840 these augmented equations. 1042 00:45:40,840 --> 00:45:44,500 So we know that point at which the circles just touch, 1043 00:45:44,500 --> 00:45:47,500 the value of r at which those circles just touch, 1044 00:45:47,500 --> 00:45:51,370 will be the place where f of x is equal to 0 1045 00:45:51,370 --> 00:45:54,720 at which these equations define a circle, 1046 00:45:54,720 --> 00:45:58,780 and at which the Jacobian of these equations-- 1047 00:45:58,780 --> 00:46:01,120 the derivatives of f with respect to x, it's determinant 1048 00:46:01,120 --> 00:46:02,980 is also equal to 0. 1049 00:46:02,980 --> 00:46:07,870 So we want to solve this system of three equations for three 1050 00:46:07,870 --> 00:46:08,440 unknowns. 1051 00:46:08,440 --> 00:46:11,980 The values of x1 and x2, where the circles just touch, 1052 00:46:11,980 --> 00:46:15,040 and the value of r, the radius of these circles 1053 00:46:15,040 --> 00:46:16,660 at which they just touch. 1054 00:46:16,660 --> 00:46:18,720 That would find the bifurcation point. 1055 00:46:18,720 --> 00:46:20,710 That will find the touching of the circles. 1056 00:46:20,710 --> 00:46:22,780 Can you see that, geometrically? 1057 00:46:22,780 --> 00:46:26,140 All these bifurcation problems work exactly that way. 1058 00:46:26,140 --> 00:46:28,060 There's some homotopy parameter. 1059 00:46:28,060 --> 00:46:30,430 In this case, that's r. 1060 00:46:30,430 --> 00:46:32,440 But there's some parameter we're varying as we 1061 00:46:32,440 --> 00:46:34,300 pass through that bifurcation. 1062 00:46:34,300 --> 00:46:37,630 We'd like to know the values of our unknowns 1063 00:46:37,630 --> 00:46:41,070 and the value of this parameter at that bifurcation point 1064 00:46:41,070 --> 00:46:44,110 or at that critical point. 1065 00:46:44,110 --> 00:46:46,440 So you find this point by solving 1066 00:46:46,440 --> 00:46:47,490 the augmented equations. 1067 00:46:47,490 --> 00:46:50,670 We have a non-linear equation-- three nonlinear equations 1068 00:46:50,670 --> 00:46:52,350 for three unknowns, and you can do 1069 00:46:52,350 --> 00:46:54,736 that with a Newton-Raphson iteration. 1070 00:46:54,736 --> 00:46:55,860 It's a non-linear equation. 1071 00:46:55,860 --> 00:46:59,860 We know Newton-Raphson is the way to solve these things. 1072 00:46:59,860 --> 00:47:03,030 To do Newton-Raphson, that means we need to compute 1073 00:47:03,030 --> 00:47:05,440 the Jacobian-- 1074 00:47:05,440 --> 00:47:06,350 this is confusing. 1075 00:47:06,350 --> 00:47:07,725 We've got to compute the Jacobian 1076 00:47:07,725 --> 00:47:10,870 of these augmented equations, the derivatives 1077 00:47:10,870 --> 00:47:12,790 of these augmented equations with respect 1078 00:47:12,790 --> 00:47:16,180 to our augmented unknowns. 1079 00:47:16,180 --> 00:47:18,760 So that will be the derivatives of f with respect 1080 00:47:18,760 --> 00:47:21,910 to x, the derivatives of f with respect 1081 00:47:21,910 --> 00:47:25,920 to r, the derivatives of the determinant of the Jacobian 1082 00:47:25,920 --> 00:47:27,060 with respect to x. 1083 00:47:27,060 --> 00:47:28,720 That's this gradient here. 1084 00:47:28,720 --> 00:47:31,000 The derivatives of the determinant of the Jacobian 1085 00:47:31,000 --> 00:47:31,930 with respect to r. 1086 00:47:31,930 --> 00:47:34,720 This is our augmented Jacobian. 1087 00:47:34,720 --> 00:47:37,960 The augmented Jacobian times the Newton-Raphson step. 1088 00:47:37,960 --> 00:47:40,210 That's equal to minus the function 1089 00:47:40,210 --> 00:47:43,439 we're trying to solve evaluated at the previous iteration. 1090 00:47:43,439 --> 00:47:44,980 Solve the system of equations, you'll 1091 00:47:44,980 --> 00:47:48,070 get your Newton-Raphson step, and you can find the solution 1092 00:47:48,070 --> 00:47:49,480 to the augmented equations. 1093 00:47:49,480 --> 00:47:52,250 It'll converge quadratically. 1094 00:47:52,250 --> 00:47:55,090 You should commit yourself, you can figure this out. 1095 00:47:55,090 --> 00:47:58,120 Don't try to compute all of these derivatives. 1096 00:47:58,120 --> 00:47:59,160 That's complicated. 1097 00:47:59,160 --> 00:48:00,770 You're going to make a mistake. 1098 00:48:00,770 --> 00:48:03,100 I'm going to make a mistake if I try to do that. 1099 00:48:03,100 --> 00:48:06,400 What do we do instead to compute the Jacobian? 1100 00:48:06,400 --> 00:48:09,170 How do we compute these derivatives? 1101 00:48:09,170 --> 00:48:10,450 What's that? 1102 00:48:10,450 --> 00:48:11,890 Finite difference, right. 1103 00:48:11,890 --> 00:48:13,930 Don't play games with it. 1104 00:48:13,930 --> 00:48:16,430 Try to use finite difference to compute the elements of this 1105 00:48:16,430 --> 00:48:17,986 instead. 1106 00:48:17,986 --> 00:48:19,610 You know the exact solution too, so you 1107 00:48:19,610 --> 00:48:20,984 can use that as an initial guess, 1108 00:48:20,984 --> 00:48:23,780 or something close to the exact solution as an initial guess 1109 00:48:23,780 --> 00:48:27,050 to see if you can find this bifurcation point. 1110 00:48:27,050 --> 00:48:29,090 So homotopy bifurcation, there are 1111 00:48:29,090 --> 00:48:32,450 ways to get good initial guesses by continuously 1112 00:48:32,450 --> 00:48:33,890 changing our problem. 1113 00:48:33,890 --> 00:48:35,810 We get a succession of problems that 1114 00:48:35,810 --> 00:48:38,920 will converge faster because we have good initial guesses, 1115 00:48:38,920 --> 00:48:42,140 and that will give us a reliable convergence, hopefully, to one 1116 00:48:42,140 --> 00:48:44,130 of the roots of the equation we're after. 1117 00:48:44,130 --> 00:48:47,840 If we're able to pick up solution branches, we can track 1118 00:48:47,840 --> 00:48:51,290 these cusps, or find these bifurcation points, 1119 00:48:51,290 --> 00:48:53,570 we might be able to find multiple solutions. 1120 00:48:53,570 --> 00:48:55,670 There are problems for which depending 1121 00:48:55,670 --> 00:48:59,090 on your choice of g in this homotopy function, 1122 00:48:59,090 --> 00:49:00,560 maybe some of the solution branches 1123 00:49:00,560 --> 00:49:02,630 are disconnected from each other. 1124 00:49:02,630 --> 00:49:05,054 Boy, that's crazy and complicated. 1125 00:49:05,054 --> 00:49:07,220 We're not going to worry about those sorts of cases, 1126 00:49:07,220 --> 00:49:09,744 but they can pop up and happen. 1127 00:49:09,744 --> 00:49:11,660 But the point is there are a robust strategies 1128 00:49:11,660 --> 00:49:14,240 you can employ to find multiple roots 1129 00:49:14,240 --> 00:49:18,230 of these equations or multiple minima in optimization problems 1130 00:49:18,230 --> 00:49:20,039 using precisely this technique. 1131 00:49:20,039 --> 00:49:22,580 You'll have a chance to practice this on this week's homework 1132 00:49:22,580 --> 00:49:24,080 assignment, and I think you should 1133 00:49:24,080 --> 00:49:25,610 take a look at this problem. 1134 00:49:25,610 --> 00:49:26,452 It's simple. 1135 00:49:26,452 --> 00:49:28,910 It looks complicated , but if you can understand this from, 1136 00:49:28,910 --> 00:49:30,993 then you'll understand everything you need to know 1137 00:49:30,993 --> 00:49:33,530 about bifurcation and homotopy. 1138 00:49:33,530 --> 00:49:35,030 Any questions? 1139 00:49:35,030 --> 00:49:36,751 Yes. 1140 00:49:36,751 --> 00:49:38,167 STUDENT: At a bifurcated point, we 1141 00:49:38,167 --> 00:49:40,730 go from having one root to three roots, per se. 1142 00:49:40,730 --> 00:49:43,200 And you said that when that occurs, 1143 00:49:43,200 --> 00:49:46,135 we need to follow each of these solutions, right? 1144 00:49:46,135 --> 00:49:46,760 PROFESSOR: Yes. 1145 00:49:46,760 --> 00:49:49,496 STUDENT: And you said that we need to step 1146 00:49:49,496 --> 00:49:50,890 in a different direction. 1147 00:49:50,890 --> 00:49:53,020 So what does that exactly mean? 1148 00:49:53,020 --> 00:49:55,930 In a 2D sense, we have this stepping forward 1149 00:49:55,930 --> 00:49:58,904 or stepping backwards. 1150 00:49:58,904 --> 00:50:00,570 PROFESSOR: This is a wonderful question. 1151 00:50:00,570 --> 00:50:05,060 So let me go backwards here. 1152 00:50:14,410 --> 00:50:16,120 Here's the bifurcation point, and when 1153 00:50:16,120 --> 00:50:17,661 we go through that bifurcation point, 1154 00:50:17,661 --> 00:50:20,800 there are now going to be three solutions. 1155 00:50:20,800 --> 00:50:23,710 And I'm going to need some initial conditions 1156 00:50:23,710 --> 00:50:27,490 for my solver to find each of these three solutions. 1157 00:50:27,490 --> 00:50:30,042 I can't give them all the same initial condition. 1158 00:50:30,042 --> 00:50:32,250 I'm going to try to track all three of these branches 1159 00:50:32,250 --> 00:50:33,666 with different initial conditions. 1160 00:50:33,666 --> 00:50:35,610 If I give them all the same initial condition, 1161 00:50:35,610 --> 00:50:39,132 and my algorithm meets the minimum threshold for what 1162 00:50:39,132 --> 00:50:40,590 an algorithm should be, they're all 1163 00:50:40,590 --> 00:50:42,048 going to converge to the same root. 1164 00:50:42,048 --> 00:50:43,410 So that's useless. 1165 00:50:43,410 --> 00:50:46,360 So I need to give them different initial conditions. 1166 00:50:46,360 --> 00:50:48,120 So what are those supposed to be? 1167 00:50:48,120 --> 00:50:50,220 In your problem, I'll give you explicit directions 1168 00:50:50,220 --> 00:50:51,510 for how to find those. 1169 00:50:51,510 --> 00:50:54,000 It turns out those initial guesses 1170 00:50:54,000 --> 00:50:57,030 should be perturbed in directions 1171 00:50:57,030 --> 00:50:59,670 that belong to the null space of the Jacobian. 1172 00:50:59,670 --> 00:51:03,480 So I want to perturb from that one solution, which was here, 1173 00:51:03,480 --> 00:51:07,680 in a direction that's parallel to the tangency 1174 00:51:07,680 --> 00:51:08,520 of those curves. 1175 00:51:08,520 --> 00:51:10,080 Those two curves were tangent with each other. 1176 00:51:10,080 --> 00:51:11,621 In the other solutions, they're going 1177 00:51:11,621 --> 00:51:15,720 to sit off in the direction that's tangent to those curves. 1178 00:51:15,720 --> 00:51:18,300 So I've got to find the vector that's in the null space, 1179 00:51:18,300 --> 00:51:21,370 or very, very close to the null space, 1180 00:51:21,370 --> 00:51:24,000 and that will tell me how to get better initial guesses 1181 00:51:24,000 --> 00:51:27,800 for picking up these other solution branches. 1182 00:51:27,800 --> 00:51:28,890 It's not trivial. 1183 00:51:28,890 --> 00:51:30,666 It's actually quite challenging. 1184 00:51:30,666 --> 00:51:32,040 You'll have a chance to practice. 1185 00:51:32,040 --> 00:51:35,160 It's part five of five parts on this problem. 1186 00:51:35,160 --> 00:51:36,750 The first four, they're relatively 1187 00:51:36,750 --> 00:51:37,791 straightforward and easy. 1188 00:51:37,791 --> 00:51:39,360 Part five is a little bit tricky. 1189 00:51:39,360 --> 00:51:41,250 You get explicit directions, and the TAs 1190 00:51:41,250 --> 00:51:43,800 know that that's the most difficult part of that problem. 1191 00:51:43,800 --> 00:51:45,610 So we'll help you through it. 1192 00:51:45,610 --> 00:51:48,150 It's something interesting to think about trying to do. 1193 00:51:48,150 --> 00:51:50,250 You may find you're only able to pick up some of these solution 1194 00:51:50,250 --> 00:51:51,500 branches, but not all of them. 1195 00:51:51,500 --> 00:51:52,830 That's OK. 1196 00:51:52,830 --> 00:51:55,500 There are ways to pick them all up if your algorithm is 1197 00:51:55,500 --> 00:51:57,664 designed well enough. 1198 00:51:57,664 --> 00:51:59,080 You may just pick up some of them, 1199 00:51:59,080 --> 00:52:00,930 and that may be the result you turn in. 1200 00:52:00,930 --> 00:52:02,220 You can look at the solution to figure out 1201 00:52:02,220 --> 00:52:04,110 how the other ones are picked up as well. 1202 00:52:04,110 --> 00:52:07,860 But you're looking at the direction 1203 00:52:07,860 --> 00:52:09,480 along these tangent curves. 1204 00:52:09,480 --> 00:52:12,896 That's a vector in the null space of the Jacobian. 1205 00:52:12,896 --> 00:52:14,520 And if you perturb in those directions, 1206 00:52:14,520 --> 00:52:16,770 then you can trace out these other roots 1207 00:52:16,770 --> 00:52:18,140 so you can find the bifurcation. 1208 00:52:18,140 --> 00:52:19,590 It's a wonderful question. 1209 00:52:19,590 --> 00:52:21,730 You'll get a chance to practice on it. 1210 00:52:21,730 --> 00:52:23,060 OK, last question. 1211 00:52:23,060 --> 00:52:25,180 STUDENT: [INAUDIBLE] 1212 00:52:25,180 --> 00:52:26,090 PROFESSOR: Yes. 1213 00:52:26,090 --> 00:52:28,020 STUDENT: Is this supposed to be jrx? 1214 00:52:32,530 --> 00:52:34,090 PROFESSOR: That's the Jacobian of f. 1215 00:52:34,090 --> 00:52:36,804 That's the derivatives of f with respect to x. 1216 00:52:36,804 --> 00:52:38,256 STUDENT: Not including r? 1217 00:52:38,256 --> 00:52:39,517 PROFESSOR: Yes. 1218 00:52:39,517 --> 00:52:41,350 Not including derivatives with respect to r. 1219 00:52:41,350 --> 00:52:42,990 Just derivatives with respect to x. 1220 00:52:42,990 --> 00:52:46,150 So this is the Jacobian of f. 1221 00:52:46,150 --> 00:52:47,860 The derivatives of f with respect to x. 1222 00:52:47,860 --> 00:52:53,020 The determinant of that will be 0 at the bifurcation point. 1223 00:52:53,020 --> 00:52:54,040 It'll turn out that-- 1224 00:52:54,040 --> 00:52:55,540 this will be the last thing and then 1225 00:52:55,540 --> 00:52:58,630 I have to get off the stage here. 1226 00:52:58,630 --> 00:53:02,170 It'll turn out that this matrix is also singular 1227 00:53:02,170 --> 00:53:03,820 at the bifurcation point. 1228 00:53:03,820 --> 00:53:08,400 It will be singular because the subspace j is singular. 1229 00:53:08,400 --> 00:53:10,390 So it'll turn out to be singular as well. 1230 00:53:10,390 --> 00:53:11,586 Don't worry about it. 1231 00:53:11,586 --> 00:53:13,210 The condition for the bifurcation point 1232 00:53:13,210 --> 00:53:15,880 is that the determine of the Jacobian is equal to 0, 1233 00:53:15,880 --> 00:53:19,540 and that point is a root of the equations you're looking at. 1234 00:53:19,540 --> 00:53:21,810 OK, thank you.