WEBVTT
00:00:01.585 --> 00:00:03.910
The following content is
provided under a Creative
00:00:03.910 --> 00:00:05.300
Commons license.
00:00:05.300 --> 00:00:07.510
Your support will help
MIT OpenCourseWare
00:00:07.510 --> 00:00:11.600
continue to offer high quality
educational resources for free.
00:00:11.600 --> 00:00:14.140
To make a donation or to
view additional materials
00:00:14.140 --> 00:00:18.100
from hundreds of MIT courses,
visit MIT OpenCourseWare
00:00:18.100 --> 00:00:18.980
at ocw.mit.edu.
00:00:24.220 --> 00:00:29.810
PROFESSOR: OK, let's go
ahead and get started again.
00:00:29.810 --> 00:00:31.760
So we had a nice
review on Monday
00:00:31.760 --> 00:00:33.350
of some of the
different concepts
00:00:33.350 --> 00:00:35.930
that we've talked about up
to this point in the course.
00:00:35.930 --> 00:00:38.990
We've covered linear
algebra, solutions of systems
00:00:38.990 --> 00:00:41.554
of nonlinear equations.
00:00:41.554 --> 00:00:43.720
There's been a lot of
fundamentals involved in that.
00:00:43.720 --> 00:00:46.629
Things are going
to get more applied
00:00:46.629 --> 00:00:48.170
as the sorts of
problems we're trying
00:00:48.170 --> 00:00:50.510
to solve get more complicated.
00:00:50.510 --> 00:00:53.420
Complicated problems require
very sophisticated methods
00:00:53.420 --> 00:00:55.280
to solve them.
00:00:55.280 --> 00:00:57.320
It's not going to be our
expectation that you're
00:00:57.320 --> 00:00:59.240
necessarily able
to implement all
00:00:59.240 --> 00:01:00.690
of these sophisticated methods.
00:01:00.690 --> 00:01:03.466
So for example, you
implemented Newton's method
00:01:03.466 --> 00:01:04.840
on your last
homework assignment.
00:01:04.840 --> 00:01:07.800
That'll be the last time we
ask you to do that by hand.
00:01:07.800 --> 00:01:10.400
You'll utilize tools
built into Matlab
00:01:10.400 --> 00:01:13.340
to solve other systems of
nonlinear equations going
00:01:13.340 --> 00:01:14.340
forward.
00:01:14.340 --> 00:01:17.150
You understand how it works
and what the limitations are.
00:01:17.150 --> 00:01:19.250
You understand how to
build different tools that
00:01:19.250 --> 00:01:21.560
will be inputs to
these functions
00:01:21.560 --> 00:01:24.740
like the function you're
trying to find the root for,
00:01:24.740 --> 00:01:28.640
or an explicit expression
for the Jacobian.
00:01:28.640 --> 00:01:32.000
You even understand how to set
certain parameters associated
00:01:32.000 --> 00:01:34.910
with these methods, and what
those particular parameter
00:01:34.910 --> 00:01:35.660
values mean.
00:01:35.660 --> 00:01:37.760
Like setting the
function norm tolerance,
00:01:37.760 --> 00:01:40.040
or setting the
step norm tolerance
00:01:40.040 --> 00:01:41.565
associated with these methods.
00:01:41.565 --> 00:01:43.190
Setting the number
of iterations you're
00:01:43.190 --> 00:01:44.300
going to let the method go.
00:01:44.300 --> 00:01:45.966
You've implemented
it, so you understand
00:01:45.966 --> 00:01:48.740
what these things mean, what the
consequences of choosing them
00:01:48.740 --> 00:01:50.780
are, how they might
increase the time associated
00:01:50.780 --> 00:01:55.280
with computations that
you're going to conduct.
00:01:55.280 --> 00:01:56.974
So we've done a lot
of fundamentals.
00:01:56.974 --> 00:01:58.640
Maybe that was difficult
for some of you
00:01:58.640 --> 00:02:00.680
who haven't seen this before.
00:02:00.680 --> 00:02:03.740
Now things are becoming
more about the practice
00:02:03.740 --> 00:02:04.690
of numerical methods.
00:02:04.690 --> 00:02:06.770
How do we actually
solve things reliably,
00:02:06.770 --> 00:02:09.680
and that's what today's
lecture will talk about.
00:02:09.680 --> 00:02:12.350
We discussed solving systems
of nonlinear equations
00:02:12.350 --> 00:02:16.140
and needing good initial
guesses for our methods.
00:02:16.140 --> 00:02:19.340
Bad initial guesses can lead
to unpredictable results.
00:02:19.340 --> 00:02:22.460
Good initial guesses, we know
the Newton-Raphson method
00:02:22.460 --> 00:02:25.640
is going to double the number of
accurate digits each iteration,
00:02:25.640 --> 00:02:27.870
and we should converge
very rapidly to a solution.
00:02:27.870 --> 00:02:30.680
So it seems like having
good initial guesses is
00:02:30.680 --> 00:02:31.950
really ideal.
00:02:31.950 --> 00:02:35.870
And this topic, homotopy
and bifurcation,
00:02:35.870 --> 00:02:39.140
will cover precisely
those issues,
00:02:39.140 --> 00:02:41.540
and the practice of getting
these good initial guesses.
00:02:41.540 --> 00:02:42.860
I think is one of the
most useful things you'll
00:02:42.860 --> 00:02:44.510
learn in this class, actually.
00:02:44.510 --> 00:02:46.790
It's a really nice way
of going about solving
00:02:46.790 --> 00:02:48.950
complicated problems.
00:02:48.950 --> 00:02:53.000
Before we begin, I want
to address a question
00:02:53.000 --> 00:02:54.909
I got via email last night.
00:02:54.909 --> 00:02:56.450
That's always good
if these questions
00:02:56.450 --> 00:02:59.600
go on [? Piaza ?] so
everyone can read them,
00:02:59.600 --> 00:03:03.044
but I thought it was a very
nice question about how
00:03:03.044 --> 00:03:04.460
do you find-- the
question was how
00:03:04.460 --> 00:03:07.680
do you find multiple roots
using the Newton-Raphson method.
00:03:07.680 --> 00:03:11.750
So if I have a polynomial,
where the function
00:03:11.750 --> 00:03:13.369
doesn't cross
through the origin,
00:03:13.369 --> 00:03:15.410
but it just touches the
origin, and goes back up.
00:03:15.410 --> 00:03:18.077
That's a root with
multiplicity in the polynomial.
00:03:18.077 --> 00:03:19.910
And can the Newton-Raphson
method find that?
00:03:19.910 --> 00:03:23.820
Because the derivative of the
function is 0 at the root.
00:03:23.820 --> 00:03:25.760
So who says the
Newton-Raphson method will
00:03:25.760 --> 00:03:27.231
be able to find those roots?
00:03:27.231 --> 00:03:27.980
What do you think?
00:03:27.980 --> 00:03:29.590
Yes?
00:03:29.590 --> 00:03:32.730
And who says it won't be
able to find those roots?
00:03:32.730 --> 00:03:34.265
Or who doesn't know?
00:03:34.265 --> 00:03:34.890
You don't know.
00:03:34.890 --> 00:03:35.831
That's OK.
00:03:35.831 --> 00:03:37.080
That's a good question, right.
00:03:37.080 --> 00:03:38.820
Do you know or don't you know?
00:03:38.820 --> 00:03:40.980
So we know the
Newton-Raphson method
00:03:40.980 --> 00:03:44.160
relies on knowing the
derivative of the function
00:03:44.160 --> 00:03:47.190
at different points
in order to take
00:03:47.190 --> 00:03:51.280
our next Newton-Raphson
step towards that solution.
00:03:51.280 --> 00:03:54.540
It may be the case, the
derivative is 0 at the root,
00:03:54.540 --> 00:03:59.109
but nearby the root, the
derivative may not be 0.
00:03:59.109 --> 00:04:01.400
And then the Newton-Raphson
steps are going to be fine.
00:04:01.400 --> 00:04:04.690
They're are always going to keep
stepping down towards the root.
00:04:04.690 --> 00:04:08.940
If you go through an analysis
of how the algorithm converges,
00:04:08.940 --> 00:04:11.970
you'll find out that you
only get linear convergence
00:04:11.970 --> 00:04:14.350
in that case, instead of
quadratic convergence.
00:04:14.350 --> 00:04:17.640
So there will be some
penalty to pay for the fact
00:04:17.640 --> 00:04:20.399
that the derivative
is 0 at the root
00:04:20.399 --> 00:04:22.890
itself, but it's not
going to be so bad.
00:04:22.890 --> 00:04:26.865
The method can still converge
to those sorts of solutions.
00:04:26.865 --> 00:04:29.610
Not all is lost there, but
the rate of convergence
00:04:29.610 --> 00:04:32.650
may be not as good as
you expected it to be.
00:04:32.650 --> 00:04:35.600
And the same is true in the
multi-dimensional case as well.
00:04:35.600 --> 00:04:38.569
It's a little hard to imagine
what a multiple root is
00:04:38.569 --> 00:04:39.360
going to look like.
00:04:39.360 --> 00:04:41.460
It's something
like tangent curves
00:04:41.460 --> 00:04:42.660
in the solution of plane.
00:04:42.660 --> 00:04:46.680
So if I have f1 of
x1 and x2 equal 0,
00:04:46.680 --> 00:04:48.720
that's a curve in
the x1, x2 plane.
00:04:48.720 --> 00:04:52.140
And I have f2 in that
same plane equal to 0.
00:04:52.140 --> 00:04:54.850
The place where they touch might
be one of these multiple roots.
00:04:54.850 --> 00:04:58.450
That's a place where the
determinant of the Jacobian
00:04:58.450 --> 00:04:59.751
may be equal to 0.
00:04:59.751 --> 00:05:01.500
You'll have the same
sort of circumstance.
00:05:01.500 --> 00:05:04.260
Nearby, the Jacobian
can be non singular,
00:05:04.260 --> 00:05:06.244
and you can take steps
towards the root,
00:05:06.244 --> 00:05:08.160
but there's a worst case
scenario where you're
00:05:08.160 --> 00:05:11.900
coming in on a direction
that's parallel
00:05:11.900 --> 00:05:15.232
that belongs to the null space
of the Jacobian at the root,
00:05:15.232 --> 00:05:16.940
and then everything's
going to slow down.
00:05:16.940 --> 00:05:18.660
You can still
converge, but it may
00:05:18.660 --> 00:05:20.780
be very slow as a consequence.
00:05:20.780 --> 00:05:22.530
I thought that was a
really nice question.
00:05:22.530 --> 00:05:25.180
It's a detailed question about
how Newton-Raphson works.
00:05:25.180 --> 00:05:27.330
It's not necessarily
going to fail
00:05:27.330 --> 00:05:28.860
under these weird circumstances.
00:05:28.860 --> 00:05:30.210
They don't come up too often.
00:05:30.210 --> 00:05:33.480
They're going to come up in
our discussion today, actually.
00:05:33.480 --> 00:05:36.720
It's not going to fail, but
it may converge very slowly
00:05:36.720 --> 00:05:38.430
to the solution.
00:05:38.430 --> 00:05:39.390
OK?
00:05:39.390 --> 00:05:40.480
Good.
00:05:40.480 --> 00:05:42.230
I'm going to remind
you, the last thing we
00:05:42.230 --> 00:05:44.180
did with Newton-Raphson
methods, we
00:05:44.180 --> 00:05:45.770
talked about one
way of fixing them.
00:05:45.770 --> 00:05:47.960
In lieu of having
good initial guesses,
00:05:47.960 --> 00:05:50.600
we may not want to take the
full Newton-Raphson step.
00:05:50.600 --> 00:05:53.870
We may want to do these quasi
Newton-Raphson methods instead.
00:05:53.870 --> 00:05:55.970
One of those was this
damp Newton-Raphson
00:05:55.970 --> 00:05:58.740
where we introduced
some damping parameter.
00:05:58.740 --> 00:06:01.640
We don't take the full step,
we take some shorter step,
00:06:01.640 --> 00:06:04.490
and we try to make the
absolute value over function
00:06:04.490 --> 00:06:06.409
become as small as possible.
00:06:06.409 --> 00:06:07.700
That's an optimization problem.
00:06:07.700 --> 00:06:11.390
It's as hard as finding the
root of the system of equations
00:06:11.390 --> 00:06:12.810
that we're looking at.
00:06:12.810 --> 00:06:14.990
So we don't solve that
optimization problem exactly.
00:06:14.990 --> 00:06:16.070
That's crazy.
00:06:16.070 --> 00:06:19.100
Instead we use this
backtracking line search method
00:06:19.100 --> 00:06:22.400
where we change our
damping parameter
00:06:22.400 --> 00:06:25.220
in some systematic way
until we get to a point
00:06:25.220 --> 00:06:26.570
where we're satisfied.
00:06:26.570 --> 00:06:30.410
Where we say OK, in the
process of changing this,
00:06:30.410 --> 00:06:33.290
we've actually found a
smaller value of our function,
00:06:33.290 --> 00:06:36.740
and we accept that
as our best guess
00:06:36.740 --> 00:06:39.890
for where the next
iterate is supposed to go.
00:06:39.890 --> 00:06:42.050
And this sort of
approach can give you
00:06:42.050 --> 00:06:44.570
global convergence for the
algorithm, which is great,
00:06:44.570 --> 00:06:47.570
but it's not globally
convergent to roots.
00:06:47.570 --> 00:06:53.330
It's globally convergent to
minima, asymptotes, and roots
00:06:53.330 --> 00:06:54.052
as well.
00:06:54.052 --> 00:06:55.260
But it's globally convergent.
00:06:55.260 --> 00:06:56.390
So it'll terminate.
00:06:56.390 --> 00:06:59.000
The algorithm will
stop, and when it stops,
00:06:59.000 --> 00:07:02.190
you can ask is
this a root or not?
00:07:02.190 --> 00:07:04.860
Do I think this is sufficiently
close to a root or not?
00:07:04.860 --> 00:07:06.632
If it's not sufficiently
close to a root,
00:07:06.632 --> 00:07:08.840
then you know you terminated
somewhere like a minima,
00:07:08.840 --> 00:07:10.535
or you rushed out
towards an asymptote,
00:07:10.535 --> 00:07:13.220
and maybe you need to start
with a new initial guess
00:07:13.220 --> 00:07:14.706
for your solution.
00:07:14.706 --> 00:07:17.070
These quasi Newton-Raphson
methods are very useful.
00:07:17.070 --> 00:07:19.100
This is what Matlab
uses in order to--
00:07:19.100 --> 00:07:20.600
it uses a more
sophisticated version
00:07:20.600 --> 00:07:22.100
of this called
the Dogleg method,
00:07:22.100 --> 00:07:23.920
but it's the same principle.
00:07:23.920 --> 00:07:25.430
It's trying to make
sure you're not
00:07:25.430 --> 00:07:27.386
taking steps that are too big.
00:07:27.386 --> 00:07:28.760
Trying to make
sure you're taking
00:07:28.760 --> 00:07:33.890
steps in the right direction
so that you're approaching
00:07:33.890 --> 00:07:35.340
the roots of your function.
00:07:35.340 --> 00:07:37.910
Or at least reducing the
absolute value of your function
00:07:37.910 --> 00:07:39.752
as close to zero
as you can get it.
00:07:39.752 --> 00:07:41.210
Are there any
questions about this?
00:07:41.210 --> 00:07:43.584
Anything we need to follow up
on in the previous lecture?
00:07:46.545 --> 00:07:49.340
No.
00:07:49.340 --> 00:07:50.900
OK, good initial guesses.
00:07:50.900 --> 00:07:52.460
This is the key,
really, for solving
00:07:52.460 --> 00:07:54.470
systems of nonlinear equations.
00:07:54.470 --> 00:07:56.860
It's the key for doing
optimization problems too.
00:07:56.860 --> 00:08:01.380
We need to have some idea of
what our solution looks like,
00:08:01.380 --> 00:08:05.280
and we want guesses that
are close to those solutions
00:08:05.280 --> 00:08:09.186
so that maybe we are in this
region of local convergence
00:08:09.186 --> 00:08:11.060
for the Newton-Raphson
method, and everything
00:08:11.060 --> 00:08:12.410
converges very quickly.
00:08:12.410 --> 00:08:15.170
So where do they come from?
00:08:15.170 --> 00:08:17.780
This is an important
question, and I'll
00:08:17.780 --> 00:08:19.409
show you how you can get them.
00:08:19.409 --> 00:08:21.200
There's another issue
too, which is related
00:08:21.200 --> 00:08:23.420
to this, which is the
non-linear equations,
00:08:23.420 --> 00:08:25.317
they can have multiple roots.
00:08:25.317 --> 00:08:27.650
If we do optimization problems--
that's our next topic--
00:08:27.650 --> 00:08:29.930
they can have multiple
minima or maxima
00:08:29.930 --> 00:08:31.850
that we're trying to seek out.
00:08:31.850 --> 00:08:34.429
Depending on our initial
guess, our numerical method
00:08:34.429 --> 00:08:36.940
may find one of these
roots or another root.
00:08:36.940 --> 00:08:39.049
It may find one of these
minima or another minima.
00:08:39.049 --> 00:08:41.130
Maybe we're looking
for the global minimum,
00:08:41.130 --> 00:08:43.130
so we need to look at them all.
00:08:43.130 --> 00:08:46.480
Are there methods that we
can use to find them all?
00:08:46.480 --> 00:08:48.920
So we'll talk about
continuation in homotopy,
00:08:48.920 --> 00:08:50.354
and we'll talk
about bifurcation.
00:08:50.354 --> 00:08:51.770
And it turns out
there are methods
00:08:51.770 --> 00:08:54.860
one can use to find all
the roots for a system
00:08:54.860 --> 00:08:55.980
of non-linear equations.
00:08:55.980 --> 00:08:58.490
In fact, the homework
assignment you have this week
00:08:58.490 --> 00:09:02.600
will ask you to do ternary
phase equilibrium problem.
00:09:02.600 --> 00:09:06.140
Vapor liquid phase equilibrium
with three components.
00:09:06.140 --> 00:09:07.760
There are five parts
to this problem.
00:09:07.760 --> 00:09:10.090
Four of them, I think, are
relatively straightforward.
00:09:10.090 --> 00:09:12.080
They're about finding
the pure component roots
00:09:12.080 --> 00:09:15.470
or randomly guessing
some initial conditions
00:09:15.470 --> 00:09:18.290
for your nonlinear
solver in order
00:09:18.290 --> 00:09:22.640
to find all of the different
azeotropes in the system.
00:09:22.640 --> 00:09:25.430
There's a fifth part that
asks you to actually implement
00:09:25.430 --> 00:09:29.150
this homotopy method, and try
to find all the roots in one
00:09:29.150 --> 00:09:30.109
go with your algorithm.
00:09:30.109 --> 00:09:31.608
Like the other
homeworks you've had,
00:09:31.608 --> 00:09:33.110
that part's a
little more tricky.
00:09:33.110 --> 00:09:34.520
Usually these homeworks
have one thing
00:09:34.520 --> 00:09:36.110
that's a little bit more
tricky than everything else.
00:09:36.110 --> 00:09:37.940
That part is a little
bit more tricky,
00:09:37.940 --> 00:09:40.630
but it's really instructive
to try to do it.
00:09:40.630 --> 00:09:42.230
We'll learn how to do it today.
00:09:42.230 --> 00:09:45.950
You can find all of the
coexisting compositions
00:09:45.950 --> 00:09:49.400
of this ternary phase
diagram, simultaneously,
00:09:49.400 --> 00:09:51.484
with one go through
this algorithm.
00:09:54.270 --> 00:09:56.490
So let's talk about
continuation first.
00:09:56.490 --> 00:09:59.270
This is an interesting
idea, and it
00:09:59.270 --> 00:10:01.910
relates to trying to find
roots of these equations,
00:10:01.910 --> 00:10:03.200
having good initial guesses.
00:10:03.200 --> 00:10:04.850
Here's a simple example.
00:10:04.850 --> 00:10:06.990
We have a third
degree polynomial,
00:10:06.990 --> 00:10:09.290
and it has three roots.
00:10:09.290 --> 00:10:12.530
We can look, graphically,
because this is one dimension,
00:10:12.530 --> 00:10:16.040
and see that one of these
roots is near minus 1 and 1/2,
00:10:16.040 --> 00:10:19.460
another root is near a 1/2,
and another root is near 1.
00:10:19.460 --> 00:10:21.830
And so I can go to my
non-linear equation solver,
00:10:21.830 --> 00:10:25.130
and give it initial guesses,
minus 1/2, 1/2, and 1,
00:10:25.130 --> 00:10:28.240
and I'm probably going to find
these roots pretty reliably.
00:10:28.240 --> 00:10:31.220
But in many dimensions,
we can't do these sorts
00:10:31.220 --> 00:10:32.990
of graphical approaches.
00:10:32.990 --> 00:10:35.160
You can't look at this function.
00:10:35.160 --> 00:10:36.830
It may be difficult
to actually get
00:10:36.830 --> 00:10:40.040
enough data to even look at it
if it's a two-dimensional set
00:10:40.040 --> 00:10:41.420
of functions.
00:10:41.420 --> 00:10:44.420
It's hard to see
where these roots are,
00:10:44.420 --> 00:10:47.570
and so continuation is a method
of transforming the problem
00:10:47.570 --> 00:10:50.780
into an easier one to solve.
00:10:50.780 --> 00:10:54.952
So let's take this problem,
this third order polynomial,
00:10:54.952 --> 00:10:55.910
and let's transform it.
00:10:55.910 --> 00:11:00.545
Let's replace the 2 with
a 2 times the lambda,
00:11:00.545 --> 00:11:02.670
and let's try to find the
roots of this polynomials
00:11:02.670 --> 00:11:05.230
as lambda grows from 0 to 1.
00:11:08.310 --> 00:11:10.650
This is sort of a weird
idea, but we actually
00:11:10.650 --> 00:11:15.720
know when lambda is equal
to 0, this goes away,
00:11:15.720 --> 00:11:18.030
and there's only one root
to this polynomial-- one
00:11:18.030 --> 00:11:19.860
real root-- which is minus 1.
00:11:19.860 --> 00:11:24.150
We actually know the answer
when lambda is equal to 0.
00:11:24.150 --> 00:11:27.990
If I make lambda a
little bit bigger than 0,
00:11:27.990 --> 00:11:31.380
there will be a root
which is close to minus 1.
00:11:31.380 --> 00:11:34.390
In fact, minus 1 is a good
initial guess for that root,
00:11:34.390 --> 00:11:37.220
and my non-linear
equation solver
00:11:37.220 --> 00:11:39.890
ought to converge very
quickly to that value.
00:11:39.890 --> 00:11:41.910
If I choose lambda
to be small, then it
00:11:41.910 --> 00:11:44.580
should be in that region
of local convergence
00:11:44.580 --> 00:11:46.860
of my Newton-Raphson
method, and I very quickly
00:11:46.860 --> 00:11:48.340
should find the next root.
00:11:48.340 --> 00:11:50.670
So I can step lambda along.
00:11:50.670 --> 00:11:55.140
I use the lambda equals 0
root as an initial guess
00:11:55.140 --> 00:11:57.870
for the next root, and
I keep increasing lambda
00:11:57.870 --> 00:12:01.896
from 0 all the way up to 1,
and when lambda is equal to 1,
00:12:01.896 --> 00:12:04.020
I'm going to find the root
to the initial problem I
00:12:04.020 --> 00:12:05.250
was looking at.
00:12:05.250 --> 00:12:06.630
Does that makes sense?
00:12:06.630 --> 00:12:10.590
So I'm continuing from one
set of solutions to the next.
00:12:10.590 --> 00:12:13.290
I vary this parameter
continuously,
00:12:13.290 --> 00:12:15.510
and for previous values
of that parameter,
00:12:15.510 --> 00:12:19.750
they serve as initial guesses
for the next solution.
00:12:19.750 --> 00:12:22.370
Here's what a code that
does that would look like.
00:12:22.370 --> 00:12:24.440
I define my lambdas.
00:12:24.440 --> 00:12:27.225
We don't know good ways
to define the lambda.
00:12:27.225 --> 00:12:28.850
I've got to figure
that out for myself,
00:12:28.850 --> 00:12:31.550
but here lambda is a vector
that goes from 0 to 1,
00:12:31.550 --> 00:12:33.230
in 1/100 increments.
00:12:33.230 --> 00:12:35.690
Maybe that's too
fine, but that's OK.
00:12:35.690 --> 00:12:38.540
And my initial guess for the
solution when lambda equals 0
00:12:38.540 --> 00:12:42.920
will be minus 1, and I loop
over all the values of lambda,
00:12:42.920 --> 00:12:46.640
and I use f0, which is the
one-dimensional non-linear
00:12:46.640 --> 00:12:48.290
equation solver
in Matlab, to find
00:12:48.290 --> 00:12:50.780
the roots of this
polynomial function--
00:12:50.780 --> 00:12:54.740
x cubed minus 2 lambda x plus 1.
00:12:54.740 --> 00:12:57.170
Here's the solution and that
becomes my initial guess
00:12:57.170 --> 00:12:58.130
for my next solution.
00:12:58.130 --> 00:13:00.255
So this loop is going to
go round and round until I
00:13:00.255 --> 00:13:02.570
get lambda equals 1,
and the solution that
00:13:02.570 --> 00:13:05.030
results there should be the
solution to the initial problem
00:13:05.030 --> 00:13:07.594
I was interested in.
00:13:07.594 --> 00:13:09.260
Here's what that looks
like graphically.
00:13:09.260 --> 00:13:11.840
I started with lambda equals 0.
00:13:11.840 --> 00:13:14.420
The root there serves as
an initial guess for lambda
00:13:14.420 --> 00:13:17.434
equals 1/100, and so
on, and these roots
00:13:17.434 --> 00:13:19.100
converge eventually,
to something that's
00:13:19.100 --> 00:13:22.780
close to minus 1 and 1/2.
00:13:22.780 --> 00:13:25.960
So the actual value is a little
bit bigger or a little bit
00:13:25.960 --> 00:13:28.900
smaller than minus 1.6.
00:13:28.900 --> 00:13:30.890
Here's one of the roots.
00:13:30.890 --> 00:13:33.640
So rather than having to
have a precise initial guess,
00:13:33.640 --> 00:13:36.280
I transformed my problem
from an easy to solve one
00:13:36.280 --> 00:13:38.320
to a more difficult
to solve one,
00:13:38.320 --> 00:13:42.190
and I used the path
along that transformation
00:13:42.190 --> 00:13:44.560
to give me rapid convergence
to better and better
00:13:44.560 --> 00:13:47.290
initial guesses to the problem
that I was interested in.
00:13:47.290 --> 00:13:47.860
Make sense?
00:13:50.760 --> 00:13:53.880
There's another way
of doing this too.
00:13:53.880 --> 00:13:56.630
You don't have to
go from 0 to 1.
00:13:56.630 --> 00:13:59.970
Let's go from lambda equals
something big back down to 1
00:13:59.970 --> 00:14:01.830
instead.
00:14:01.830 --> 00:14:05.820
So if lambda's big,
this term dominates
00:14:05.820 --> 00:14:07.704
the polynomial equation.
00:14:07.704 --> 00:14:09.870
This is the biggest term
in the polynomial equation,
00:14:09.870 --> 00:14:13.920
and it can either balance x
cubed, or it can balance 1.
00:14:13.920 --> 00:14:16.820
It can't balance against
both of them, it turns out.
00:14:16.820 --> 00:14:18.420
That
00:14:18.420 --> 00:14:23.850
So if we have this term large,
and it balances against 1, then
00:14:23.850 --> 00:14:26.170
when lambda's large, an
approximation for the root
00:14:26.170 --> 00:14:27.750
will be 1 over 2 lambda.
00:14:27.750 --> 00:14:30.540
If x equals 1 over 2 lambda,
and these are the two largest
00:14:30.540 --> 00:14:33.100
terms in the equation,
then these two will cancel,
00:14:33.100 --> 00:14:34.730
and f will be very close to 0.
00:14:34.730 --> 00:14:37.400
So that's one root when
lambda is very large.
00:14:37.400 --> 00:14:39.650
If I balance these other two
terms against each other,
00:14:39.650 --> 00:14:41.358
I'll see there are
two other roots, which
00:14:41.358 --> 00:14:43.890
are plus or minus the
square root of 2 lambda.
00:14:43.890 --> 00:14:47.910
So let's start with a large
lambda instead, and these
00:14:47.910 --> 00:14:49.830
as initial guesses
for the roots--
00:14:49.830 --> 00:14:52.737
and let's trace them
back to lambda equals 1.
00:14:52.737 --> 00:14:54.570
When lambda equals 1,
we recover the problem
00:14:54.570 --> 00:14:57.090
we were interested in before.
00:14:57.090 --> 00:14:59.340
So here, I start with three
different initial guesses,
00:14:59.340 --> 00:15:02.340
and I trace them
back, and I'm actually
00:15:02.340 --> 00:15:05.494
able to find all three roots
of the initial equation.
00:15:05.494 --> 00:15:07.410
So there's something
close to minus 1 and 1/2,
00:15:07.410 --> 00:15:10.410
something close to 1/2,
and something close to 1.
00:15:10.410 --> 00:15:16.125
So in one go, I found all
three roots using continuation.
00:15:16.125 --> 00:15:17.000
Is this clear so far?
00:15:17.000 --> 00:15:17.499
Sam.
00:15:17.499 --> 00:15:22.484
STUDENT: Is that one go or is
it three separate [INAUDIBLE]..
00:15:22.484 --> 00:15:24.150
PROFESSOR: That's an
excellent question.
00:15:24.150 --> 00:15:26.310
So I had to initiate
this process with three
00:15:26.310 --> 00:15:29.950
separate guesses, but it
turned out in this case,
00:15:29.950 --> 00:15:33.420
all three guesses converged
to three distinct roots
00:15:33.420 --> 00:15:35.800
in the end.
00:15:35.800 --> 00:15:37.530
it's one go in the
sense that there's
00:15:37.530 --> 00:15:40.350
one trajectory that
I had to follow,
00:15:40.350 --> 00:15:42.250
one passive lambda as
I had to go through,
00:15:42.250 --> 00:15:44.487
but I had to solve
three times to follow
00:15:44.487 --> 00:15:45.570
each of these three paths.
00:15:50.480 --> 00:15:53.642
There's actually no reason
to stop at lambda equals 1.
00:15:53.642 --> 00:15:56.100
I stopped there because that's
the solution to the problem.
00:15:56.100 --> 00:15:58.490
I'm interested in, but there's
actually something richer
00:15:58.490 --> 00:15:59.060
to see here.
00:15:59.060 --> 00:16:03.530
If I keep decreasing lambda,
eventually these two branches
00:16:03.530 --> 00:16:06.800
combine with each other,
and then they jump down,
00:16:06.800 --> 00:16:09.230
and they follow the
original branch that I
00:16:09.230 --> 00:16:13.370
had that went between
minus 1 at lambda equals 0,
00:16:13.370 --> 00:16:18.180
and this root about minus
1.6 at lambda equals 1.
00:16:18.180 --> 00:16:21.140
So the paths that these
trajectories follow
00:16:21.140 --> 00:16:22.820
can be very, very
complicated, actually.
00:16:22.820 --> 00:16:25.710
A little bit unwieldy
or unpredictable.
00:16:25.710 --> 00:16:30.050
But oftentimes, they can result
in predictions of all three
00:16:30.050 --> 00:16:31.692
roots or multiple
roots associated
00:16:31.692 --> 00:16:32.900
with the system of equations.
00:16:32.900 --> 00:16:33.400
Yeah.
00:16:33.400 --> 00:16:36.394
STUDENT: Going back to the
first [INAUDIBLE] solve
00:16:36.394 --> 00:16:41.130
from lambda equals [INAUDIBLE]
00:16:41.130 --> 00:16:42.231
PROFESSOR: Zero to one?
00:16:42.231 --> 00:16:43.133
STUDENT: Zero to one.
00:16:43.133 --> 00:16:46.025
So then would you find
only one solution there?
00:16:46.025 --> 00:16:46.650
PROFESSOR: Yes.
00:16:46.650 --> 00:16:49.350
So going from 0 to 1,
I found one solution.
00:16:49.350 --> 00:16:51.570
Going from infinity--
00:16:51.570 --> 00:16:52.560
10 is big enough.
00:16:52.560 --> 00:16:54.905
10 to 1, I was
able to find three.
00:16:54.905 --> 00:16:56.280
I didn't know
beforehand that I'm
00:16:56.280 --> 00:16:57.910
going to be able to do that.
00:16:57.910 --> 00:17:00.210
It happened to be the
case that we could
00:17:00.210 --> 00:17:03.040
with this by construction.
00:17:03.040 --> 00:17:06.000
This is a particular problem
where it works out that way.
00:17:06.000 --> 00:17:06.800
Other questions?
00:17:09.720 --> 00:17:10.594
So this is neat.
00:17:10.594 --> 00:17:12.510
Like we've done before
with iterative methods,
00:17:12.510 --> 00:17:16.589
we turn hard to solve problems
into a sequence of easier
00:17:16.589 --> 00:17:17.710
to solve problems.
00:17:17.710 --> 00:17:21.060
Good initial guesses converge
fast with Newton-Raphson.
00:17:21.060 --> 00:17:24.690
We have a sequence of better and
better initial guesses, which
00:17:24.690 --> 00:17:29.160
all converge very quickly
to eventually, the solution
00:17:29.160 --> 00:17:32.196
that we're looking for
evaluated at lambda equals 1.
00:17:32.196 --> 00:17:34.320
So this is one way of
getting good initial guesses.
00:17:34.320 --> 00:17:37.230
You have a sequence of
good initial guesses.
00:17:37.230 --> 00:17:40.100
Make sense?
00:17:40.100 --> 00:17:41.080
So we're just here.
00:17:41.080 --> 00:17:43.810
We had this polynomial
problem and we injected lambda
00:17:43.810 --> 00:17:44.930
in a particular place.
00:17:44.930 --> 00:17:47.530
We injected it there because
I knew it was going to work.
00:17:47.530 --> 00:17:49.154
I knew the problem
could be transformed
00:17:49.154 --> 00:17:51.790
in this way in order to work
out and distinguish this.
00:17:51.790 --> 00:17:56.200
It's not always obvious
how to transform a problem
00:17:56.200 --> 00:17:59.410
to make it look this
way, but oftentimes we
00:17:59.410 --> 00:18:02.380
have physical problems in
which there's a parameter
00:18:02.380 --> 00:18:04.960
that we can do continuation
with that is obvious.
00:18:04.960 --> 00:18:08.380
So think about fluid
mechanics problems.
00:18:08.380 --> 00:18:11.770
Those are nonlinear partial
differential equations.
00:18:11.770 --> 00:18:14.590
So we're trying to solve say,
the Navier-Stokes equations.
00:18:14.590 --> 00:18:18.980
Rather than jump in and try
to solve that right away,
00:18:18.980 --> 00:18:20.050
we might do continuation.
00:18:20.050 --> 00:18:23.470
We might find a solution at
very small Reynolds numbers,
00:18:23.470 --> 00:18:25.390
where the equations
are almost linear,
00:18:25.390 --> 00:18:27.760
exact solutions are
known in that case,
00:18:27.760 --> 00:18:31.870
and we might progressively step
the Reynolds number up, and use
00:18:31.870 --> 00:18:36.535
our solution set small Reynolds
numbers as initial guesses
00:18:36.535 --> 00:18:39.160
for increasingly large Reynolds
numbers until we get to the one
00:18:39.160 --> 00:18:40.690
that we're interested in.
00:18:40.690 --> 00:18:43.300
You can imagine doing that
in mass transfer problems
00:18:43.300 --> 00:18:45.520
where you vary
the Peclet number.
00:18:45.520 --> 00:18:47.390
In multicomponent phase
equilibria problems,
00:18:47.390 --> 00:18:49.060
this might happen by
varying temperature
00:18:49.060 --> 00:18:51.360
or pressure until we get to
the temperature or pressure
00:18:51.360 --> 00:18:53.020
that we're interested in.
00:18:53.020 --> 00:18:55.300
If we have reaction
equilibria problems,
00:18:55.300 --> 00:18:57.610
we might vary the
different reaction rates
00:18:57.610 --> 00:18:59.890
until we get to the right
balance of reaction rates
00:18:59.890 --> 00:19:02.830
to determine the
equilibrium compositions.
00:19:02.830 --> 00:19:05.380
So they're often natural
parameters in the problem
00:19:05.380 --> 00:19:07.630
that we're free to adjust.
00:19:07.630 --> 00:19:10.320
We can transform from
easy to solve problems
00:19:10.320 --> 00:19:12.850
to hard to solve problems
in a continuous fashion.
00:19:15.620 --> 00:19:19.770
It's not always going to work
out the way that I showed.
00:19:19.770 --> 00:19:23.220
It's not always going
to be so nice and neat.
00:19:23.220 --> 00:19:25.441
It may be quite a
bit more complicated,
00:19:25.441 --> 00:19:27.190
and sometimes there
are problems for which
00:19:27.190 --> 00:19:32.080
there isn't a natural
parameter available to vary.
00:19:32.080 --> 00:19:35.770
In those cases, there's another
strategy one can employ.
00:19:35.770 --> 00:19:38.440
It's called homotopy.
00:19:38.440 --> 00:19:41.270
This is the transformation
from one problem
00:19:41.270 --> 00:19:45.310
into another problem in
a continuous fashion.
00:19:45.310 --> 00:19:48.790
So oftentimes, we're
seeking a set of roots.
00:19:48.790 --> 00:19:52.044
x star, which are a function
of a parameter lambda,
00:19:52.044 --> 00:19:54.460
and they're the roots of an
equation that looks like this.
00:19:54.460 --> 00:19:57.340
So they're the roots
of this function, h,
00:19:57.340 --> 00:19:59.590
which is a linear
superposition of a function
00:19:59.590 --> 00:20:02.770
f and a function g.
00:20:02.770 --> 00:20:07.870
When lambda equals 0, h is equal
to g, and so the roots of h
00:20:07.870 --> 00:20:09.760
are the roots of g.
00:20:09.760 --> 00:20:13.450
And when lambda
equals 1, h is equal
00:20:13.450 --> 00:20:19.260
to f, and so the roots
of h are the roots of f.
00:20:19.260 --> 00:20:22.470
So we might transform-- we
make very lambda continuously
00:20:22.470 --> 00:20:25.740
between 0 and 1 in a
construction like this.
00:20:25.740 --> 00:20:28.530
So we might transform the
roots from the roots of g
00:20:28.530 --> 00:20:30.570
into the roots of f.
00:20:30.570 --> 00:20:34.290
Maybe the roots of g are easy
to find, but the roots of f
00:20:34.290 --> 00:20:35.610
are hard for us to find.
00:20:38.300 --> 00:20:41.790
There's a smooth transformation
though, from g to f.
00:20:41.790 --> 00:20:45.740
When we have those sorts
of smooth transformations,
00:20:45.740 --> 00:20:47.900
there's a chance things
will be well-behaved,
00:20:47.900 --> 00:20:50.390
or at least where we
encounter bad behavior--
00:20:50.390 --> 00:20:53.000
and I'll show you what I mean
by bad behavior in a minute--
00:20:53.000 --> 00:20:54.830
there's a reliable
way to interpret
00:20:54.830 --> 00:20:56.550
what's going on there.
00:20:56.550 --> 00:20:59.630
So we vary lambda in small
increments from 0 to 1,
00:20:59.630 --> 00:21:04.190
and the solution x
star for some lambda i
00:21:04.190 --> 00:21:06.140
is used as the
initial guess to find
00:21:06.140 --> 00:21:08.750
the roots at some lambda,
which is a little bit bigger
00:21:08.750 --> 00:21:09.830
than lambda i--
00:21:09.830 --> 00:21:11.420
our next iterate in lambda.
00:21:14.130 --> 00:21:15.860
Does this idea make sense?
00:21:15.860 --> 00:21:17.120
Where do f and g come from?
00:21:20.127 --> 00:21:21.960
So this is a big
question-- f is the problem
00:21:21.960 --> 00:21:24.240
you want to solve,
typically, and g
00:21:24.240 --> 00:21:26.610
is some auxiliary problem.
00:21:26.610 --> 00:21:29.440
It can be arbitrarily chosen.
00:21:29.440 --> 00:21:31.590
There's nothing to tell
you why one g should
00:21:31.590 --> 00:21:34.290
be better than another
one, but oftentimes, we
00:21:34.290 --> 00:21:37.620
choose them in a physical way.
00:21:37.620 --> 00:21:39.930
So there will be a
physical connection
00:21:39.930 --> 00:21:41.880
between the f problem
and the g problem,
00:21:41.880 --> 00:21:44.730
and I'll show you
that right now.
00:21:44.730 --> 00:21:46.110
So here's an example.
00:21:46.110 --> 00:21:47.550
We looked at this
example before.
00:21:47.550 --> 00:21:49.466
We want to find the roots
of the van der Waals
00:21:49.466 --> 00:21:50.550
equation of state.
00:21:50.550 --> 00:21:53.190
It's written in terms of the
reduced pressure, the reduced
00:21:53.190 --> 00:21:55.430
molar volume, and the
reduced temperature,
00:21:55.430 --> 00:22:00.040
and let's find those roots for a
given pressure and temperature.
00:22:00.040 --> 00:22:04.620
So this is a function f of
the molar volume equals 0,
00:22:04.620 --> 00:22:07.410
and we're looking for
these three roots here.
00:22:07.410 --> 00:22:13.441
There's a root down here, which
is the molar volume is low.
00:22:13.441 --> 00:22:15.690
Not a lot of volume per
number of moles, so maybe this
00:22:15.690 --> 00:22:17.280
is a liquid like root.
00:22:17.280 --> 00:22:18.870
Out here, the molar
volume is high.
00:22:18.870 --> 00:22:21.519
We have a lot of volume, the
number of moles and materials.
00:22:21.519 --> 00:22:23.310
This is a vapor like
root, and then there's
00:22:23.310 --> 00:22:26.390
an unstable root
in between them.
00:22:26.390 --> 00:22:27.770
So three roots to find.
00:22:27.770 --> 00:22:30.105
We'd like to be able
to find them in one go,
00:22:30.105 --> 00:22:32.480
or we'd like to have some
reliable mechanisms for finding
00:22:32.480 --> 00:22:33.440
them.
00:22:33.440 --> 00:22:35.250
Homotopy is one way to do this.
00:22:35.250 --> 00:22:38.150
So let's construct a new
function h of the molar
00:22:38.150 --> 00:22:41.360
volume, which is lambda times f.
00:22:41.360 --> 00:22:46.400
We want to find the roots of
f plus 1 minus lambda times g.
00:22:46.400 --> 00:22:49.020
And as g, let's use
something physical.
00:22:49.020 --> 00:22:52.700
So let's let g be a function
that represents the ideal gas
00:22:52.700 --> 00:22:54.320
equation of state.
00:22:54.320 --> 00:22:57.650
So PV is nRT, but I've
written everything
00:22:57.650 --> 00:23:00.560
in terms of the reduced
pressure, molar volume,
00:23:00.560 --> 00:23:03.040
and temperature of the van
der Waals equation of states.
00:23:03.040 --> 00:23:05.690
So you pick up this
factor of 8/3 there.
00:23:05.690 --> 00:23:08.630
But this is the ideal
gas equation of state.
00:23:08.630 --> 00:23:10.609
Its roots will be the
roots of the ideal gas
00:23:10.609 --> 00:23:11.400
equation of states.
00:23:11.400 --> 00:23:14.600
I'm going to transform from
something that's easy to solve.
00:23:14.600 --> 00:23:18.080
I know how to find the molar
volume here, there's just one,
00:23:18.080 --> 00:23:20.360
to something that's
hard for us to solve.
00:23:20.360 --> 00:23:22.160
It has three roots--
00:23:22.160 --> 00:23:24.560
the roots of them van der
Waals equation of state.
00:23:24.560 --> 00:23:26.450
So when lambda equals 0,
I'll get the roots associated
00:23:26.450 --> 00:23:28.220
with the ideal gas, and
then lambda equals 1,
00:23:28.220 --> 00:23:30.386
I'll get the roots associated
with the van der Waals
00:23:30.386 --> 00:23:31.864
equation.
00:23:31.864 --> 00:23:33.292
Yes?
00:23:33.292 --> 00:23:36.148
STUDENT: If you're
starting with lambda as 0
00:23:36.148 --> 00:23:39.836
and you get the roots of
g, you only get one root,
00:23:39.836 --> 00:23:42.180
but you're trying to find
two roots [INAUDIBLE]..
00:23:42.180 --> 00:23:43.560
So how does that work?
00:23:43.560 --> 00:23:44.601
PROFESSOR: I'll show you.
00:23:49.890 --> 00:23:51.620
Here's what the process
might look like.
00:23:51.620 --> 00:23:54.846
I've got to start with a reduced
temperature and pressure.
00:23:54.846 --> 00:23:56.720
Here's my initial guess
for the molar volume.
00:23:59.280 --> 00:24:01.840
It's the root of our
ideal gas equation.
00:24:01.840 --> 00:24:04.940
Here's the definition of
f, and the definition of g,
00:24:04.940 --> 00:24:06.960
and the definition of h.
00:24:06.960 --> 00:24:10.160
l times f plus 1
minus l times g,
00:24:10.160 --> 00:24:12.510
and I'm going to loop
over all of my lambdas,
00:24:12.510 --> 00:24:15.030
solve the equation subject
to some initial guess,
00:24:15.030 --> 00:24:18.090
and update my initial guess
every time I update lambda.
00:24:20.631 --> 00:24:21.130
That's fine.
00:24:21.130 --> 00:24:21.963
That's the code you.
00:24:21.963 --> 00:24:24.529
Can implement the code if you
want or play around with it.
00:24:24.529 --> 00:24:26.320
So what happens if I
try to carry this out?
00:24:26.320 --> 00:24:28.750
So I'm going to vary
lambda between 0 and 1.
00:24:28.750 --> 00:24:32.350
I start with my initial
ideal gas guess for the root,
00:24:32.350 --> 00:24:34.780
and as lambda grows
to 1, the molar volume
00:24:34.780 --> 00:24:37.540
shrinks until I get
to lambda equals 1,
00:24:37.540 --> 00:24:40.680
and I've found one
root of my equation.
00:24:40.680 --> 00:24:42.730
One root, good.
00:24:42.730 --> 00:24:45.250
There's not necessarily any
reason to stop at lambda
00:24:45.250 --> 00:24:46.940
equals 1.
00:24:46.940 --> 00:24:48.580
That's the root I
wanted, but there's
00:24:48.580 --> 00:24:50.950
no reason I have to stop there.
00:24:50.950 --> 00:24:52.030
So I could continue.
00:24:52.030 --> 00:24:54.424
If I keep pushing lambda
up higher and higher,
00:24:54.424 --> 00:24:56.590
eventually I'll get to a
point where all of a sudden
00:24:56.590 --> 00:24:59.770
the molar volume jumps
down to a different place.
00:24:59.770 --> 00:25:01.270
This is the result
of the algorithm.
00:25:01.270 --> 00:25:03.970
I just told lambda to go bigger.
00:25:03.970 --> 00:25:06.190
But there will be a
discontinuity or a cusp
00:25:06.190 --> 00:25:08.860
here that I jump off
of, and I find myself
00:25:08.860 --> 00:25:14.800
on a different solution
branch, which is sort of crazy.
00:25:14.800 --> 00:25:17.330
So now I can do
something clever.
00:25:17.330 --> 00:25:19.850
I found myself on a
different solution branch.
00:25:19.850 --> 00:25:24.260
Why not try decreasing lambda
along that solution branch
00:25:24.260 --> 00:25:24.909
instead?
00:25:24.909 --> 00:25:26.450
Maybe I'll find a
different solution,
00:25:26.450 --> 00:25:28.250
and it turns out
that's what happens.
00:25:28.250 --> 00:25:30.583
All right, so i jump down to
this other solution branch,
00:25:30.583 --> 00:25:34.490
and now I try decreasing
lambda from 2 back down to 1,
00:25:34.490 --> 00:25:38.670
and sure enough, I found
the vapor molar volume,
00:25:38.670 --> 00:25:40.400
and now I found the
liquid molar volume.
00:25:42.894 --> 00:25:44.810
There's no reason to
stop and lambda equals 1.
00:25:44.810 --> 00:25:48.856
So if I keep decreasing
lambda, eventually I'll
00:25:48.856 --> 00:25:50.480
hit another cusp,
and I'll jump back up
00:25:50.480 --> 00:25:53.030
to the original solution branch.
00:25:53.030 --> 00:25:55.320
So I do my homotopy.
00:25:55.320 --> 00:25:58.100
I vary lambda from 0 to
1, and I find one root,
00:25:58.100 --> 00:26:01.220
but if I keep going, I
might identify a cusp,
00:26:01.220 --> 00:26:03.290
and jump off that cross.
00:26:03.290 --> 00:26:06.980
I might reverse my
direction with the homotopy,
00:26:06.980 --> 00:26:08.490
and I could find another root.
00:26:08.490 --> 00:26:11.894
I might hit another
cusp as well.
00:26:11.894 --> 00:26:13.310
Now, when I hit a
cusp, maybe it's
00:26:13.310 --> 00:26:15.740
best not to jump off
the cusp, but instead
00:26:15.740 --> 00:26:17.930
to reverse direction
at the cusp,
00:26:17.930 --> 00:26:19.830
and try to trace it out
the other direction.
00:26:19.830 --> 00:26:22.360
And if I do that, if I get
right to this last point,
00:26:22.360 --> 00:26:24.050
and I try to reverse
direction again,
00:26:24.050 --> 00:26:27.020
I can trace out one
more solution path,
00:26:27.020 --> 00:26:28.940
and I can find the third
root to the equation.
00:26:28.940 --> 00:26:32.300
So in one go, if
I'm paying attention
00:26:32.300 --> 00:26:35.930
to how my roots are changing--
if they change by too much,
00:26:35.930 --> 00:26:40.390
I might switch the direction
with which I change lambda,
00:26:40.390 --> 00:26:45.050
and I could find all
of these roots at once.
00:26:45.050 --> 00:26:47.370
So three roots in one go.
00:26:49.901 --> 00:26:50.400
Yes.
00:26:50.400 --> 00:26:51.288
STUDENT: Is this
like trial and error?
00:26:51.288 --> 00:26:53.160
How do you know when to change?
00:26:53.160 --> 00:26:55.770
PROFESSOR: This is a
wonderful question.
00:26:55.770 --> 00:26:59.100
So in this case, you can
do it by trial and error.
00:26:59.100 --> 00:27:01.110
You can detect these jumps.
00:27:01.110 --> 00:27:02.280
That's even better.
00:27:02.280 --> 00:27:04.470
You look for jumps in the
value of the solution that
00:27:04.470 --> 00:27:07.500
are of sufficient magnitude,
and when you detect them,
00:27:07.500 --> 00:27:09.042
you can reverse the direction.
00:27:09.042 --> 00:27:10.500
There's an even
more systematic way
00:27:10.500 --> 00:27:13.109
to do this, which is called
arclength continuation.
00:27:13.109 --> 00:27:14.650
Give me one second,
and I'll give you
00:27:14.650 --> 00:27:16.440
slide on arclength
continuation, which
00:27:16.440 --> 00:27:20.640
is a systematic way of doing
exactly this process of tracing
00:27:20.640 --> 00:27:24.480
out this path in
the solution plane.
00:27:24.480 --> 00:27:27.810
Roots versus homotopy parameter.
00:27:27.810 --> 00:27:32.300
These cusps here, are sometimes
called turning points,
00:27:32.300 --> 00:27:34.190
and they have a
special property,
00:27:34.190 --> 00:27:39.810
which is the determinant of
the Jacobian of the homotopy--
00:27:39.810 --> 00:27:44.170
the homotopy function--
is going to be equal to 0.
00:27:44.170 --> 00:27:47.440
The Jacobian is going to be
singular right at these cusps.
00:27:47.440 --> 00:27:50.740
So you can detect
them by tracking what
00:27:50.740 --> 00:27:52.580
the Jacobian is doing right.
00:27:52.580 --> 00:27:54.580
The Jacobian is going to
be non-singular at most
00:27:54.580 --> 00:27:56.038
of these other
points, but when you
00:27:56.038 --> 00:28:00.070
hit one of these turning points,
the Jacobian will be singular.
00:28:00.070 --> 00:28:03.770
There isn't a well-defined
direction for the homotopy
00:28:03.770 --> 00:28:05.450
to proceed along.
00:28:05.450 --> 00:28:07.120
It hits this cusp
where the slope
00:28:07.120 --> 00:28:09.640
is infinite in this plane.
00:28:09.640 --> 00:28:12.340
There isn't a well-defined
direction to step along here.
00:28:15.290 --> 00:28:16.980
The question was--
00:28:16.980 --> 00:28:19.670
OK, it seems a little ad-hoc.
00:28:19.670 --> 00:28:21.470
You've got this
2D plane, and it's
00:28:21.470 --> 00:28:24.500
easy to guess when you jump
from one solution to another,
00:28:24.500 --> 00:28:26.930
and trace the path back.
00:28:26.930 --> 00:28:27.716
That's fine.
00:28:27.716 --> 00:28:29.090
There's a systematic
way of doing
00:28:29.090 --> 00:28:31.790
this, which is called
arclength continuation, which
00:28:31.790 --> 00:28:36.890
says I know that there's some
curve in this solution plane.
00:28:36.890 --> 00:28:38.330
There is some curve here.
00:28:38.330 --> 00:28:40.260
I've shown it to
you graphically.
00:28:40.260 --> 00:28:43.160
Let's try to parametrize
that curve in terms
00:28:43.160 --> 00:28:45.380
of a measure of the arclength.
00:28:45.380 --> 00:28:49.100
Call it s, the distance
along the curve.
00:28:49.100 --> 00:28:52.740
So the roots are a
function of lambda,
00:28:52.740 --> 00:28:55.160
which can be a function of
the distance along the curve,
00:28:55.160 --> 00:28:58.610
and lambda is a function of
the distance along the curve,
00:28:58.610 --> 00:29:01.700
and there is a
constraint which says
00:29:01.700 --> 00:29:04.370
s has to satisfy an
arclength formula.
00:29:04.370 --> 00:29:08.030
This is the formula for
the arclength in this plane
00:29:08.030 --> 00:29:10.200
of roots versus lambda.
00:29:10.200 --> 00:29:12.360
The derivative of this
function with respect to s
00:29:12.360 --> 00:29:15.575
squared plus the derivative
of lambda with respect to s
00:29:15.575 --> 00:29:16.565
squared equals 1.
00:29:16.565 --> 00:29:20.570
That defines s as a measure
of length along this curve.
00:29:20.570 --> 00:29:24.050
You learned arclength formulas
in the context of integration
00:29:24.050 --> 00:29:24.680
at one point.
00:29:24.680 --> 00:29:29.490
This is the differential
version of that same formula.
00:29:29.490 --> 00:29:31.260
So I've got a
differential equation.
00:29:31.260 --> 00:29:33.480
Well, it's actually a
differential algebraic
00:29:33.480 --> 00:29:35.130
equation.
00:29:35.130 --> 00:29:37.380
This is an algebraic
constraint that
00:29:37.380 --> 00:29:40.620
needs to be satisfied
for all values of s,
00:29:40.620 --> 00:29:43.789
and it depends on how
the roots change with s.
00:29:43.789 --> 00:29:45.330
We'll learn how to
solve differential
00:29:45.330 --> 00:29:49.500
algebraic equations in
the two sections from now.
00:29:49.500 --> 00:29:52.290
So we don't quite know how to
solve something like this yet,
00:29:52.290 --> 00:29:54.498
but if we know how to solve
those sorts of equations,
00:29:54.498 --> 00:29:57.150
then it's clear that
we can trace out
00:29:57.150 --> 00:29:58.920
this entire curve in
the solution plane,
00:29:58.920 --> 00:30:01.378
and then go back and try to
find the places where the curve
00:30:01.378 --> 00:30:04.060
intersects lambda equals 1.
00:30:04.060 --> 00:30:07.650
So if we follow this path by
solving this equation from s
00:30:07.650 --> 00:30:11.364
equals 0 out to
some very large s,
00:30:11.364 --> 00:30:13.530
then we should be able to
find all of the roots that
00:30:13.530 --> 00:30:15.175
are connected to the path.
00:30:15.175 --> 00:30:17.155
Yes.
00:30:17.155 --> 00:30:20.620
STUDENT: If there's a turning
point that [INAUDIBLE] then
00:30:20.620 --> 00:30:23.590
does that imply multiplicity?
00:30:23.590 --> 00:30:25.090
PROFESSOR: Yes.
00:30:25.090 --> 00:30:28.730
So it's a real problem.
00:30:28.730 --> 00:30:30.590
That turning point
singularity is actually
00:30:30.590 --> 00:30:33.470
built into this arclength
formula as well.
00:30:33.470 --> 00:30:37.340
This is the derivative
of x with respect
00:30:37.340 --> 00:30:41.240
to s squared, which takes
the direction you're
00:30:41.240 --> 00:30:43.070
moving in out of the problem.
00:30:43.070 --> 00:30:45.750
This is a length,
not a direction.
00:30:45.750 --> 00:30:48.050
So when we get to
these turning points,
00:30:48.050 --> 00:30:49.769
the solution methods
for these equations
00:30:49.769 --> 00:30:51.560
are going to have to
know that I was headed
00:30:51.560 --> 00:30:54.500
in a downward direction first.
00:30:54.500 --> 00:30:56.750
I can get to this turning
point, and then the question
00:30:56.750 --> 00:30:59.490
is which direction do I move
in from the turning point?
00:30:59.490 --> 00:31:01.480
Do I go up or do I go down?
00:31:01.480 --> 00:31:03.590
Which way does the
solution path change?
00:31:03.590 --> 00:31:06.920
The slope isn't defined there,
so it's quite challenging
00:31:06.920 --> 00:31:09.470
to figure out exactly
how to proceed.
00:31:09.470 --> 00:31:10.730
But it is what you say.
00:31:10.730 --> 00:31:15.620
It's like the solution has
multiplicity at that point.
00:31:15.620 --> 00:31:20.110
Curves are tangent in
the solution plane.
00:31:20.110 --> 00:31:21.680
It's a great question.
00:31:21.680 --> 00:31:25.606
OK, so that's the notion
of arclength continuation.
00:31:25.606 --> 00:31:26.980
We may talk about
this later when
00:31:26.980 --> 00:31:28.938
we talk about differential
algebraic equations,
00:31:28.938 --> 00:31:31.780
and we'll talk about some
equivalent sets of equations
00:31:31.780 --> 00:31:33.370
that we know how
to solve, but they
00:31:33.370 --> 00:31:35.380
have to satisfy this
constraint that s
00:31:35.380 --> 00:31:38.302
is a measure of arclength
along the curve.
00:31:38.302 --> 00:31:39.010
But there you go.
00:31:39.010 --> 00:31:42.220
You can find all the solutions,
at least all of the solutions
00:31:42.220 --> 00:31:46.270
connected to this path defined
by the homotopy equation.
00:31:46.270 --> 00:31:48.340
You can find all of
them in one go using
00:31:48.340 --> 00:31:50.262
some clever methodologies.
00:31:55.090 --> 00:31:58.580
Oftentimes, we're not
just worried about these
00:31:58.580 --> 00:32:02.660
turning points or cusps,
we encounter bifurcations
00:32:02.660 --> 00:32:03.480
in the solution.
00:32:03.480 --> 00:32:06.620
So we may have a path that
has one root along it,
00:32:06.620 --> 00:32:08.450
and then that path
may split, and there
00:32:08.450 --> 00:32:10.310
may be more than one root.
00:32:10.310 --> 00:32:12.380
Here's a simple example.
00:32:12.380 --> 00:32:16.880
Find the real roots
of x cubed minus rx.
00:32:16.880 --> 00:32:21.360
If r is negative, there will
always be one real root,
00:32:21.360 --> 00:32:24.230
and it'll be x equals 0.
00:32:24.230 --> 00:32:28.730
If r is equal to 0, there will
still be one real root, it's 0.
00:32:28.730 --> 00:32:32.530
That root will have
multiplicity 3 in that case.
00:32:32.530 --> 00:32:35.920
And if r becomes
positive, I'll suddenly
00:32:35.920 --> 00:32:38.860
have three real
roots instead of one.
00:32:38.860 --> 00:32:41.170
So as I vary the parameter
r from a negative number
00:32:41.170 --> 00:32:45.250
to a positive number, I'll go
from one real root to three.
00:32:45.250 --> 00:32:47.250
In the solution plane,
that will look like this.
00:32:47.250 --> 00:32:49.670
Here are the roots
as a function of r.
00:32:49.670 --> 00:32:50.900
r is less than 0.
00:32:50.900 --> 00:32:53.540
All the roots are equal to 0.
00:32:53.540 --> 00:32:56.630
When I hit r equals
0, I have multiplicity
00:32:56.630 --> 00:33:01.080
associated with the root,
and the solution bifurcates,
00:33:01.080 --> 00:33:04.240
and there will be three
possible solutions now.
00:33:04.240 --> 00:33:06.560
If I was trying to do a
homotopy problem where I very r
00:33:06.560 --> 00:33:09.620
continuously from some negative
number to some positive number,
00:33:09.620 --> 00:33:11.450
I'll hit this
point, and I'll want
00:33:11.450 --> 00:33:16.170
to follow all three paths
to find solutions out here
00:33:16.170 --> 00:33:18.340
at positive r.
00:33:18.340 --> 00:33:19.870
But I can detect those points.
00:33:19.870 --> 00:33:23.590
Just like I was said before,
at these weird turning points
00:33:23.590 --> 00:33:27.670
or bifurcations where we have
multiplicity in the roots,
00:33:27.670 --> 00:33:29.540
the Jacobian in is
going to be singular.
00:33:29.540 --> 00:33:32.050
The Jacobian of the homotopy
is going to be singular.
00:33:32.050 --> 00:33:33.620
Its determinant
will be equal to 0.
00:33:33.620 --> 00:33:37.294
So I can detect these points
by checking the determinant
00:33:37.294 --> 00:33:37.960
of the Jacobian.
00:33:37.960 --> 00:33:40.000
And when I find
one, I want to do
00:33:40.000 --> 00:33:43.900
something smart to try
to follow these paths.
00:33:43.900 --> 00:33:46.780
Your homework this week
has a problem like that.
00:33:46.780 --> 00:33:51.430
You're trying to find all the
azeotropes of this ternary
00:33:51.430 --> 00:33:55.120
vapor liquid equilibrium
system, and you'll
00:33:55.120 --> 00:33:57.940
have a homotopy parameter
that you change,
00:33:57.940 --> 00:33:59.320
and it'll bifurcate.
00:33:59.320 --> 00:34:01.240
OK, so as you change
the parameter,
00:34:01.240 --> 00:34:04.090
you'll go from one path to
two, and then those two paths
00:34:04.090 --> 00:34:05.410
will split again.
00:34:05.410 --> 00:34:09.280
You'll be splitting from a
single component solution
00:34:09.280 --> 00:34:14.770
into a two component azeotrope,
into a three component
00:34:14.770 --> 00:34:15.880
azeotrope.
00:34:15.880 --> 00:34:19.300
And you'll want to detect
where these bifurcations occur,
00:34:19.300 --> 00:34:21.800
and try to follow
solution branches off
00:34:21.800 --> 00:34:23.320
of those bifurcations,
and there's
00:34:23.320 --> 00:34:25.403
some good hints in the
problem for how to do that.
00:34:28.261 --> 00:34:29.969
So occasionally, we'll
encounter problems
00:34:29.969 --> 00:34:32.880
that switch from having
one solution to having
00:34:32.880 --> 00:34:35.269
many solutions.
00:34:35.269 --> 00:34:37.560
You've seen how this happens
in the discontinuous sense
00:34:37.560 --> 00:34:40.300
with these turning points.
00:34:40.300 --> 00:34:44.219
Here, for a given
value of lambda,
00:34:44.219 --> 00:34:47.520
all of a sudden it's clear that
not only is there one solution
00:34:47.520 --> 00:34:49.440
here, but there's another
solution down there.
00:34:49.440 --> 00:34:50.670
My path has to choose.
00:34:50.670 --> 00:34:53.580
Do I want this solution
or the one below?
00:34:53.580 --> 00:34:55.170
I jump off this turning point.
00:34:55.170 --> 00:34:58.170
That's a sort of bifurcation.
00:34:58.170 --> 00:35:00.439
These are continuous
bifurcations.
00:35:00.439 --> 00:35:01.980
They're often easier
to take care of.
00:35:05.370 --> 00:35:06.870
So the bifurcations
in a homotopy
00:35:06.870 --> 00:35:08.400
let us find multiple solutions.
00:35:08.400 --> 00:35:10.050
Every time we detect
a bifurcation,
00:35:10.050 --> 00:35:13.230
if we just track those
solution branches,
00:35:13.230 --> 00:35:15.750
we split our algorithm up to
track each of the branches
00:35:15.750 --> 00:35:19.200
separately, they'll
terminate at different roots
00:35:19.200 --> 00:35:21.450
to the homotopy equation.
00:35:21.450 --> 00:35:23.730
And when we get to the
value of lambda equals 1
00:35:23.730 --> 00:35:25.364
in our homotopy, we're there.
00:35:25.364 --> 00:35:27.405
We found multiple roots
to the original equation.
00:35:32.380 --> 00:35:34.990
These are often of
great physical interest.
00:35:37.780 --> 00:35:40.628
Do you know what sort of things
these bifurcations represent?
00:35:43.980 --> 00:35:47.160
One occurs in the van
der Waals equation.
00:35:47.160 --> 00:35:49.470
So if I start at very high
temperatures and a given
00:35:49.470 --> 00:35:51.600
pressure, the van
der Waals equation
00:35:51.600 --> 00:35:56.030
will have how many real roots?
00:35:56.030 --> 00:35:58.430
One-- a gas.
00:35:58.430 --> 00:36:00.960
And then as I turn
the temperature down,
00:36:00.960 --> 00:36:07.330
there is a propensity for
liquefaction of that gas phase.
00:36:07.330 --> 00:36:10.120
The formation of a liquid in
coexistence with the vapor.
00:36:10.120 --> 00:36:13.486
So I'll go from having one
root to having three roots
00:36:13.486 --> 00:36:15.610
all of a sudden, and there
will be this bifurcation
00:36:15.610 --> 00:36:17.830
point where that
transition happens
00:36:17.830 --> 00:36:18.880
in a continuous fashion.
00:36:22.044 --> 00:36:23.460
STUDENT: Can I ask
you a question?
00:36:23.460 --> 00:36:23.820
PROFESSOR: Yes.
00:36:23.820 --> 00:36:24.760
STUDENT: [INAUDIBLE]
00:36:24.760 --> 00:36:25.760
PROFESSOR: Yeah.
00:36:25.760 --> 00:36:28.236
STUDENT: So as you early put
it out that the turning point,
00:36:28.236 --> 00:36:31.091
the determinant of the Jacobian
[INAUDIBLE] including lambda
00:36:31.091 --> 00:36:32.350
is singular.
00:36:32.350 --> 00:36:34.050
But here you're writing j of x.
00:36:34.050 --> 00:36:36.450
Is that [INAUDIBLE]
00:36:36.450 --> 00:36:39.120
PROFESSOR: Excuse me.
00:36:39.120 --> 00:36:39.942
This is a typo.
00:36:39.942 --> 00:36:41.400
So there should be
a little h here.
00:36:41.400 --> 00:36:44.940
This is the determinant
of the Jacobian
00:36:44.940 --> 00:36:49.230
of the homotopy
equation at that root.
00:36:49.230 --> 00:36:53.010
Where there's a bifurcation,
the Jacobian will be singular.
00:36:53.010 --> 00:36:53.700
I apologize.
00:36:53.700 --> 00:36:54.360
That's a little bit--
00:36:54.360 --> 00:36:55.360
STUDENT: And that lambda.
00:36:55.360 --> 00:36:56.484
PROFESSOR: And that lambda.
00:36:56.484 --> 00:37:02.042
STUDENT: Just to make it clear,
you have n state variables.
00:37:02.042 --> 00:37:04.831
Like x is how many
unknowns you have,
00:37:04.831 --> 00:37:07.250
and then you're adding
another unknown to lambda,
00:37:07.250 --> 00:37:08.882
another parameter.
00:37:08.882 --> 00:37:12.630
This Jacobian has
n plus 1 dimension.
00:37:12.630 --> 00:37:18.987
This is also [INAUDIBLE]
Is this correct?
00:37:18.987 --> 00:37:19.820
PROFESSOR: Well, no.
00:37:19.820 --> 00:37:25.220
Actually, the Jacobian
with respect to x only
00:37:25.220 --> 00:37:28.902
is also singular at these
turning and bifurcation points.
00:37:28.902 --> 00:37:30.110
STUDENT: [INAUDIBLE] as well.
00:37:30.110 --> 00:37:31.534
PROFESSOR: Yes.
00:37:31.534 --> 00:37:33.950
It's also true what you said,
that if I have the Jacobian,
00:37:33.950 --> 00:37:35.180
and I take the
derivatives with respect
00:37:35.180 --> 00:37:36.650
to lambda, that will
also be singular,
00:37:36.650 --> 00:37:38.775
but it will be singular
because the subspace, which
00:37:38.775 --> 00:37:41.400
is the derivatives
with respect to x,
00:37:41.400 --> 00:37:43.610
gives you a singular component.
00:37:46.460 --> 00:37:48.200
The condition is
the same as here.
00:37:48.200 --> 00:37:52.220
So it's the Jacobian of
h, taking the derivatives
00:37:52.220 --> 00:37:53.960
with respect to x--
00:37:53.960 --> 00:37:56.660
all the different x's of h.
00:37:56.660 --> 00:37:58.970
The determinant of that
matrix at a particular lambda
00:37:58.970 --> 00:38:01.250
will be equal to 0,
and the value of lambda
00:38:01.250 --> 00:38:04.140
will be the lambda at which
you have this turning point,
00:38:04.140 --> 00:38:05.540
or which you have a bifurcation.
00:38:09.500 --> 00:38:12.030
Here's what it looks like
in a two-dimensional sense.
00:38:12.030 --> 00:38:14.680
So I'll have my
homotopy equation
00:38:14.680 --> 00:38:17.770
where I'm trying to
find the roots of h.
00:38:17.770 --> 00:38:20.100
Say it's a two-dimensional--
x is dimensions.
00:38:20.100 --> 00:38:22.990
So it has x1 and
x2, and these curves
00:38:22.990 --> 00:38:25.720
represent h1 equals
0 and h2 equals 0,
00:38:25.720 --> 00:38:30.580
and they intersect at one point
for a given value of lambda.
00:38:30.580 --> 00:38:32.780
There's the root.
00:38:32.780 --> 00:38:35.400
Now I increase the
value of lambda.
00:38:35.400 --> 00:38:37.340
The homotopy equations change.
00:38:37.340 --> 00:38:39.930
So these curves change
shape, and all of a sudden I
00:38:39.930 --> 00:38:43.500
go from crossing curves
to tangent curves,
00:38:43.500 --> 00:38:46.380
and we know tangent
curves in the plane
00:38:46.380 --> 00:38:50.304
will have a Jacobian
which is singular.
00:38:50.304 --> 00:38:51.720
And this is the
bifurcation point.
00:38:51.720 --> 00:38:53.580
These two curves become
tangent, and as I
00:38:53.580 --> 00:38:59.120
go to a bigger value of lambda,
they'll change shape again,
00:38:59.120 --> 00:39:01.250
and I'll now have
multiple roots.
00:39:01.250 --> 00:39:05.060
I'll bifurcate from
one solution to two,
00:39:05.060 --> 00:39:07.310
or one solution to three,
but the bifurcation
00:39:07.310 --> 00:39:09.620
will occur through
this transition
00:39:09.620 --> 00:39:11.810
where I have tangent
curves, and we know there,
00:39:11.810 --> 00:39:14.030
the determinant of
the Jacobian is 0.
00:39:14.030 --> 00:39:16.144
There's a singularity
under those conditions.
00:39:16.144 --> 00:39:17.060
Does that makes sense?
00:39:24.571 --> 00:39:26.820
OK, so a couple of things
about the practice, and I'll
00:39:26.820 --> 00:39:29.220
give you a simple example you
can try out, which I think
00:39:29.220 --> 00:39:32.530
is an interesting
one, if pathological.
00:39:32.530 --> 00:39:37.212
So in practice, it's hard to hit
this bifurcation point exactly.
00:39:37.212 --> 00:39:38.670
And I'm telling
you that you should
00:39:38.670 --> 00:39:40.025
be detecting these branches.
00:39:40.025 --> 00:39:41.400
You should follow
these branches,
00:39:41.400 --> 00:39:43.920
but clearly, I've got to know
where this point is in order
00:39:43.920 --> 00:39:46.590
to follow the branches.
00:39:46.590 --> 00:39:48.384
That's a problem.
00:39:48.384 --> 00:39:49.800
If the determinant
of the Jacobian
00:39:49.800 --> 00:39:52.530
is 0 at the bifurcation
point, then it's
00:39:52.530 --> 00:39:56.150
going to change from
positive to negative as I
00:39:56.150 --> 00:39:58.560
cross the bifurcation
point with lambda.
00:39:58.560 --> 00:40:00.109
Actually, I don't
necessarily have
00:40:00.109 --> 00:40:02.400
to find the point where the
determinant of the Jacobian
00:40:02.400 --> 00:40:03.190
is equal to 0.
00:40:03.190 --> 00:40:05.430
What I really should
do is try to track
00:40:05.430 --> 00:40:07.690
the sign of the determinant
of the Jacobian,
00:40:07.690 --> 00:40:09.150
and see when it
goes from positive
00:40:09.150 --> 00:40:11.230
signed to negative signed.
00:40:11.230 --> 00:40:13.320
And when it does
that, it will have
00:40:13.320 --> 00:40:15.690
crossed through one of
these bifurcation points
00:40:15.690 --> 00:40:18.590
or one of these cusps.
00:40:18.590 --> 00:40:20.610
That's the point at
which I should stop,
00:40:20.610 --> 00:40:23.790
and say, OK, there's possibly
other solution branches
00:40:23.790 --> 00:40:25.530
to follow.
00:40:25.530 --> 00:40:28.320
So as I step from my smaller
lambda to my bigger lambda,
00:40:28.320 --> 00:40:29.790
the sign changes.
00:40:29.790 --> 00:40:31.794
I want to step in some
different directions
00:40:31.794 --> 00:40:33.460
than the direction I
was heading before,
00:40:33.460 --> 00:40:35.940
and those different
directions will trace out
00:40:35.940 --> 00:40:38.010
the other solution branches.
00:40:38.010 --> 00:40:40.560
It turns out those
different directions
00:40:40.560 --> 00:40:45.090
belong to the null
space of the Jacobian.
00:40:45.090 --> 00:40:46.590
So if you find
vectors that are very
00:40:46.590 --> 00:40:49.590
close to the null
space of the Jacobian,
00:40:49.590 --> 00:40:51.600
those directions
are the directions
00:40:51.600 --> 00:40:54.439
you should go to find these
other solution branches.
00:40:54.439 --> 00:40:55.480
So it's very interesting.
00:40:55.480 --> 00:40:55.660
Yeah.
00:40:55.660 --> 00:40:58.201
STUDENT: Can you have a Jacobian
that goes from positive down
00:40:58.201 --> 00:41:00.590
to 0 and [INAUDIBLE]?
00:41:00.590 --> 00:41:03.150
PROFESSOR: That may be possible.
00:41:03.150 --> 00:41:05.880
You won't encounter that in
the problem that you're doing.
00:41:05.880 --> 00:41:08.460
That would be a very
difficult problem
00:41:08.460 --> 00:41:09.955
to do branch detection on.
00:41:09.955 --> 00:41:11.580
Let's Suppose that
was the case and you
00:41:11.580 --> 00:41:14.910
want to find exactly where
that bifurcation point is.
00:41:14.910 --> 00:41:17.350
These bifurcation points are
of physical significance.
00:41:17.350 --> 00:41:18.974
In the van der Waals
equation of state,
00:41:18.974 --> 00:41:21.500
that bifurcation point
is the critical point.
00:41:21.500 --> 00:41:24.000
It's the critical temperature
at which that phase separates.
00:41:24.000 --> 00:41:26.480
Maybe I want to
know that exactly.
00:41:26.480 --> 00:41:29.610
Well, all I need to do is
solve an augmented system
00:41:29.610 --> 00:41:30.910
of equations.
00:41:30.910 --> 00:41:33.790
So let's solve a
system of equations,
00:41:33.790 --> 00:41:37.530
which is our original
hematology equation equal to 0,
00:41:37.530 --> 00:41:40.860
and determinant of the Jacobian
of this homotopy equation
00:41:40.860 --> 00:41:42.420
equal to 0.
00:41:42.420 --> 00:41:45.450
This is if h had dimension n.
00:41:45.450 --> 00:41:47.700
This is now n plus 1
equations, and let's
00:41:47.700 --> 00:41:49.350
solve it for n plus 1 unknowns.
00:41:49.350 --> 00:41:54.000
Let's solve it for x and for
lambda. x has dimension n,
00:41:54.000 --> 00:41:56.070
and we add one more unknown--
00:41:56.070 --> 00:41:58.410
the value of the homotopy
parameter at which
00:41:58.410 --> 00:42:00.144
this bifurcation occurred.
00:42:00.144 --> 00:42:01.560
So we can find
that point exactly.
00:42:01.560 --> 00:42:03.630
So there may be cases
where this happens
00:42:03.630 --> 00:42:05.520
where it doesn't
continuously transition
00:42:05.520 --> 00:42:06.910
from positive to negative.
00:42:06.910 --> 00:42:10.920
It may hit 0 and go
back up, but that's
00:42:10.920 --> 00:42:15.130
a really challenging case to
handle, but you can handle it.
00:42:15.130 --> 00:42:19.350
You can find precisely
the point at which
00:42:19.350 --> 00:42:21.570
this bifurcation
occurs by solving
00:42:21.570 --> 00:42:23.992
these augmented equations.
00:42:23.992 --> 00:42:25.950
It's just another non-linear
equation to solve.
00:42:25.950 --> 00:42:26.450
Yes.
00:42:26.450 --> 00:42:30.416
STUDENT: I'm not sure
about the previous slide.
00:42:30.416 --> 00:42:35.550
So in the previous slide,
why is the [INAUDIBLE]
00:42:35.550 --> 00:42:37.630
PROFESSOR: Let me
go back to here.
00:42:37.630 --> 00:42:40.900
So let's let the
blue line here, be
00:42:40.900 --> 00:42:43.990
the place where the first
element of h is equal to 0.
00:42:43.990 --> 00:42:46.690
So this is all the
values of x at which
00:42:46.690 --> 00:42:48.970
the first element of our
homotopy top equation
00:42:48.970 --> 00:42:51.110
is equal to 0.
00:42:51.110 --> 00:42:53.150
And the red curve
here, let's let
00:42:53.150 --> 00:42:57.230
that be all the values of x at
which the second element of h
00:42:57.230 --> 00:42:58.820
is equal to 0.
00:42:58.820 --> 00:43:03.170
And so h itself, is only equal
in 0 where both of these curves
00:43:03.170 --> 00:43:06.730
are equal to 0, and
that's their intersection.
00:43:06.730 --> 00:43:10.690
So they intersect here, and
this star is the solution.
00:43:10.690 --> 00:43:14.530
It is the root of this
non-linear equation up here.
00:43:14.530 --> 00:43:17.380
Now, I increase lambda.
00:43:17.380 --> 00:43:19.400
I make lambda a
little bit bigger.
00:43:19.400 --> 00:43:21.070
That changes h.
00:43:21.070 --> 00:43:22.170
h is a function of lambda.
00:43:22.170 --> 00:43:27.260
So h changes, and these curves
change shape in the plane.
00:43:27.260 --> 00:43:29.300
And at that bifurcation
point, these two curves
00:43:29.300 --> 00:43:31.800
will become tangent
to each other.
00:43:31.800 --> 00:43:34.970
There will be one root,
but in this solution plane,
00:43:34.970 --> 00:43:37.220
the two curves will be tangent.
00:43:37.220 --> 00:43:38.660
And we saw before
that when we get
00:43:38.660 --> 00:43:40.720
these sort of
tangency conditions
00:43:40.720 --> 00:43:43.160
or the equivalent in
higher dimensions,
00:43:43.160 --> 00:43:44.750
the Jacobian of
our function we're
00:43:44.750 --> 00:43:47.850
trying to find the roots
of will be singular.
00:43:47.850 --> 00:43:50.740
It's like a multiple root.
00:43:50.740 --> 00:43:53.750
And when we increase
lambda further,
00:43:53.750 --> 00:43:56.397
curves change shape again.
00:43:56.397 --> 00:43:58.730
I'm not changing the red one
because that's complicated.
00:43:58.730 --> 00:44:00.370
I just changed the
blue one for you
00:44:00.370 --> 00:44:01.760
as a graphical illustration.
00:44:01.760 --> 00:44:03.680
So the blue one changes
shape, and now I've
00:44:03.680 --> 00:44:06.980
gone from having one
solution to having three.
00:44:06.980 --> 00:44:11.000
So in order to go from
one solution to three,
00:44:11.000 --> 00:44:12.560
in a continuous
fashion, I have to go
00:44:12.560 --> 00:44:16.730
through this tangency state,
and that defines the bifurcation
00:44:16.730 --> 00:44:17.685
point.
00:44:17.685 --> 00:44:18.560
Does that make sense?
00:44:23.220 --> 00:44:26.070
Here's a simple example
you should try out.
00:44:26.070 --> 00:44:27.750
So I'll present it
to you, and then you
00:44:27.750 --> 00:44:30.060
should try to
solve it in Matlab.
00:44:30.060 --> 00:44:32.920
You actually know the
solution, geometrically,
00:44:32.920 --> 00:44:34.140
so it's not too hard.
00:44:34.140 --> 00:44:36.712
So here's a function.
00:44:36.712 --> 00:44:38.170
The first element
to that function,
00:44:38.170 --> 00:44:41.304
it's a function of two
unknowns-- x1 and x2.
00:44:41.304 --> 00:44:42.720
The first element
to that function
00:44:42.720 --> 00:44:47.550
is an equation for a circle of
radius r with center at minus 3
00:44:47.550 --> 00:44:49.830
minus 1, and the
second element is
00:44:49.830 --> 00:44:56.310
an equation for a circle of
radius r with center at 2, 2.
00:44:56.310 --> 00:44:59.290
So there's two circles.
00:44:59.290 --> 00:45:04.090
We want to find the radius
where the circles just touch,
00:45:04.090 --> 00:45:06.730
and that point is a
bifurcation point.
00:45:06.730 --> 00:45:11.050
If r is perfect,
these two just touch.
00:45:11.050 --> 00:45:14.650
If r is a little too big,
the two circles overlap,
00:45:14.650 --> 00:45:16.930
and there are two
solutions to this problem.
00:45:16.930 --> 00:45:19.240
And if r is a little
too small, there
00:45:19.240 --> 00:45:22.210
are no solutions
to this problem.
00:45:22.210 --> 00:45:25.180
So the perfect r for touching
is a bifurcation point.
00:45:25.180 --> 00:45:27.640
It has all the properties
of a bifurcation point,
00:45:27.640 --> 00:45:28.750
just like we discussed.
00:45:28.750 --> 00:45:31.012
It's a multidimensional
equation and it bifurcates.
00:45:31.012 --> 00:45:33.220
You know the solution to
this problem, geometrically.
00:45:33.220 --> 00:45:35.780
You could work it out
in a couple of minutes,
00:45:35.780 --> 00:45:39.100
but you could also
solve it using
00:45:39.100 --> 00:45:40.840
these augmented equations.
00:45:40.840 --> 00:45:44.500
So we know that point at
which the circles just touch,
00:45:44.500 --> 00:45:47.500
the value of r at which
those circles just touch,
00:45:47.500 --> 00:45:51.370
will be the place where
f of x is equal to 0
00:45:51.370 --> 00:45:54.720
at which these equations
define a circle,
00:45:54.720 --> 00:45:58.780
and at which the Jacobian
of these equations--
00:45:58.780 --> 00:46:01.120
the derivatives of f with
respect to x, it's determinant
00:46:01.120 --> 00:46:02.980
is also equal to 0.
00:46:02.980 --> 00:46:07.870
So we want to solve this system
of three equations for three
00:46:07.870 --> 00:46:08.440
unknowns.
00:46:08.440 --> 00:46:11.980
The values of x1 and x2,
where the circles just touch,
00:46:11.980 --> 00:46:15.040
and the value of r, the
radius of these circles
00:46:15.040 --> 00:46:16.660
at which they just touch.
00:46:16.660 --> 00:46:18.720
That would find the
bifurcation point.
00:46:18.720 --> 00:46:20.710
That will find the
touching of the circles.
00:46:20.710 --> 00:46:22.780
Can you see that, geometrically?
00:46:22.780 --> 00:46:26.140
All these bifurcation problems
work exactly that way.
00:46:26.140 --> 00:46:28.060
There's some homotopy parameter.
00:46:28.060 --> 00:46:30.430
In this case, that's r.
00:46:30.430 --> 00:46:32.440
But there's some parameter
we're varying as we
00:46:32.440 --> 00:46:34.300
pass through that bifurcation.
00:46:34.300 --> 00:46:37.630
We'd like to know the
values of our unknowns
00:46:37.630 --> 00:46:41.070
and the value of this parameter
at that bifurcation point
00:46:41.070 --> 00:46:44.110
or at that critical point.
00:46:44.110 --> 00:46:46.440
So you find this
point by solving
00:46:46.440 --> 00:46:47.490
the augmented equations.
00:46:47.490 --> 00:46:50.670
We have a non-linear equation--
three nonlinear equations
00:46:50.670 --> 00:46:52.350
for three unknowns,
and you can do
00:46:52.350 --> 00:46:54.736
that with a
Newton-Raphson iteration.
00:46:54.736 --> 00:46:55.860
It's a non-linear equation.
00:46:55.860 --> 00:46:59.860
We know Newton-Raphson is the
way to solve these things.
00:46:59.860 --> 00:47:03.030
To do Newton-Raphson, that
means we need to compute
00:47:03.030 --> 00:47:05.440
the Jacobian--
00:47:05.440 --> 00:47:06.350
this is confusing.
00:47:06.350 --> 00:47:07.725
We've got to
compute the Jacobian
00:47:07.725 --> 00:47:10.870
of these augmented
equations, the derivatives
00:47:10.870 --> 00:47:12.790
of these augmented
equations with respect
00:47:12.790 --> 00:47:16.180
to our augmented unknowns.
00:47:16.180 --> 00:47:18.760
So that will be the
derivatives of f with respect
00:47:18.760 --> 00:47:21.910
to x, the derivatives
of f with respect
00:47:21.910 --> 00:47:25.920
to r, the derivatives of the
determinant of the Jacobian
00:47:25.920 --> 00:47:27.060
with respect to x.
00:47:27.060 --> 00:47:28.720
That's this gradient here.
00:47:28.720 --> 00:47:31.000
The derivatives of the
determinant of the Jacobian
00:47:31.000 --> 00:47:31.930
with respect to r.
00:47:31.930 --> 00:47:34.720
This is our augmented Jacobian.
00:47:34.720 --> 00:47:37.960
The augmented Jacobian times
the Newton-Raphson step.
00:47:37.960 --> 00:47:40.210
That's equal to
minus the function
00:47:40.210 --> 00:47:43.439
we're trying to solve evaluated
at the previous iteration.
00:47:43.439 --> 00:47:44.980
Solve the system of
equations, you'll
00:47:44.980 --> 00:47:48.070
get your Newton-Raphson step,
and you can find the solution
00:47:48.070 --> 00:47:49.480
to the augmented equations.
00:47:49.480 --> 00:47:52.250
It'll converge quadratically.
00:47:52.250 --> 00:47:55.090
You should commit yourself,
you can figure this out.
00:47:55.090 --> 00:47:58.120
Don't try to compute all
of these derivatives.
00:47:58.120 --> 00:47:59.160
That's complicated.
00:47:59.160 --> 00:48:00.770
You're going to make a mistake.
00:48:00.770 --> 00:48:03.100
I'm going to make a mistake
if I try to do that.
00:48:03.100 --> 00:48:06.400
What do we do instead
to compute the Jacobian?
00:48:06.400 --> 00:48:09.170
How do we compute
these derivatives?
00:48:09.170 --> 00:48:10.450
What's that?
00:48:10.450 --> 00:48:11.890
Finite difference, right.
00:48:11.890 --> 00:48:13.930
Don't play games with it.
00:48:13.930 --> 00:48:16.430
Try to use finite difference
to compute the elements of this
00:48:16.430 --> 00:48:17.986
instead.
00:48:17.986 --> 00:48:19.610
You know the exact
solution too, so you
00:48:19.610 --> 00:48:20.984
can use that as
an initial guess,
00:48:20.984 --> 00:48:23.780
or something close to the exact
solution as an initial guess
00:48:23.780 --> 00:48:27.050
to see if you can find
this bifurcation point.
00:48:27.050 --> 00:48:29.090
So homotopy
bifurcation, there are
00:48:29.090 --> 00:48:32.450
ways to get good initial
guesses by continuously
00:48:32.450 --> 00:48:33.890
changing our problem.
00:48:33.890 --> 00:48:35.810
We get a succession
of problems that
00:48:35.810 --> 00:48:38.920
will converge faster because
we have good initial guesses,
00:48:38.920 --> 00:48:42.140
and that will give us a reliable
convergence, hopefully, to one
00:48:42.140 --> 00:48:44.130
of the roots of the
equation we're after.
00:48:44.130 --> 00:48:47.840
If we're able to pick up
solution branches, we can track
00:48:47.840 --> 00:48:51.290
these cusps, or find
these bifurcation points,
00:48:51.290 --> 00:48:53.570
we might be able to
find multiple solutions.
00:48:53.570 --> 00:48:55.670
There are problems
for which depending
00:48:55.670 --> 00:48:59.090
on your choice of g in
this homotopy function,
00:48:59.090 --> 00:49:00.560
maybe some of the
solution branches
00:49:00.560 --> 00:49:02.630
are disconnected
from each other.
00:49:02.630 --> 00:49:05.054
Boy, that's crazy
and complicated.
00:49:05.054 --> 00:49:07.220
We're not going to worry
about those sorts of cases,
00:49:07.220 --> 00:49:09.744
but they can pop up and happen.
00:49:09.744 --> 00:49:11.660
But the point is there
are a robust strategies
00:49:11.660 --> 00:49:14.240
you can employ to
find multiple roots
00:49:14.240 --> 00:49:18.230
of these equations or multiple
minima in optimization problems
00:49:18.230 --> 00:49:20.039
using precisely this technique.
00:49:20.039 --> 00:49:22.580
You'll have a chance to practice
this on this week's homework
00:49:22.580 --> 00:49:24.080
assignment, and I
think you should
00:49:24.080 --> 00:49:25.610
take a look at this problem.
00:49:25.610 --> 00:49:26.452
It's simple.
00:49:26.452 --> 00:49:28.910
It looks complicated , but if
you can understand this from,
00:49:28.910 --> 00:49:30.993
then you'll understand
everything you need to know
00:49:30.993 --> 00:49:33.530
about bifurcation and homotopy.
00:49:33.530 --> 00:49:35.030
Any questions?
00:49:35.030 --> 00:49:36.751
Yes.
00:49:36.751 --> 00:49:38.167
STUDENT: At a
bifurcated point, we
00:49:38.167 --> 00:49:40.730
go from having one root
to three roots, per se.
00:49:40.730 --> 00:49:43.200
And you said that
when that occurs,
00:49:43.200 --> 00:49:46.135
we need to follow each of
these solutions, right?
00:49:46.135 --> 00:49:46.760
PROFESSOR: Yes.
00:49:46.760 --> 00:49:49.496
STUDENT: And you said
that we need to step
00:49:49.496 --> 00:49:50.890
in a different direction.
00:49:50.890 --> 00:49:53.020
So what does that exactly mean?
00:49:53.020 --> 00:49:55.930
In a 2D sense, we have
this stepping forward
00:49:55.930 --> 00:49:58.904
or stepping backwards.
00:49:58.904 --> 00:50:00.570
PROFESSOR: This is a
wonderful question.
00:50:00.570 --> 00:50:05.060
So let me go backwards here.
00:50:14.410 --> 00:50:16.120
Here's the bifurcation
point, and when
00:50:16.120 --> 00:50:17.661
we go through that
bifurcation point,
00:50:17.661 --> 00:50:20.800
there are now going
to be three solutions.
00:50:20.800 --> 00:50:23.710
And I'm going to need
some initial conditions
00:50:23.710 --> 00:50:27.490
for my solver to find each
of these three solutions.
00:50:27.490 --> 00:50:30.042
I can't give them all the
same initial condition.
00:50:30.042 --> 00:50:32.250
I'm going to try to track
all three of these branches
00:50:32.250 --> 00:50:33.666
with different
initial conditions.
00:50:33.666 --> 00:50:35.610
If I give them all the
same initial condition,
00:50:35.610 --> 00:50:39.132
and my algorithm meets the
minimum threshold for what
00:50:39.132 --> 00:50:40.590
an algorithm should
be, they're all
00:50:40.590 --> 00:50:42.048
going to converge
to the same root.
00:50:42.048 --> 00:50:43.410
So that's useless.
00:50:43.410 --> 00:50:46.360
So I need to give them
different initial conditions.
00:50:46.360 --> 00:50:48.120
So what are those
supposed to be?
00:50:48.120 --> 00:50:50.220
In your problem, I'll give
you explicit directions
00:50:50.220 --> 00:50:51.510
for how to find those.
00:50:51.510 --> 00:50:54.000
It turns out those
initial guesses
00:50:54.000 --> 00:50:57.030
should be perturbed
in directions
00:50:57.030 --> 00:50:59.670
that belong to the null
space of the Jacobian.
00:50:59.670 --> 00:51:03.480
So I want to perturb from that
one solution, which was here,
00:51:03.480 --> 00:51:07.680
in a direction that's
parallel to the tangency
00:51:07.680 --> 00:51:08.520
of those curves.
00:51:08.520 --> 00:51:10.080
Those two curves were
tangent with each other.
00:51:10.080 --> 00:51:11.621
In the other solutions,
they're going
00:51:11.621 --> 00:51:15.720
to sit off in the direction
that's tangent to those curves.
00:51:15.720 --> 00:51:18.300
So I've got to find the vector
that's in the null space,
00:51:18.300 --> 00:51:21.370
or very, very close
to the null space,
00:51:21.370 --> 00:51:24.000
and that will tell me how to
get better initial guesses
00:51:24.000 --> 00:51:27.800
for picking up these
other solution branches.
00:51:27.800 --> 00:51:28.890
It's not trivial.
00:51:28.890 --> 00:51:30.666
It's actually quite challenging.
00:51:30.666 --> 00:51:32.040
You'll have a
chance to practice.
00:51:32.040 --> 00:51:35.160
It's part five of five
parts on this problem.
00:51:35.160 --> 00:51:36.750
The first four,
they're relatively
00:51:36.750 --> 00:51:37.791
straightforward and easy.
00:51:37.791 --> 00:51:39.360
Part five is a
little bit tricky.
00:51:39.360 --> 00:51:41.250
You get explicit
directions, and the TAs
00:51:41.250 --> 00:51:43.800
know that that's the most
difficult part of that problem.
00:51:43.800 --> 00:51:45.610
So we'll help you through it.
00:51:45.610 --> 00:51:48.150
It's something interesting
to think about trying to do.
00:51:48.150 --> 00:51:50.250
You may find you're only able to
pick up some of these solution
00:51:50.250 --> 00:51:51.500
branches, but not all of them.
00:51:51.500 --> 00:51:52.830
That's OK.
00:51:52.830 --> 00:51:55.500
There are ways to pick them
all up if your algorithm is
00:51:55.500 --> 00:51:57.664
designed well enough.
00:51:57.664 --> 00:51:59.080
You may just pick
up some of them,
00:51:59.080 --> 00:52:00.930
and that may be the
result you turn in.
00:52:00.930 --> 00:52:02.220
You can look at the
solution to figure out
00:52:02.220 --> 00:52:04.110
how the other ones
are picked up as well.
00:52:04.110 --> 00:52:07.860
But you're looking
at the direction
00:52:07.860 --> 00:52:09.480
along these tangent curves.
00:52:09.480 --> 00:52:12.896
That's a vector in the
null space of the Jacobian.
00:52:12.896 --> 00:52:14.520
And if you perturb
in those directions,
00:52:14.520 --> 00:52:16.770
then you can trace
out these other roots
00:52:16.770 --> 00:52:18.140
so you can find the bifurcation.
00:52:18.140 --> 00:52:19.590
It's a wonderful question.
00:52:19.590 --> 00:52:21.730
You'll get a chance
to practice on it.
00:52:21.730 --> 00:52:23.060
OK, last question.
00:52:23.060 --> 00:52:25.180
STUDENT: [INAUDIBLE]
00:52:25.180 --> 00:52:26.090
PROFESSOR: Yes.
00:52:26.090 --> 00:52:28.020
STUDENT: Is this
supposed to be jrx?
00:52:32.530 --> 00:52:34.090
PROFESSOR: That's
the Jacobian of f.
00:52:34.090 --> 00:52:36.804
That's the derivatives
of f with respect to x.
00:52:36.804 --> 00:52:38.256
STUDENT: Not including r?
00:52:38.256 --> 00:52:39.517
PROFESSOR: Yes.
00:52:39.517 --> 00:52:41.350
Not including derivatives
with respect to r.
00:52:41.350 --> 00:52:42.990
Just derivatives
with respect to x.
00:52:42.990 --> 00:52:46.150
So this is the Jacobian of f.
00:52:46.150 --> 00:52:47.860
The derivatives of
f with respect to x.
00:52:47.860 --> 00:52:53.020
The determinant of that will
be 0 at the bifurcation point.
00:52:53.020 --> 00:52:54.040
It'll turn out that--
00:52:54.040 --> 00:52:55.540
this will be the
last thing and then
00:52:55.540 --> 00:52:58.630
I have to get off
the stage here.
00:52:58.630 --> 00:53:02.170
It'll turn out that this
matrix is also singular
00:53:02.170 --> 00:53:03.820
at the bifurcation point.
00:53:03.820 --> 00:53:08.400
It will be singular because
the subspace j is singular.
00:53:08.400 --> 00:53:10.390
So it'll turn out to
be singular as well.
00:53:10.390 --> 00:53:11.586
Don't worry about it.
00:53:11.586 --> 00:53:13.210
The condition for
the bifurcation point
00:53:13.210 --> 00:53:15.880
is that the determine of
the Jacobian is equal to 0,
00:53:15.880 --> 00:53:19.540
and that point is a root of the
equations you're looking at.
00:53:19.540 --> 00:53:21.810
OK, thank you.