WEBVTT
00:00:00.000 --> 00:00:02.020
The following
content is provided
00:00:02.020 --> 00:00:06.060
by MIT OpenCourseWare under
a Creative Commons license.
00:00:06.060 --> 00:00:07.750
Additional information
about our license
00:00:07.750 --> 00:00:10.380
and MIT OpenCourseWare
in general
00:00:10.380 --> 00:00:11.930
is available at ocw.mit.edu.
00:00:15.280 --> 00:00:16.620
PROFESSOR: OK.
00:00:16.620 --> 00:00:18.060
We're ready to go.
00:00:18.060 --> 00:00:23.250
So this will be the last lecture
that I give maybe, almost,
00:00:23.250 --> 00:00:26.260
and the last one
that we'll videotape,
00:00:26.260 --> 00:00:30.340
and then Monday we start
on the presentations.
00:00:30.340 --> 00:00:32.100
And we'd better get
rolling on those
00:00:32.100 --> 00:00:35.730
because time will
run out on us, and I
00:00:35.730 --> 00:00:39.420
want everybody who would
like to to have a chance
00:00:39.420 --> 00:00:43.150
to talk about work.
00:00:43.150 --> 00:00:43.760
OK.
00:00:43.760 --> 00:00:44.260
Can I begin?
00:00:47.510 --> 00:00:53.520
So I promised to speak
about inverse problems,
00:00:53.520 --> 00:00:55.980
ill-posed problems.
00:00:55.980 --> 00:00:59.280
It's a giant
subject, but there's
00:00:59.280 --> 00:01:02.340
one thing I didn't do,
one beautiful thing that I
00:01:02.340 --> 00:01:05.580
didn't do well at the
end of the last lecture.
00:01:05.580 --> 00:01:14.990
And I if you don't mind I'd like
to go back to that, because --
00:01:14.990 --> 00:01:20.990
the number one example for
optimization is least squares,
00:01:20.990 --> 00:01:24.150
minimizing b minus A*x squared.
00:01:24.150 --> 00:01:28.700
Here's b; here
are all the A*x's.
00:01:28.700 --> 00:01:30.440
Think of it as a
line, but of course
00:01:30.440 --> 00:01:33.670
it could be a
n-dimensional subspace.
00:01:33.670 --> 00:01:35.890
And then this is the winner.
00:01:35.890 --> 00:01:38.880
And the reason it's the winner
is that it's a right angle,
00:01:38.880 --> 00:01:40.990
it's a true projection.
00:01:40.990 --> 00:01:43.250
And there's a second
problem, which
00:01:43.250 --> 00:01:48.180
is minimize this thing,
the distance to w,
00:01:48.180 --> 00:01:52.600
where w lies on this
perpendicular direction.
00:01:52.600 --> 00:01:56.870
And, of course,
the closest point w
00:01:56.870 --> 00:02:00.280
to the b in this
direction is there.
00:02:00.280 --> 00:02:05.260
So those are the actual winners
that get the little stars.
00:02:05.260 --> 00:02:11.720
And when we're there, the
length of this squared
00:02:11.720 --> 00:02:16.760
plus the length of this squared
equals the length of b squared.
00:02:16.760 --> 00:02:20.340
We have right angles,
and Pythagoras,
00:02:20.340 --> 00:02:24.800
a squared plus b squared
equals c squared is right.
00:02:24.800 --> 00:02:31.200
But then, so what I added just
quickly at the end was the key
00:02:31.200 --> 00:02:39.990
point about -- suppose I take
any allowed A*x and any allowed
00:02:39.990 --> 00:02:41.810
w.
00:02:41.810 --> 00:02:45.430
Then I would expect -- and I
look at that distance which is
00:02:45.430 --> 00:02:49.370
larger than this, and I look at
this distance which is larger
00:02:49.370 --> 00:02:58.610
than this, so of course -- now
I'm not looking at the optimal
00:02:58.610 --> 00:03:02.590
ones but instead looking
at these solid line ones,
00:03:02.590 --> 00:03:08.570
and then the b minus A*x squared
is bigger than it has to be.
00:03:08.570 --> 00:03:12.730
The b minus w squared
is bigger, so the sum
00:03:12.730 --> 00:03:16.040
is b squared plus another term.
00:03:16.040 --> 00:03:21.130
So that's weak duality,
that for any choice,
00:03:21.130 --> 00:03:25.750
some inequality holds, since
this is never negative --
00:03:25.750 --> 00:03:30.650
another way to say weak duality
would be to say that this is
00:03:30.650 --> 00:03:32.260
always greater or equal to.
00:03:32.260 --> 00:03:34.810
But what I didn't
throw in last time
00:03:34.810 --> 00:03:37.510
was: there's this
beautiful expression
00:03:37.510 --> 00:03:41.220
for what the difference is.
00:03:44.210 --> 00:03:47.760
If you don't have the optimum,
this is what you miss by.
00:03:47.760 --> 00:03:52.110
And one part of the beauty is
that that tells us right away
00:03:52.110 --> 00:03:54.590
how to recognize the optimum.
00:03:54.590 --> 00:03:58.400
Duality holds, equality holds
without this term when this
00:03:58.400 --> 00:04:01.700
term is 0, when b is
equal to A*x plus w,
00:04:01.700 --> 00:04:08.660
and of course that's the good
dashed line case when that
00:04:08.660 --> 00:04:13.040
vector plus that
vector is exactly b.
00:04:13.040 --> 00:04:16.710
A*x and the w make b.
00:04:16.710 --> 00:04:21.730
Those are the dashed
lines; that's optimal.
00:04:21.730 --> 00:04:26.860
So somehow the optimality
condition is this one,
00:04:26.860 --> 00:04:30.130
and this is a condition that
connects the two problems.
00:04:30.130 --> 00:04:35.140
See, here we had a minimization
of this. w didn't appear.
00:04:35.140 --> 00:04:38.170
In the dual problem we had
a minimization of this.
00:04:38.170 --> 00:04:40.480
x didn't appear.
00:04:40.480 --> 00:04:48.230
But when we get both problems
right then they connect.
00:04:48.230 --> 00:04:54.910
And this is the step that
in my three-step framework
00:04:54.910 --> 00:05:00.450
for applied math that dominated
18.085, there was the x,
00:05:00.450 --> 00:05:07.140
there was the b minus Ax,
there was the shift over to w.
00:05:07.140 --> 00:05:13.980
So a matrix A entered there, a
matrix A transpose entered here
00:05:13.980 --> 00:05:19.890
to give A transpose w equals
0, and here is the bridge.
00:05:19.890 --> 00:05:23.150
That's the bridge between them
which in this simple model
00:05:23.150 --> 00:05:28.400
problem is -- I usually write
that bridge with a physical
00:05:28.400 --> 00:05:33.250
constant c, and in this
simple model problem c was 1,
00:05:33.250 --> 00:05:34.660
or c was the identity.
00:05:37.450 --> 00:05:43.960
So then this equals this, which
is our optimality condition.
00:05:43.960 --> 00:05:47.030
So when we have that
bridge between one problem
00:05:47.030 --> 00:05:52.810
and the dual problem
we've got them both right.
00:05:52.810 --> 00:05:55.380
So I'm happy to find that.
00:05:55.380 --> 00:05:59.590
But I have a problem for you.
00:05:59.590 --> 00:06:04.120
You may say, where did
that identity come from?
00:06:04.120 --> 00:06:07.170
So that's simply an identity.
00:06:07.170 --> 00:06:12.690
And it came from just
multiplying it out.
00:06:12.690 --> 00:06:18.480
If I multiply out, take b minus
A*x transposed times b minus
00:06:18.480 --> 00:06:19.530
A*x.
00:06:19.530 --> 00:06:21.840
Take this transposed
times itself.
00:06:21.840 --> 00:06:24.580
b transposed b This
transpose this.
00:06:24.580 --> 00:06:26.380
It works.
00:06:26.380 --> 00:06:31.100
The terms cancel, and
this is an identity.
00:06:31.100 --> 00:06:35.760
Now that's a simple
identity in geometry,
00:06:35.760 --> 00:06:39.200
so my puzzle for
you is take this --
00:06:39.200 --> 00:06:41.790
let me draw the identity here.
00:06:41.790 --> 00:06:45.030
So here's my picture.
00:06:45.030 --> 00:06:47.750
I'm going up to any w.
00:06:47.750 --> 00:06:54.010
And here is b in this picture.
00:06:54.010 --> 00:06:55.900
So b minus w is there.
00:07:01.330 --> 00:07:03.380
This says that
some vector squared
00:07:03.380 --> 00:07:05.780
plus that squared equals that
squared plus that squared.
00:07:05.780 --> 00:07:08.870
So now I just have to
find all those vectors
00:07:08.870 --> 00:07:12.740
and ask you prove it.
00:07:12.740 --> 00:07:15.320
So let me name
all these vectors.
00:07:18.630 --> 00:07:22.170
So here is b minus A*x.
00:07:22.170 --> 00:07:23.290
Right?
00:07:23.290 --> 00:07:24.310
That's b minus A*x.
00:07:27.120 --> 00:07:29.000
And here's w.
00:07:29.000 --> 00:07:31.077
OK.
00:07:31.077 --> 00:07:32.160
Let me get them all right.
00:07:32.160 --> 00:07:37.800
I have to get b in there
and here is b minus w,
00:07:37.800 --> 00:07:39.660
and here's b minus A*x.
00:07:39.660 --> 00:07:40.920
And what's the fourth one?
00:07:40.920 --> 00:07:43.720
The fourth one is
this mysterious one.
00:07:43.720 --> 00:07:47.270
So I go up b minus A*x, w.
00:07:47.270 --> 00:07:50.090
I'll have to put that here.
00:07:50.090 --> 00:07:53.950
I'm going up this
same w, straight up.
00:07:53.950 --> 00:07:57.860
There is the
mysterious fourth guy:
00:07:57.860 --> 00:08:04.490
b minus A*x minus w is there,
and this was the b minus A*x.
00:08:08.410 --> 00:08:15.830
And this identity holds.
00:08:15.830 --> 00:08:18.510
It's gotta be, like,
Pythagoras could
00:08:18.510 --> 00:08:26.150
have figured it out, but how?
00:08:26.150 --> 00:08:30.860
Let me draw this very
same picture again.
00:08:30.860 --> 00:08:41.240
So it's this line, this line,
this line, and this line.
00:08:41.240 --> 00:08:49.180
And let me just call
those a, b, c, and d.
00:08:49.180 --> 00:08:52.140
And they connect.
00:08:52.140 --> 00:08:58.210
Oh sorry, there's a
rather important fact.
00:08:58.210 --> 00:09:03.170
So this statement here
then says that a squared --
00:09:03.170 --> 00:09:06.660
no what does it say?
00:09:06.660 --> 00:09:10.610
a squared plus c squared,
maybe, is the first two,
00:09:10.610 --> 00:09:14.840
and this one is b
squared plus d squared.
00:09:14.840 --> 00:09:17.060
And why?
00:09:17.060 --> 00:09:21.950
Please submit a proof
to get an A. OK.
00:09:21.950 --> 00:09:24.380
Let me get the picture right.
00:09:24.380 --> 00:09:28.470
So this is just
dotted lines here.
00:09:28.470 --> 00:09:33.580
So that's a rectangle,
and there is d.
00:09:33.580 --> 00:09:39.700
So I have four lines
starting at the same point.
00:09:39.700 --> 00:09:43.770
Two of the lines go to the
corners of a rectangle.
00:09:43.770 --> 00:09:49.650
So it's a geometry problem,
which struck me early this
00:09:49.650 --> 00:09:52.320
morning -- too
early this morning.
00:09:52.320 --> 00:10:00.360
But it must be, of course, it
can't be news to the world.
00:10:00.360 --> 00:10:04.590
But I think I've got it right:
a squared plus c squared
00:10:04.590 --> 00:10:07.920
equals b squared plus d squared.
00:10:07.920 --> 00:10:12.370
We take any point and
connect to the four corners
00:10:12.370 --> 00:10:17.630
of a rectangle, and that holds.
00:10:17.630 --> 00:10:19.610
And the question is why?
00:10:22.400 --> 00:10:25.730
It holds from
algebra, but somehow
00:10:25.730 --> 00:10:31.230
we also ought to be able to
get it out of Pythagoras.
00:10:31.230 --> 00:10:39.860
So as far as I know none of
these angles are special.
00:10:39.860 --> 00:10:43.880
That might look equal to that
or something, but it's not,
00:10:43.880 --> 00:10:44.830
I don't think.
00:10:44.830 --> 00:10:45.720
I don't think.
00:10:45.720 --> 00:10:49.890
You know, that point is off --
this d point, it's anywhere.
00:10:49.890 --> 00:10:53.000
It's off center, and
here's the rectangle
00:10:53.000 --> 00:10:56.670
that it's connecting
to the corners of.
00:10:56.670 --> 00:10:58.010
So there you go.
00:10:58.010 --> 00:10:59.440
Open problem.
00:10:59.440 --> 00:11:00.420
Why is that true?
00:11:03.350 --> 00:11:09.150
And I'm sure it is, so this
isn't a wild goose chase,
00:11:09.150 --> 00:11:16.170
but I'm just wondering what
proofs can we find for that.
00:11:16.170 --> 00:11:16.800
OK.
00:11:16.800 --> 00:11:18.730
Can I leave you with that?
00:11:18.730 --> 00:11:21.340
I'll leave that on
the board and hope
00:11:21.340 --> 00:11:27.950
you'll listen to my presentation
about ill-posed problems,
00:11:27.950 --> 00:11:32.060
but I hope you copied
that little picture
00:11:32.060 --> 00:11:43.540
and will either email me
or hard copy, or whatever.
00:11:43.540 --> 00:11:46.050
Anyway let me know a
good way to prove it.
00:11:46.050 --> 00:11:46.940
OK.
00:11:46.940 --> 00:11:48.600
Now the lecture begins.
00:11:48.600 --> 00:11:50.090
Today's lecture begins.
00:11:50.090 --> 00:11:59.800
Oh, I had one other comment
about the interior point method
00:11:59.800 --> 00:12:02.260
with the barrier problem.
00:12:02.260 --> 00:12:07.740
I got down, at the
end, to an equation --
00:12:07.740 --> 00:12:12.930
you remember I was
taking the derivative,
00:12:12.930 --> 00:12:19.990
solving the barrier problem,
which was minimizing c*x minus
00:12:19.990 --> 00:12:23.170
some multiple of
the barrier log x_i.
00:12:28.650 --> 00:12:34.170
So I took the derivative of
this quantity and set it to 0.
00:12:34.170 --> 00:12:44.830
And the derivative in
respect to x gave me
00:12:44.830 --> 00:12:48.980
the equation c
equals alpha over x.
00:12:48.980 --> 00:12:51.980
Anyway, I kind of lost my
nerve, but all I want to say
00:12:51.980 --> 00:12:57.590
is this is right and it
leads to Newton's method
00:12:57.590 --> 00:13:00.550
that we spoke about.
00:13:00.550 --> 00:13:03.020
I won't go back to that.
00:13:03.020 --> 00:13:04.870
All right.
00:13:04.870 --> 00:13:05.740
Inverse problems.
00:13:08.610 --> 00:13:12.320
I think maybe the
best thing I can
00:13:12.320 --> 00:13:17.940
do in one lecture
about inverse problems
00:13:17.940 --> 00:13:22.150
is, first of all, to get
a general picture of what
00:13:22.150 --> 00:13:25.070
are they.
00:13:25.070 --> 00:13:32.360
Secondly, to mention areas that
we will all know about, where
00:13:32.360 --> 00:13:35.180
these inverse problems enter.
00:13:35.180 --> 00:13:37.870
And then thirdly,
to look a little bit
00:13:37.870 --> 00:13:44.570
at the integral
equations that often
00:13:44.570 --> 00:13:46.130
describe inverse problems.
00:13:46.130 --> 00:13:48.860
Inverse problems come from
many sources, not only
00:13:48.860 --> 00:13:51.490
integral equations,
but integral equations
00:13:51.490 --> 00:13:54.270
are responsible for quite a few.
00:13:54.270 --> 00:13:56.160
But let's think about others.
00:13:56.160 --> 00:14:00.590
OK, so really, I plan now
to list various examples.
00:14:04.390 --> 00:14:06.690
And number one
I've already spoken
00:14:06.690 --> 00:14:16.417
about: find velocities
from positions.
00:14:16.417 --> 00:14:17.250
Take the derivative.
00:14:22.430 --> 00:14:30.340
So taking the derivative is
a process that makes things
00:14:30.340 --> 00:14:45.850
bigger, so when we go
the other way we're --
00:14:45.850 --> 00:14:51.690
so the difficulty with the
problem is that a small change
00:14:51.690 --> 00:14:56.450
in the position data may
be a very large change
00:14:56.450 --> 00:14:57.250
in the velocity.
00:14:57.250 --> 00:15:01.130
For example,
suppose the position
00:15:01.130 --> 00:15:09.920
is the correct
position, say x of t,
00:15:09.920 --> 00:15:11.770
plus some noise
term that's going
00:15:11.770 --> 00:15:19.370
to be small, small in
size, small in amplitude,
00:15:19.370 --> 00:15:24.180
but not small in derivative.
00:15:24.180 --> 00:15:30.980
Maybe like sine
of t over epsilon.
00:15:30.980 --> 00:15:36.940
So that would be a case in
which a small noise term
00:15:36.940 --> 00:15:41.040
in the position -- so
this is the position --
00:15:41.040 --> 00:15:44.120
has a big effect
on the derivative.
00:15:44.120 --> 00:15:47.160
And that's why the
problem is ill posed.
00:15:47.160 --> 00:15:52.080
The problem is ill posed -- and
it goes with our intuition that
00:15:52.080 --> 00:15:57.300
high frequency -- this 1 over
epsilon down below is producing
00:15:57.300 --> 00:15:59.610
a high frequency
oscillation here.
00:15:59.610 --> 00:16:02.510
And of course,
everybody realizes
00:16:02.510 --> 00:16:09.520
the velocity is
the correct, dx/dt,
00:16:09.520 --> 00:16:14.230
plus the derivative of
this, which brings out a 1
00:16:14.230 --> 00:16:17.010
over epsilon, maybe
it's a cosine.
00:16:19.720 --> 00:16:21.640
Maybe it's a cosine.
00:16:21.640 --> 00:16:25.520
So the 1 over epsilon
cancels the epsilon; that's
00:16:25.520 --> 00:16:27.070
a cosine of t over epsilon.
00:16:29.810 --> 00:16:31.290
So this was small.
00:16:31.290 --> 00:16:37.430
A small change in position
produced an order of 1 change
00:16:37.430 --> 00:16:39.860
in the velocity.
00:16:39.860 --> 00:16:44.170
So if we only know
position within epsilon,
00:16:44.170 --> 00:16:46.070
we're in trouble.
00:16:46.070 --> 00:16:49.360
And this is exactly the
point of ill-posed problems.
00:16:49.360 --> 00:16:52.230
Then our velocity could be that.
00:16:52.230 --> 00:16:56.120
I could make this example worse
if I increase the frequency
00:16:56.120 --> 00:16:58.420
further -- put an
epsilon squared there.
00:17:01.430 --> 00:17:04.580
Then the amplitude
would still be small,
00:17:04.580 --> 00:17:08.660
but when I take the derivative,
there'd be a 1 over epsilon,
00:17:08.660 --> 00:17:12.420
the amplitude would
actually be very large.
00:17:12.420 --> 00:17:17.140
So I was just modest to keep
epsilon and epsilon there,
00:17:17.140 --> 00:17:20.270
so that they canceled
each other and produced
00:17:20.270 --> 00:17:22.850
an effect on the derivative.
00:17:22.850 --> 00:17:23.810
So that's the problem.
00:17:28.240 --> 00:17:31.690
If you have noisy
data about position,
00:17:31.690 --> 00:17:34.010
how can you get velocity?
00:17:34.010 --> 00:17:37.360
OK, so that's like
example number one.
00:17:37.360 --> 00:17:41.560
It's very important,
and I actually, I
00:17:41.560 --> 00:17:45.000
have no magic recipe for it.
00:17:45.000 --> 00:17:50.810
But let me mention other
problems that you'll know about
00:17:50.810 --> 00:17:54.770
like, well, seismology.
00:18:02.360 --> 00:18:05.310
A typical inverse
problem in seismology
00:18:05.310 --> 00:18:18.960
would be find the densities,
find earth density,
00:18:18.960 --> 00:18:31.680
say from travel times of waves,
from wave travel time, which
00:18:31.680 --> 00:18:33.410
is what we can measure.
00:18:33.410 --> 00:18:36.360
So that's seismology
and also, of course,
00:18:36.360 --> 00:18:40.940
everybody understands
that this is what
00:18:40.940 --> 00:18:45.620
oil exploration depends on.
00:18:45.620 --> 00:18:52.530
You set off an explosion on
the surface of the earth,
00:18:52.530 --> 00:18:56.900
the wave travels into the
earth and some part of it
00:18:56.900 --> 00:19:01.330
bounces back, and in
fact, maybe several pieces
00:19:01.330 --> 00:19:03.410
bounce back at different times.
00:19:03.410 --> 00:19:11.750
And from those results you
have to sort of back project
00:19:11.750 --> 00:19:13.560
to find the density.
00:19:13.560 --> 00:19:15.660
So back projection
is a word that
00:19:15.660 --> 00:19:18.180
comes into several of
these applications.
00:19:18.180 --> 00:19:28.050
Of course, another one would
be the medical ones: CT scans,
00:19:28.050 --> 00:19:37.630
MRI, PET -- all these
ways to take measurements.
00:19:37.630 --> 00:19:42.980
And from those measurements
you have to find the density,
00:19:42.980 --> 00:19:46.120
so you're looking
for density of tissue
00:19:46.120 --> 00:19:50.260
because you hope that would
allow you to distinguish
00:19:50.260 --> 00:19:54.190
a tumor from normal tissue.
00:19:54.190 --> 00:19:58.320
So that's a giant
area of applications.
00:19:58.320 --> 00:20:05.450
Oh, another one would be find
the density of the earth --
00:20:05.450 --> 00:20:10.270
let's say another way to find
the density of the earth,
00:20:10.270 --> 00:20:15.300
another bit of
information we have --
00:20:15.300 --> 00:20:17.870
from the gravitational field.
00:20:17.870 --> 00:20:25.540
You see, that's
what we can measure:
00:20:25.540 --> 00:20:33.970
the effect of gravity, the
effect of the earth's density.
00:20:33.970 --> 00:20:36.930
We measure the effect, and
we want to know the cause.
00:20:36.930 --> 00:20:39.010
That's the problem.
00:20:39.010 --> 00:20:44.000
And this reminds me that there
is a special lecture coming
00:20:44.000 --> 00:20:51.900
by Professor Wunsch, Carl Wunsch
at MIT -- he's outstanding --
00:20:51.900 --> 00:20:57.710
and that's Wednesday,
May 10 at 4 o'clock.
00:21:03.210 --> 00:21:06.840
And in fact, his abstract,
which you might see somewhere --
00:21:06.840 --> 00:21:13.270
I can post it on the course
website -- his abstract tells,
00:21:13.270 --> 00:21:16.310
he's solving a very,
very large-scale,
00:21:16.310 --> 00:21:19.200
ill-posed optimization problem.
00:21:19.200 --> 00:21:20.940
Least squares problem.
00:21:20.940 --> 00:21:23.340
Perfect for this course.
00:21:23.340 --> 00:21:28.170
So those are familiar.
00:21:28.170 --> 00:21:30.320
The books I've been
looking at just
00:21:30.320 --> 00:21:32.200
list whole lots of examples.
00:21:32.200 --> 00:21:32.740
Let me see.
00:21:32.740 --> 00:21:34.680
Oh, scattering.
00:21:34.680 --> 00:21:36.970
Let me just keep going here.
00:21:36.970 --> 00:21:40.090
This is number five.
00:21:40.090 --> 00:21:42.520
Scattering.
00:21:42.520 --> 00:21:49.430
From scattering
data, find the shape
00:21:49.430 --> 00:21:57.610
of the obstacle that's
responsible for the scattering.
00:21:57.610 --> 00:22:02.080
So that's a giant example with
many air force applications
00:22:02.080 --> 00:22:04.460
and many other applications.
00:22:04.460 --> 00:22:11.020
But we recognize, if
you want to identify
00:22:11.020 --> 00:22:19.610
some object by scattering, by
radar data and other scattering
00:22:19.610 --> 00:22:21.890
data.
00:22:21.890 --> 00:22:25.770
Well, there are
just lots of others.
00:22:28.470 --> 00:22:37.910
Oh, and then the general
question of: we have a Laplace
00:22:37.910 --> 00:22:45.080
or a Poisson equation,
which is the divergence
00:22:45.080 --> 00:22:57.990
of some inhomogeneous material
property equals some f of x, y.
00:22:57.990 --> 00:23:03.610
OK, so what we're usually
doing, in this course and most
00:23:03.610 --> 00:23:07.550
courses, is the direct
problem of find u,
00:23:07.550 --> 00:23:12.710
so the direct problem is find u.
00:23:15.920 --> 00:23:18.410
And what's the inverse problem?
00:23:21.550 --> 00:23:28.791
The inverse problem is we know
u and f and we have to find c.
00:23:28.791 --> 00:23:29.790
So find the coefficient.
00:23:39.140 --> 00:23:46.850
Well, how much information --
it may not be instantly clear
00:23:46.850 --> 00:23:49.350
whether it's possible.
00:23:49.350 --> 00:23:51.100
In fact, it probably
is not possible,
00:23:51.100 --> 00:23:53.550
and that's what makes
the problem ill posed.
00:23:53.550 --> 00:24:00.530
Yet if you have
enough measurements
00:24:00.530 --> 00:24:08.310
of inputs and outputs, you
could reconstruct the matrix.
00:24:08.310 --> 00:24:11.430
Of course, a person like me is
going to think about the matrix
00:24:11.430 --> 00:24:12.440
question.
00:24:12.440 --> 00:24:18.640
Suppose I'm looking
for the matrix A.
00:24:18.640 --> 00:24:22.190
Can I call this number seven?
00:24:22.190 --> 00:24:24.740
And since it's in
matrix notation,
00:24:24.740 --> 00:24:26.260
it doesn't take much space.
00:24:31.100 --> 00:24:36.520
Usually I know the matrix,
and I know b and I want x.
00:24:36.520 --> 00:24:39.550
In the inverse
problem I know b and x
00:24:39.550 --> 00:24:41.640
and I want to know the matrix.
00:24:41.640 --> 00:24:43.840
What was the matrix
that produced it.
00:24:43.840 --> 00:24:46.630
Well obviously one
pair b, x is not
00:24:46.630 --> 00:24:52.530
going to be enough to produce
the matrix, but enough will.
00:24:52.530 --> 00:24:55.950
But then if there's noise --
this is the point, of course,
00:24:55.950 --> 00:24:57.190
that there's always noise.
00:24:57.190 --> 00:25:01.140
So that's what I now
have to deal with.
00:25:01.140 --> 00:25:07.830
The main thing is how to deal
with noise in the data, noise
00:25:07.830 --> 00:25:11.740
in the measurements, because
if the problem is ill posed,
00:25:11.740 --> 00:25:15.990
we saw even in that
simple cooked up example
00:25:15.990 --> 00:25:21.800
that a small amount of noise
could produce a big difference
00:25:21.800 --> 00:25:28.290
in the solution.
00:25:28.290 --> 00:25:29.300
OK.
00:25:29.300 --> 00:25:29.800
Right.
00:25:29.800 --> 00:25:32.980
So those are examples.
00:25:32.980 --> 00:25:37.650
Now I wanted -- because math
courses and this one never
00:25:37.650 --> 00:25:40.500
mention integral equations,
I thought I would write one
00:25:40.500 --> 00:25:42.830
on the board.
00:25:42.830 --> 00:25:47.180
And these examples
fit in this --
00:25:47.180 --> 00:25:49.860
if I describe them
mathematically or another whole
00:25:49.860 --> 00:25:55.580
list of problems that I'm
seeing in the books on ill-posed
00:25:55.580 --> 00:26:01.480
problems -- very often they are
integral equations of the first
00:26:01.480 --> 00:26:02.960
kind.
00:26:02.960 --> 00:26:09.250
Again, the direct problem is
-- I'd better write it down --
00:26:09.250 --> 00:26:12.510
the direct problem
-- this is known.
00:26:16.060 --> 00:26:24.930
So the direct problem
is given the K, which
00:26:24.930 --> 00:26:31.840
is a bit like c over here,
find the u, solve for u.
00:26:34.930 --> 00:26:36.720
Solve for the unknown u.
00:26:36.720 --> 00:26:38.350
It's a linear problem.
00:26:38.350 --> 00:26:45.340
It's an A*x equals b problem,
only it's in function space.
00:26:45.340 --> 00:26:49.090
And of course, one way
to solve it will be,
00:26:49.090 --> 00:26:50.950
probably the way to
solve it numerically
00:26:50.950 --> 00:26:53.650
will be somehow
make it discrete,
00:26:53.650 --> 00:26:56.490
bring it down to
a matrix problem.
00:26:56.490 --> 00:26:59.650
That's what we
would eventually do.
00:26:59.650 --> 00:27:04.880
But in function space,
integral equations
00:27:04.880 --> 00:27:07.700
played a very, very
important historical role
00:27:07.700 --> 00:27:11.590
in the development
of function spaces.
00:27:11.590 --> 00:27:15.700
And now, then the
inverse problem would
00:27:15.700 --> 00:27:24.450
be given u find the x, I guess.
00:27:24.450 --> 00:27:25.350
Something like that.
00:27:25.350 --> 00:27:26.960
That would be possible.
00:27:26.960 --> 00:27:32.670
That would be one possible:
inverse number one.
00:27:32.670 --> 00:27:36.470
But I wanted to make some
comments on integral equations,
00:27:36.470 --> 00:27:39.580
just so you would
have seen them.
00:27:39.580 --> 00:27:44.460
The integral could go up to
x or it could go up to what?
00:27:47.860 --> 00:27:52.170
And Volterra and Fredholm are
the names associated with those
00:27:52.170 --> 00:27:55.700
two possibilities,
but these are both --
00:27:55.700 --> 00:28:04.740
whether Volterra or Fredholm
-- they're both ill posed,
00:28:04.740 --> 00:28:07.510
whereas if I want to make
them well posed -- well,
00:28:07.510 --> 00:28:11.130
we've seen how to make
a problem well posed.
00:28:11.130 --> 00:28:15.940
I have this operator A,
which has a terrible inverse
00:28:15.940 --> 00:28:19.390
or no inverse at
all, and the way
00:28:19.390 --> 00:28:24.620
I improve it is add a little
multiple of the identity.
00:28:27.590 --> 00:28:34.110
I'm supposing that I know about
A, that it's not negative,
00:28:34.110 --> 00:28:39.550
that its eigenvalues can be
very, very small or 0 but not
00:28:39.550 --> 00:28:40.660
negative.
00:28:40.660 --> 00:28:45.050
So when I add a little bit,
it pushes the eigenvalues away
00:28:45.050 --> 00:28:49.330
from 0 up at least
as far as alpha.
00:28:49.330 --> 00:28:59.120
And over here, if I add in alpha
u of x, that's what it does,
00:28:59.120 --> 00:29:00.970
of course.
00:29:00.970 --> 00:29:05.180
I've added alpha times
the identity operator here
00:29:05.180 --> 00:29:09.750
and that's given me an
equation of the second kind.
00:29:09.750 --> 00:29:21.420
So those three minutes were just
to say something about language
00:29:21.420 --> 00:29:26.530
and to look at an integral
equation, something
00:29:26.530 --> 00:29:29.950
we don't do enough of.
00:29:29.950 --> 00:29:37.440
Integral equations -- well,
some problems on nice domains,
00:29:37.440 --> 00:29:40.710
you can turn differential
equations into integral
00:29:40.710 --> 00:29:45.610
equations, and it pays
off big time to do that.
00:29:45.610 --> 00:29:51.700
Professor White in EE, if he
taught a course like this,
00:29:51.700 --> 00:29:54.260
it would end up with
half a dozen lectures
00:29:54.260 --> 00:29:55.880
on integral equations
because he's
00:29:55.880 --> 00:30:00.360
an expert in converting
the differential equation
00:30:00.360 --> 00:30:01.590
to an integral equation.
00:30:01.590 --> 00:30:04.760
He would convert
Laplace's equation
00:30:04.760 --> 00:30:09.190
to an integral equation, and
the Green's function would enter
00:30:09.190 --> 00:30:16.220
and he would solve it there.
00:30:16.220 --> 00:30:19.980
OK, so how does velocity fit?
00:30:19.980 --> 00:30:24.210
Well, everybody can see that
the integral of velocity --
00:30:24.210 --> 00:30:30.970
so example, the velocity example
is that the integral from 0
00:30:30.970 --> 00:30:39.140
to x of the velocity, that's
of course integral from 0 to x
00:30:39.140 --> 00:30:48.620
of -- or 0 to t maybe
would be a better --
00:30:48.620 --> 00:30:53.530
so suppose velocity
is d position, dx/ds.
00:30:53.530 --> 00:31:00.240
So it's x of t minus x of 0.
00:31:00.240 --> 00:31:02.390
So this is position.
00:31:07.890 --> 00:31:11.090
This is velocity.
00:31:11.090 --> 00:31:19.850
And in the inverse problem,
position is known, say by GPS,
00:31:19.850 --> 00:31:29.150
and velocity is
unknown, to find by GPS.
00:31:29.150 --> 00:31:34.850
So GPS will give you a
measurement of position.
00:31:34.850 --> 00:31:37.720
Maybe you know
something about GPS.
00:31:37.720 --> 00:31:42.780
You know that there
are satellites
00:31:42.780 --> 00:31:50.890
up there whose position
is known very exactly,
00:31:50.890 --> 00:31:54.510
and they have a very very
accurate atomic clock on them,
00:31:54.510 --> 00:31:59.010
so that times are
accurately known.
00:31:59.010 --> 00:32:04.000
So they send signals down to
your little hundred dollar
00:32:04.000 --> 00:32:09.520
receiver, which of course
has a ten-dollar clock in it.
00:32:09.520 --> 00:32:17.440
So there'll be some errors,
partly due to the clock,
00:32:17.440 --> 00:32:24.260
largely due to the
cheap time keeper.
00:32:24.260 --> 00:32:28.930
But actually the way you
get real accuracy out of GPS
00:32:28.930 --> 00:32:32.400
is to have two
receivers, and then you
00:32:32.400 --> 00:32:41.950
can cancel the clock errors and
get less than a meter accuracy.
00:32:41.950 --> 00:32:44.552
If you take account of
all the sources of error,
00:32:44.552 --> 00:32:46.010
you can get it down
to centimeters.
00:32:49.930 --> 00:32:56.990
So GPS is giving you -- just
your single receiver is still
00:32:56.990 --> 00:32:58.120
good enough.
00:32:58.120 --> 00:33:06.005
It's measuring the travel
time and since the signals
00:33:06.005 --> 00:33:08.560
are coming with
the speed of light,
00:33:08.560 --> 00:33:12.360
that tells us the distance
from each satellite.
00:33:12.360 --> 00:33:12.860
Right?
00:33:12.860 --> 00:33:18.240
Here is your receiver R. Here
is satellite number one, two,
00:33:18.240 --> 00:33:27.290
three, four up in the sky,
and you know these distances.
00:33:34.670 --> 00:33:38.150
But you don't know
the time very well.
00:33:38.150 --> 00:33:42.870
So with four satellites,
I'm able to find
00:33:42.870 --> 00:33:47.580
the position of the
receiver is somewhere
00:33:47.580 --> 00:33:51.910
in space and some moment of
time that this clock is not
00:33:51.910 --> 00:33:53.040
good enough to tell us.
00:33:53.040 --> 00:33:54.710
So we have to solve for that.
00:33:54.710 --> 00:34:04.360
So four receivers sending
signals to -- I'm sorry,
00:34:04.360 --> 00:34:08.120
four satellites sending
signals to the receiver,
00:34:08.120 --> 00:34:14.970
we can solve that problem and
find the position and time.
00:34:14.970 --> 00:34:21.170
That's the fundamental
idea of GPS.
00:34:21.170 --> 00:34:24.210
Now of course, you
get better results
00:34:24.210 --> 00:34:29.010
if there's a fifth receiver, a
fifth satellite, and a sixth.
00:34:29.010 --> 00:34:30.720
The more the better.
00:34:30.720 --> 00:34:34.630
And of course then it's
going to be least squares.
00:34:34.630 --> 00:34:36.970
Because you're still
looking for four unknowns,
00:34:36.970 --> 00:34:43.300
but now you have six
distances, pseudo-ranges,
00:34:43.300 --> 00:34:49.540
so we would use least
squares, so by least squares.
00:34:49.540 --> 00:34:50.040
OK.
00:34:52.690 --> 00:34:54.390
But what about velocity?
00:34:54.390 --> 00:35:04.880
Suppose your receiver is
moving, as of course it
00:35:04.880 --> 00:35:11.220
is if you rent a car
that has GPS installed
00:35:11.220 --> 00:35:17.200
to tell you where to turn.
00:35:17.200 --> 00:35:18.860
And of course, it
has to have a map
00:35:18.860 --> 00:35:30.290
system installed so that it can
look up for the map position.
00:35:30.290 --> 00:35:34.900
So for many purposes
you need velocity,
00:35:34.900 --> 00:35:39.610
and I'm not an expert
on that subject at all.
00:35:39.610 --> 00:35:47.930
I just comment that one way
to get velocity is to take
00:35:47.930 --> 00:35:55.306
differences, so the velocity is
approximately x of t plus delta
00:35:55.306 --> 00:36:01.040
t minus x -- or x
of t_2, let's say,
00:36:01.040 --> 00:36:08.040
x of t_2 minus x of
t_1 over t_2 minus t_1.
00:36:08.040 --> 00:36:14.180
But if we want to get
velocity near a certain time,
00:36:14.180 --> 00:36:17.600
then these t's better
be near that time,
00:36:17.600 --> 00:36:21.760
because the velocity's
changing, so we better
00:36:21.760 --> 00:36:27.000
be measuring it at the
time we're wanting it.
00:36:27.000 --> 00:36:31.680
And then, if they're very close,
then we're dividing by a small
00:36:31.680 --> 00:36:38.080
number and the
difference -- the noise,
00:36:38.080 --> 00:36:45.755
the error in measurements x, is
multiplied by that 1 over delta
00:36:45.755 --> 00:36:47.750
t.
00:36:47.750 --> 00:36:56.670
And one way to avoid it is to
go into the frequency domain.
00:36:56.670 --> 00:36:58.940
So this is like an
interesting option
00:36:58.940 --> 00:37:01.560
in a lot of these problems.
00:37:01.560 --> 00:37:05.230
Can you operate better
in the frequency domain?
00:37:05.230 --> 00:37:09.070
Of course, the ill-posedness
is not going to go away.
00:37:09.070 --> 00:37:12.010
It comes as we saw
from high frequency.
00:37:12.010 --> 00:37:15.010
But if we go into
the frequency domain,
00:37:15.010 --> 00:37:20.000
and if these GPS satellites are
sending at a certain frequency
00:37:20.000 --> 00:37:26.090
and as we move, of course --
the Doppler effect, of course,
00:37:26.090 --> 00:37:31.020
is the fact that as
the receiver moves,
00:37:31.020 --> 00:37:36.380
the frequency it
observes change a little.
00:37:36.380 --> 00:37:36.930
Right?
00:37:36.930 --> 00:37:40.370
Just says, like, the
noise of a train going by
00:37:40.370 --> 00:37:43.340
is the familiar example.
00:37:43.340 --> 00:37:46.310
Nobody ever sees a
train going by anymore.
00:37:46.310 --> 00:37:52.110
But it's the same idea.
00:37:52.110 --> 00:37:55.740
But you hear traffic go by.
00:37:55.740 --> 00:37:59.300
Actually, I guess that that's
how we cross the street, come
00:37:59.300 --> 00:38:03.850
to think of it, by listening
to the noise of cars,
00:38:03.850 --> 00:38:07.450
and our global
internal Doppler says
00:38:07.450 --> 00:38:10.650
the cars are going away from us,
in which case we don't worry,
00:38:10.650 --> 00:38:14.230
or it says the car's
coming fast, in which case
00:38:14.230 --> 00:38:18.520
we're careful, or it says
the car's coming slowly,
00:38:18.520 --> 00:38:21.480
and we get across first.
00:38:21.480 --> 00:38:25.390
So we use Doppler.
00:38:25.390 --> 00:38:33.150
I don't know exactly how, how
our human audio system builds
00:38:33.150 --> 00:38:34.110
in Doppler.
00:38:34.110 --> 00:38:38.680
Anyway, Doppler would be
a change to the frequency
00:38:38.680 --> 00:38:44.590
domain and a restatement
and perhaps an improvement
00:38:44.590 --> 00:38:46.180
in the problem.
00:38:46.180 --> 00:38:52.470
OK, so those are
examples without math.
00:38:52.470 --> 00:38:58.940
Now here's a small bit of
math as the lecture ends.
00:38:58.940 --> 00:39:04.330
So the only math was
this simple example.
00:39:04.330 --> 00:39:07.040
So I guess I got one
more board over here.
00:39:07.040 --> 00:39:10.500
I'm going to put this geometry
problem that you've been
00:39:10.500 --> 00:39:14.350
thinking about out of sight.
00:39:14.350 --> 00:39:18.170
OK, so what's the key?
00:39:25.160 --> 00:39:26.130
It's this Tikhonov.
00:39:30.120 --> 00:39:31.610
Tikhonov Regularization.
00:39:37.300 --> 00:39:40.950
Tikhonov Regularization.
00:39:40.950 --> 00:39:42.760
And it's adding alpha.
00:39:46.370 --> 00:39:51.700
Add alpha to the
least squares problem.
00:39:51.700 --> 00:39:56.000
And I thought it was
amusing to notice,
00:39:56.000 --> 00:40:03.120
Tikhonov was born in 1906, so
this is a hundred years exactly
00:40:03.120 --> 00:40:08.090
since he proposed this method.
00:40:08.090 --> 00:40:16.070
It's one of about five methods
that the books describe.
00:40:16.070 --> 00:40:19.800
And it's the one I'll speak
about here at the end.
00:40:19.800 --> 00:40:26.960
I'll just mention that one
which might come up in a project
00:40:26.960 --> 00:40:29.010
possibly.
00:40:29.010 --> 00:40:32.930
Other methods are used
in iterative methods,
00:40:32.930 --> 00:40:38.020
like conjugate gradients,
and stop when you're ahead.
00:40:38.020 --> 00:40:41.410
See if you push conjugate
gradients on and on and on,
00:40:41.410 --> 00:40:48.600
then eventually it's
going to produce
00:40:48.600 --> 00:40:53.270
your exact ill-posed
matrix with the big inverse
00:40:53.270 --> 00:40:55.590
and unrealistic solution.
00:40:58.240 --> 00:40:59.340
So you stop.
00:40:59.340 --> 00:41:03.060
The same way for an
integral equation.
00:41:03.060 --> 00:41:09.090
We discretize that so we
get a matrix product, which
00:41:09.090 --> 00:41:10.900
we can solve.
00:41:10.900 --> 00:41:17.460
But if we refine the mesh so
that the matrix gets bigger,
00:41:17.460 --> 00:41:19.100
it gets more ill posed.
00:41:19.100 --> 00:41:23.080
So the closer we get, the
closer the discrete problem
00:41:23.080 --> 00:41:28.020
gets to the true integral
equation, the more sick it is.
00:41:28.020 --> 00:41:31.770
So there's some point
at which you are OK,
00:41:31.770 --> 00:41:36.230
and then if you go too
far, you're worse off.
00:41:36.230 --> 00:41:39.750
So that happens with
conjugate gradients.
00:41:39.750 --> 00:41:41.720
Now what's the Tikhonov idea?
00:41:41.720 --> 00:41:46.180
So the Tikhonov idea is: look
at them -- as you all know --
00:41:46.180 --> 00:41:53.440
it's the minimum -- I'll
just call it A again --
00:41:53.440 --> 00:41:58.910
A*x minus b squared
plus alpha x squared.
00:41:58.910 --> 00:42:03.570
That would be the
simplest penalty
00:42:03.570 --> 00:42:05.900
term, regularization term.
00:42:05.900 --> 00:42:11.690
So this leads to
the normal equation:
00:42:11.690 --> 00:42:22.050
A transpose A plus alpha*I, u
hat, which depends on alpha,
00:42:22.050 --> 00:42:23.930
equals b.
00:42:23.930 --> 00:42:24.550
OK.
00:42:29.000 --> 00:42:31.580
So u_alpha is the
solution to that.
00:42:31.580 --> 00:42:36.090
OK, now let's let noise come in.
00:42:36.090 --> 00:42:39.570
So noise in b.
00:42:39.570 --> 00:42:50.230
Noise yields a b_delta, say
a delta amount of noise.
00:42:50.230 --> 00:42:55.550
A b_delta is the
measured observation.
00:42:55.550 --> 00:42:57.600
See, we don't know the exact b.
00:42:57.600 --> 00:42:59.910
This is what we measured.
00:42:59.910 --> 00:43:08.370
And we can suppose that the size
of the noise is -- let's say --
00:43:08.370 --> 00:43:09.470
delta.
00:43:09.470 --> 00:43:12.090
So delta measures the noise.
00:43:12.090 --> 00:43:12.590
OK.
00:43:16.300 --> 00:43:18.880
I mean what's my question here?
00:43:18.880 --> 00:43:23.150
Always the question is:
what do you take for alpha?
00:43:23.150 --> 00:43:26.260
What should that parameter be?
00:43:26.260 --> 00:43:29.170
If you take alpha
very small or 0,
00:43:29.170 --> 00:43:34.130
then your problem is ill
posed, and the answer you get
00:43:34.130 --> 00:43:37.940
is destroyed by the noise.
00:43:37.940 --> 00:43:44.570
If you take alpha very large,
then you're overriding the real
00:43:44.570 --> 00:43:49.160
problem -- you're
over-regularizing it.
00:43:49.160 --> 00:43:51.330
You're over-smoothing it.
00:43:51.330 --> 00:43:53.620
So we don't want to
take alpha too large,
00:43:53.620 --> 00:43:57.050
but we can't take
alpha too small either.
00:43:57.050 --> 00:44:07.720
And the theory will say
that if we use an alpha --
00:44:07.720 --> 00:44:12.380
so the theory will say
this, essentially this.
00:44:12.380 --> 00:44:17.450
It'll say that the u hat alpha,
the difference between u hat
00:44:17.450 --> 00:44:21.950
alpha and u hat alpha with
the noise, coming from the --
00:44:21.950 --> 00:44:23.800
do you see what I mean by it?
00:44:23.800 --> 00:44:27.200
This comes from the true
b, which we don't know,
00:44:27.200 --> 00:44:30.270
but it's the answer
we would like to know.
00:44:30.270 --> 00:44:34.700
This comes from the
measured b with noise in it,
00:44:34.700 --> 00:44:40.200
and it turns out that
this is of size delta
00:44:40.200 --> 00:44:43.280
over square root of alpha.
00:44:43.280 --> 00:44:48.090
That's a rather neat
result. It gives us a guide
00:44:48.090 --> 00:44:49.780
because we want
that to be small.
00:44:54.210 --> 00:44:57.300
So we're assuming that we
have some idea about delta,
00:44:57.300 --> 00:45:04.720
and it tells us, again, that
if I take alpha very small,
00:45:04.720 --> 00:45:09.340
then I'm not learning anything.
00:45:09.340 --> 00:45:12.720
So you see, actually,
that we want,
00:45:12.720 --> 00:45:19.300
we need, delta over square
root of alpha to go to 0.
00:45:23.320 --> 00:45:25.630
Maybe we're reducing the noise.
00:45:25.630 --> 00:45:27.430
Maybe we have a
sequence of measurements
00:45:27.430 --> 00:45:29.690
that get better and better.
00:45:29.690 --> 00:45:31.830
So delta goes to 0.
00:45:31.830 --> 00:45:36.610
We would like to get to
the right answer then,
00:45:36.610 --> 00:45:40.020
but as delta goes
to 0, we better not
00:45:40.020 --> 00:45:42.390
let alpha go to 0 faster.
00:45:44.950 --> 00:45:51.140
The message here is -- so I
haven't derived this estimate
00:45:51.140 --> 00:45:57.750
but just written a conclusion,
which is that as the noise goes
00:45:57.750 --> 00:46:03.730
to 0 -- that means our
measurements are getting better
00:46:03.730 --> 00:46:06.390
and better -- we do want
to let alpha go to 0,
00:46:06.390 --> 00:46:08.750
but we can't overdo it.
00:46:08.750 --> 00:46:15.460
And a good choice,
a good choice is:
00:46:15.460 --> 00:46:19.400
let alpha be delta to the 2/3.
00:46:19.400 --> 00:46:23.200
That turns out, from
these little estimates,
00:46:23.200 --> 00:46:27.550
to be the best choice, because
then square root of alpha
00:46:27.550 --> 00:46:28.480
is delta to the 1/3.
00:46:32.010 --> 00:46:37.930
This then is delta divided
by delta to the 1/3.
00:46:37.930 --> 00:46:41.330
This is delta to the 2/3,
and we can't do better.
00:46:45.560 --> 00:46:48.150
So that's the balancing.
00:46:48.150 --> 00:46:52.830
That's the balance when you
take that value of alpha.
00:46:52.830 --> 00:46:55.580
Then, as you reduce
the noise, the error
00:46:55.580 --> 00:46:59.930
gets reduced in the
same proportion.
00:46:59.930 --> 00:47:03.970
It's not the full
reduction in delta,
00:47:03.970 --> 00:47:07.670
but the fraction, the 2/3 power.
00:47:07.670 --> 00:47:12.000
OK, so that board
summarizes, you
00:47:12.000 --> 00:47:16.130
could say, the theory
of regularization.
00:47:16.130 --> 00:47:21.520
And there's a lot more
to say about that theory,
00:47:21.520 --> 00:47:27.220
but I think this is perhaps
the right point to stop.
00:47:27.220 --> 00:47:31.720
OK so that's applications.
00:47:31.720 --> 00:47:37.730
One other application, if
I can add one last one,
00:47:37.730 --> 00:47:42.330
because it's quite important and
it comes up in life sciences.
00:47:47.420 --> 00:47:49.900
So in life sciences
you might have --
00:47:49.900 --> 00:47:54.240
in genomics you might have
a lot of genes acting.
00:47:54.240 --> 00:48:00.080
So the action of n
genes, n being large,
00:48:00.080 --> 00:48:04.720
produces some expression,
the expression of the gene,
00:48:04.720 --> 00:48:12.670
and it depends on how much
of those genes are present.
00:48:12.670 --> 00:48:17.280
But what everybody wants to know
is which genes are important,
00:48:17.280 --> 00:48:26.200
which genes control the blue or
brown eyes, or male or female.
00:48:26.200 --> 00:48:29.190
Not clear, right?
00:48:29.190 --> 00:48:36.580
We have an idea from
biological experiments
00:48:36.580 --> 00:48:41.790
where the important genes
lie on the whole genome,
00:48:41.790 --> 00:48:45.620
where to find them
in the chromosome,
00:48:45.620 --> 00:48:48.520
but this is what we measure.
00:48:48.520 --> 00:48:57.490
We observe male or female,
and we can change the genes,
00:48:57.490 --> 00:49:01.660
but we can't get --
there are a lot of genes,
00:49:01.660 --> 00:49:05.970
and there are more
unknowns than,
00:49:05.970 --> 00:49:08.370
more dimensions
than sample points.
00:49:08.370 --> 00:49:11.590
We're really up against it here.
00:49:11.590 --> 00:49:15.010
And we're really up
against it because --
00:49:15.010 --> 00:49:17.400
so what's going to measure
the importance of a gene?
00:49:17.400 --> 00:49:20.480
How important is
gene number two?
00:49:20.480 --> 00:49:22.910
Well, the importance
of gene number two
00:49:22.910 --> 00:49:26.960
is identified by the
size of the derivative.
00:49:29.720 --> 00:49:34.420
That quantity, if
it's big, tells me
00:49:34.420 --> 00:49:38.200
that the expression depends
strongly on gene number two.
00:49:38.200 --> 00:49:41.800
If it's small, it says gene
number two can be ignored.
00:49:41.800 --> 00:49:45.020
That's exactly what
the Whitehead and Broad
00:49:45.020 --> 00:49:47.340
Institutes want to know.
00:49:47.340 --> 00:49:49.970
Well, for cancer,
of course, as well.
00:49:49.970 --> 00:49:55.960
And my only point
here in this lecture
00:49:55.960 --> 00:49:59.160
is to say that
again, we're trying
00:49:59.160 --> 00:50:01.720
to estimate a derivative.
00:50:01.720 --> 00:50:05.340
We're estimating a
derivative from few samples
00:50:05.340 --> 00:50:10.280
in high dimension, and
it's certainly ill posed.
00:50:10.280 --> 00:50:12.920
And it certainly has
to be regularized,
00:50:12.920 --> 00:50:16.520
and it certainly has to
be studied and solved.
00:50:16.520 --> 00:50:20.100
So that's maybe a
seventh example,
00:50:20.100 --> 00:50:26.590
and with that, I'll
stop the final lecture
00:50:26.590 --> 00:50:28.870
and just bring
down one last time
00:50:28.870 --> 00:50:37.760
this little bit of
geometry to give you
00:50:37.760 --> 00:50:40.540
something interesting
to do for the weekend.
00:50:40.540 --> 00:50:45.340
OK, so I'll see you
Monday then, with a talk
00:50:45.340 --> 00:50:52.440
about different methods for
solving linear equations
00:50:52.440 --> 00:50:53.790
and others coming.
00:50:53.790 --> 00:50:58.810
And time is, you know, this
semester will run out on us.
00:50:58.810 --> 00:51:03.770
So please send me
emails to volunteer.
00:51:03.770 --> 00:51:07.400
And don't think you
have to be perfect.
00:51:07.400 --> 00:51:11.170
If you've got transparencies,
got the topic in mind,
00:51:11.170 --> 00:51:15.160
got the question in mind,
got some numerical results,
00:51:15.160 --> 00:51:16.440
you're ready.
00:51:16.440 --> 00:51:17.071
OK.
00:51:17.071 --> 00:51:17.570
Good.
00:51:17.570 --> 00:51:18.620
Have a good weekend.
00:51:18.620 --> 00:51:19.870
Bye.