WEBVTT
00:00:03.890 --> 00:00:09.130
Yes, OK, four, three,
two, one, OK, I
00:00:09.130 --> 00:00:10.880
see you guys are
in a happy mood.
00:00:10.880 --> 00:00:13.980
I don't know if that
means 18.06 is ending,
00:00:13.980 --> 00:00:16.680
or, the quiz was good.
00:00:16.680 --> 00:00:21.030
Uh, my birthday
conference was going
00:00:21.030 --> 00:00:24.880
on at the time of the quiz, and
in the conference, of course,
00:00:24.880 --> 00:00:27.190
everybody had to
say nice things,
00:00:27.190 --> 00:00:30.390
but I was wondering,
what would my 18.06
00:00:30.390 --> 00:00:36.200
class be saying, because it was
at the exactly the same time.
00:00:36.200 --> 00:00:39.300
But, what I know from
the grades so far,
00:00:39.300 --> 00:00:45.440
they're basically close to, and
maybe slightly above the grades
00:00:45.440 --> 00:00:48.590
that you got on quiz two.
00:00:48.590 --> 00:00:52.790
So, very satisfactory.
00:00:52.790 --> 00:00:56.550
And, then we have a
final exam coming up,
00:00:56.550 --> 00:01:00.780
and today's lecture,
as I told you by email,
00:01:00.780 --> 00:01:05.019
will be a first
step in the review,
00:01:05.019 --> 00:01:07.890
and then on Wednesday
I'll do all I can
00:01:07.890 --> 00:01:13.560
in reviewing the whole course.
00:01:13.560 --> 00:01:16.300
So my topic today
is -- actually,
00:01:16.300 --> 00:01:24.740
this is a lecture I have never
given before in this way,
00:01:24.740 --> 00:01:28.310
and it will -- well,
four subspaces,
00:01:28.310 --> 00:01:32.190
that's certainly fundamental,
and you know that,
00:01:32.190 --> 00:01:34.810
so I want to speak
about left-inverses
00:01:34.810 --> 00:01:37.150
and right-inverses and
then something called
00:01:37.150 --> 00:01:39.190
pseudo-inverses.
00:01:39.190 --> 00:01:43.110
And pseudo-inverses,
let me say right away,
00:01:43.110 --> 00:01:47.240
that comes in near the
end of chapter seven,
00:01:47.240 --> 00:01:52.170
and that would not be
expected on the final.
00:01:52.170 --> 00:01:54.840
But you'll see that
what I'm talking about
00:01:54.840 --> 00:02:01.600
is really the basic stuff that,
for an m-by-n matrix of rank r,
00:02:01.600 --> 00:02:06.080
we're going back to the most
fundamental picture in linear
00:02:06.080 --> 00:02:07.130
algebra.
00:02:07.130 --> 00:02:12.020
Nobody could forget
that picture, right?
00:02:12.020 --> 00:02:15.890
When you're my age, even,
you'll remember the row space,
00:02:15.890 --> 00:02:17.560
and the null space.
00:02:17.560 --> 00:02:21.130
Orthogonal complements over
there, the column space
00:02:21.130 --> 00:02:23.530
and the null space of
A transpose column,
00:02:23.530 --> 00:02:26.190
orthogonal
complements over here.
00:02:26.190 --> 00:02:29.220
And I want to speak
about inverses.
00:02:29.220 --> 00:02:30.320
OK.
00:02:30.320 --> 00:02:33.960
And I want to identify the
different possibilities.
00:02:33.960 --> 00:02:37.980
So first of all,
when does a matrix
00:02:37.980 --> 00:02:42.440
have a just a perfect
inverse, two-sided, you know,
00:02:42.440 --> 00:02:51.600
so the two-sided inverse is what
we just call inverse, right?
00:02:51.600 --> 00:02:57.720
And, so that means that
there's a matrix that
00:02:57.720 --> 00:03:02.010
produces the identity, whether
we write it on the left
00:03:02.010 --> 00:03:03.670
or on the right.
00:03:03.670 --> 00:03:10.170
And just tell me, how
are the numbers r,
00:03:10.170 --> 00:03:16.580
the rank, n the number of
columns, m the number of rows,
00:03:16.580 --> 00:03:19.070
how are those
numbers related when
00:03:19.070 --> 00:03:21.320
we have an invertible matrix?
00:03:21.320 --> 00:03:23.690
So this is the
matrix which was --
00:03:23.690 --> 00:03:26.540
chapter two was all
about matrices like this,
00:03:26.540 --> 00:03:30.420
the beginning of the course,
what was the relation of th-
00:03:30.420 --> 00:03:36.110
of r, m, and n,
for the nice case?
00:03:36.110 --> 00:03:39.360
They're all the same, all equal.
00:03:39.360 --> 00:03:43.260
So this is the case when r=m=n.
00:03:43.260 --> 00:03:46.910
Square matrix, full
rank, period, just --
00:03:46.910 --> 00:03:50.480
so I'll use the words full rank.
00:03:50.480 --> 00:03:51.840
OK, good.
00:03:51.840 --> 00:03:53.800
Everybody knows that.
00:03:53.800 --> 00:03:55.860
OK.
00:03:55.860 --> 00:03:56.875
Then chapter three.
00:03:59.660 --> 00:04:03.240
We began to deal with matrices
that were not of full rank,
00:04:03.240 --> 00:04:07.050
and they could have any rank,
and we learned what the rank
00:04:07.050 --> 00:04:08.480
was.
00:04:08.480 --> 00:04:12.140
And then we focused,
if you remember
00:04:12.140 --> 00:04:17.110
on some cases like
full column rank.
00:04:17.110 --> 00:04:20.810
Now, can you remember what was
the deal with full column rank?
00:04:20.810 --> 00:04:25.020
So, now, I think this
is the case in which we
00:04:25.020 --> 00:04:28.650
have a left-inverse,
and I'll try to find it.
00:04:32.930 --> 00:04:35.130
So we have a --
00:04:35.130 --> 00:04:37.800
what was the situation there?
00:04:37.800 --> 00:04:43.780
It's the case of full column
rank, and that means --
00:04:43.780 --> 00:04:47.160
what does that mean about r?
00:04:47.160 --> 00:04:51.770
It equals, what's
the deal with r, now,
00:04:51.770 --> 00:04:54.460
if we have full
column rank, I mean
00:04:54.460 --> 00:04:59.730
the columns are independent,
but maybe not the rows.
00:04:59.730 --> 00:05:03.780
So what is r equal
to in this case?
00:05:03.780 --> 00:05:04.450
n.
00:05:04.450 --> 00:05:05.780
Thanks.
00:05:05.780 --> 00:05:06.480
n.
00:05:06.480 --> 00:05:06.979
r=n.
00:05:06.979 --> 00:05:10.270
The n columns are
independent, but probably, we
00:05:10.270 --> 00:05:12.920
have more rows.
00:05:12.920 --> 00:05:17.060
What's the picture, and then
what's the null space for this?
00:05:17.060 --> 00:05:19.990
So the n columns
are independent,
00:05:19.990 --> 00:05:22.155
what's the null
space in this case?
00:05:25.420 --> 00:05:27.450
So of course, you
know what I'm asking.
00:05:27.450 --> 00:05:30.690
You're saying, why is this guy
asking something, I know that--
00:05:30.690 --> 00:05:32.680
I think about it in my sleep,
00:05:32.680 --> 00:05:33.260
right?
00:05:33.260 --> 00:05:36.900
So the null space of this
matrix if the rank is
00:05:36.900 --> 00:05:47.280
n, the null space is what
vectors are in the null space?
00:05:47.280 --> 00:05:48.410
Just the zero vector.
00:05:51.320 --> 00:05:51.820
Right?
00:05:51.820 --> 00:05:53.360
The columns are independent.
00:05:53.360 --> 00:05:54.300
Independent columns.
00:05:59.020 --> 00:06:03.320
No combination of the columns
gives zero except that one.
00:06:03.320 --> 00:06:06.270
And what's my picture over, --
00:06:06.270 --> 00:06:10.050
let me redraw my picture --
00:06:10.050 --> 00:06:14.330
the row space is everything.
00:06:18.450 --> 00:06:20.860
No.
00:06:20.860 --> 00:06:23.030
Is that right?
00:06:23.030 --> 00:06:26.580
Let's see, I often get
these turned around, right?
00:06:26.580 --> 00:06:31.640
So what's the deal?
00:06:31.640 --> 00:06:34.810
The columns are
independent, right?
00:06:34.810 --> 00:06:39.330
So the rank should be the full
number of columns, so what
00:06:39.330 --> 00:06:41.250
does that tell us?
00:06:41.250 --> 00:06:43.051
There's no null space, right.
00:06:43.051 --> 00:06:43.550
OK.
00:06:43.550 --> 00:06:45.400
The row space is
the whole thing.
00:06:45.400 --> 00:06:48.510
Yes, I won't even
draw the picture.
00:06:48.510 --> 00:06:51.640
And what was the deal with --
00:06:51.640 --> 00:06:56.960
and these were very important
in least squares problems
00:06:56.960 --> 00:06:58.750
because --
00:06:58.750 --> 00:07:06.230
So, what more is true here?
00:07:06.230 --> 00:07:09.460
If we have full column rank,
the null space is zero,
00:07:09.460 --> 00:07:14.640
we have independent
columns, the unique --
00:07:14.640 --> 00:07:22.990
so we have zero or
one solutions to Ax=b.
00:07:25.790 --> 00:07:28.660
There may not be any solutions,
but if there's a solution,
00:07:28.660 --> 00:07:32.910
there's only one solution
because other solutions are
00:07:32.910 --> 00:07:35.470
found by adding on stuff
from the null space,
00:07:35.470 --> 00:07:39.490
and there's nobody
there to add on.
00:07:39.490 --> 00:07:43.250
So the particular
solution is the solution,
00:07:43.250 --> 00:07:45.860
if there is a
particular solution.
00:07:45.860 --> 00:07:49.030
But of course, the
rows might not be -
00:07:49.030 --> 00:07:51.990
are probably not independent
-- and therefore,
00:07:51.990 --> 00:07:56.880
so right-hand sides won't end
up with a zero equal zero after
00:07:56.880 --> 00:08:00.990
elimination, so sometimes
we may have no solution,
00:08:00.990 --> 00:08:02.710
or one solution.
00:08:02.710 --> 00:08:03.380
OK.
00:08:03.380 --> 00:08:10.350
And what I want to say is
that for this matrix A --
00:08:10.350 --> 00:08:13.910
oh, yes, tell me something about
A transpose A in this case.
00:08:13.910 --> 00:08:20.240
So this whole part of the board,
now, is devoted to this case.
00:08:20.240 --> 00:08:23.690
What's the deal
with A transpose A?
00:08:23.690 --> 00:08:27.280
I've emphasized over and over
how important that combination
00:08:27.280 --> 00:08:32.120
is, for a rectangular
matrix, A transpose A
00:08:32.120 --> 00:08:37.120
is the good thing to look
at, and if the rank is n,
00:08:37.120 --> 00:08:39.750
if the null space
has only zero in it,
00:08:39.750 --> 00:08:43.340
then the same is true
of A transpose A.
00:08:43.340 --> 00:08:49.250
That's the beautiful fact, that
if the rank of A is n, well,
00:08:49.250 --> 00:08:52.300
we know this will be an
n by n symmetric matrix,
00:08:52.300 --> 00:08:53.820
and it will be full rank.
00:08:53.820 --> 00:08:55.230
So this is invertible.
00:08:55.230 --> 00:08:58.450
This matrix is invertible.
00:08:58.450 --> 00:08:59.960
That matrix is invertible.
00:08:59.960 --> 00:09:04.070
And now I want to show
you that A itself has
00:09:04.070 --> 00:09:06.880
a one-sided inverse.
00:09:06.880 --> 00:09:08.450
Here it is.
00:09:08.450 --> 00:09:17.190
The inverse of that, which
exists, times A transpose,
00:09:17.190 --> 00:09:20.390
there is a one-sided --
shall I call it A inverse?
00:09:20.390 --> 00:09:24.160
-- left of the matrix A.
00:09:24.160 --> 00:09:28.680
Why do I say that?
00:09:28.680 --> 00:09:35.740
Because if I multiply this
guy by A, what do I get?
00:09:35.740 --> 00:09:37.630
What does that
multiplication give?
00:09:37.630 --> 00:09:40.360
Of course, you
know it instantly,
00:09:40.360 --> 00:09:44.110
because I just put
the parentheses there,
00:09:44.110 --> 00:09:47.000
I have A transpose A
inverse times A transpose A
00:09:47.000 --> 00:09:49.090
so, of course,
it's the identity.
00:09:49.090 --> 00:09:52.120
So it's a left inverse.
00:09:52.120 --> 00:09:58.830
And this was the totally
crucial case for least squares,
00:09:58.830 --> 00:10:02.920
because you remember that least
squares, the central equation
00:10:02.920 --> 00:10:06.230
of least squares had this
matrix, A transpose A,
00:10:06.230 --> 00:10:08.920
as its coefficient matrix.
00:10:08.920 --> 00:10:11.770
And in the case of
full column rank,
00:10:11.770 --> 00:10:15.910
that matrix is
invertible, and we're go.
00:10:15.910 --> 00:10:19.830
So that's the case where
there is a left-inverse.
00:10:19.830 --> 00:10:26.140
So A does whatever it does,
we can find a matrix that
00:10:26.140 --> 00:10:30.530
brings it back to the identity.
00:10:30.530 --> 00:10:33.710
Now, is it true that,
in the other order --
00:10:33.710 --> 00:10:36.830
so A inverse left times
A is the identity.
00:10:42.131 --> 00:10:42.630
Right?
00:10:42.630 --> 00:10:46.100
This matrix is m by n.
00:10:46.100 --> 00:10:49.580
This matrix is n by m.
00:10:49.580 --> 00:10:52.060
The identity matrix is n by n.
00:10:52.060 --> 00:10:53.560
All good.
00:10:53.560 --> 00:10:56.300
All good if you're n.
00:10:56.300 --> 00:11:02.850
But if you try to put that
matrix on the other side,
00:11:02.850 --> 00:11:05.280
it would fail.
00:11:05.280 --> 00:11:12.240
If the full column rank --
if this is smaller than m,
00:11:12.240 --> 00:11:14.480
the case where they're
equals is the beautiful case,
00:11:14.480 --> 00:11:16.270
but that's all set.
00:11:16.270 --> 00:11:18.100
Now, we're looking
at the case where
00:11:18.100 --> 00:11:21.440
the columns are independent
but the rows are not.
00:11:21.440 --> 00:11:25.370
So this is invertible,
but what matrix is not
00:11:25.370 --> 00:11:26.670
invertible?
00:11:26.670 --> 00:11:30.890
A A transpose is
bad for this case.
00:11:30.890 --> 00:11:32.570
A transpose A is good.
00:11:32.570 --> 00:11:35.710
So we can multiply on the
left, everything good,
00:11:35.710 --> 00:11:39.130
we get the left inverse.
00:11:39.130 --> 00:11:42.140
But it would not be
a two-sided inverse.
00:11:42.140 --> 00:11:46.680
A rectangular matrix can't
have a two-sided inverse,
00:11:46.680 --> 00:11:50.900
because there's got to be
some null space, right?
00:11:50.900 --> 00:11:53.840
If I have a matrix
that's rectangular,
00:11:53.840 --> 00:11:58.920
then either that
matrix or its transpose
00:11:58.920 --> 00:12:02.340
has some null space, because
if n and m are different,
00:12:02.340 --> 00:12:06.440
then there's going to be
some free variables around,
00:12:06.440 --> 00:12:09.380
and we'll have some null
space in that direction.
00:12:09.380 --> 00:12:17.070
OK, tell me the corresponding
picture for the opposite case.
00:12:17.070 --> 00:12:20.840
So now I'm going to ask
you about right-inverses.
00:12:20.840 --> 00:12:22.325
A right-inverse.
00:12:26.010 --> 00:12:28.970
And you can fill
this all out, this
00:12:28.970 --> 00:12:31.410
is going to be the
case of full row rank.
00:12:34.420 --> 00:12:41.870
And then r is equal to m, now,
the m rows are independent,
00:12:41.870 --> 00:12:45.270
but the columns are not.
00:12:45.270 --> 00:12:47.000
So what's the deal on that?
00:12:47.000 --> 00:12:50.640
Well, just exactly
the flip of this one.
00:12:50.640 --> 00:12:57.400
The null space of A
transpose contains only zero,
00:12:57.400 --> 00:13:00.480
because there are no
combinations of the rows that
00:13:00.480 --> 00:13:02.390
give the zero row.
00:13:02.390 --> 00:13:03.750
We have independent rows.
00:13:08.860 --> 00:13:13.660
And in a minute, I'll give
an example of all these.
00:13:13.660 --> 00:13:17.545
So, how many solutions
to Ax=b in this case?
00:13:23.159 --> 00:13:24.200
The rows are independent.
00:13:27.050 --> 00:13:30.980
So we can always solve Ax=b.
00:13:30.980 --> 00:13:34.540
Whenever elimination
never produces a zero row,
00:13:34.540 --> 00:13:38.290
so we never get into that
zero equal one problem,
00:13:38.290 --> 00:13:43.470
so Ax=b always has a
solution, but too many.
00:13:43.470 --> 00:13:50.140
So there will be some null
space, the null space of A --
00:13:50.140 --> 00:13:54.460
what will be the dimension
of A's null space?
00:13:54.460 --> 00:13:58.910
How many free
variables have we got?
00:13:58.910 --> 00:14:03.410
How many special solutions in
that null space have we got?
00:14:03.410 --> 00:14:06.090
So how many free
variables in this setup?
00:14:06.090 --> 00:14:12.680
We've got n columns,
so n variables,
00:14:12.680 --> 00:14:16.530
and this tells us
how many are pivot
00:14:16.530 --> 00:14:19.170
variables, that tells us
how many pivots there are,
00:14:19.170 --> 00:14:21.930
so there are n-m free variables.
00:14:21.930 --> 00:14:27.940
So there are infinitely
many solutions to Ax=b.
00:14:27.940 --> 00:14:37.220
We have n-m free
variables in this case.
00:14:37.220 --> 00:14:38.240
OK.
00:14:38.240 --> 00:14:45.600
Now I wanted to ask about
this idea of a right-inverse.
00:14:45.600 --> 00:14:46.720
OK.
00:14:46.720 --> 00:14:52.710
So I'm going to have a matrix
A, my matrix A, and now
00:14:52.710 --> 00:14:54.960
there's going to be some
inverse on the right that
00:14:54.960 --> 00:14:57.710
will give the identity matrix.
00:14:57.710 --> 00:15:05.500
So it will be A times A inverse
on the right, will be I.
00:15:05.500 --> 00:15:12.290
And can you tell me
what, just by comparing
00:15:12.290 --> 00:15:18.560
with what we had up there,
what will be the right-inverse,
00:15:18.560 --> 00:15:21.150
we even have a formula for it.
00:15:21.150 --> 00:15:22.760
There will be other --
00:15:22.760 --> 00:15:24.820
actually, there are
other left-inverses,
00:15:24.820 --> 00:15:26.610
that's our favorite.
00:15:26.610 --> 00:15:28.350
There will be other
right-inverses,
00:15:28.350 --> 00:15:31.620
but tell me our favorite here,
what's the nice right-inverse?
00:15:35.240 --> 00:15:39.470
The nice right-inverse
will be, well, there we
00:15:39.470 --> 00:15:43.820
had A transpose A
was good, now it
00:15:43.820 --> 00:15:46.860
will be A A transpose
that's good.
00:15:46.860 --> 00:15:49.680
The good matrix,
the good right --
00:15:49.680 --> 00:15:53.200
the thing we can invert
is A A transpose,
00:15:53.200 --> 00:16:00.540
so now if I just do it that way,
there sits the right-inverse.
00:16:00.540 --> 00:16:03.930
You see how completely parallel
it is to the one above?
00:16:11.220 --> 00:16:11.964
Right.
00:16:11.964 --> 00:16:13.130
So that's the right-inverse.
00:16:13.130 --> 00:16:20.960
So that's the case
when there is --
00:16:20.960 --> 00:16:25.390
In terms of this
picture, tell me
00:16:25.390 --> 00:16:29.770
what the null spaces are like
so far for these three cases.
00:16:29.770 --> 00:16:32.530
What about case
one, where we had
00:16:32.530 --> 00:16:36.930
a two-sided inverse, full
rank, everything great.
00:16:36.930 --> 00:16:41.290
The null spaces were,
like, gone, right?
00:16:41.290 --> 00:16:44.480
The null spaces were
just the zero vectors.
00:16:44.480 --> 00:16:49.590
Then I took case two,
this null space was gone.
00:16:52.990 --> 00:16:58.890
Case three, this null space was
gone, and then case four is,
00:16:58.890 --> 00:17:04.250
like, the most general case when
this picture is all there --
00:17:04.250 --> 00:17:10.680
when all the null spaces --
this has dimension r, of course,
00:17:10.680 --> 00:17:14.700
this has dimension n-r,
this has dimension r,
00:17:14.700 --> 00:17:26.800
this has dimension m-r, and the
final case will be when r is
00:17:26.800 --> 00:17:29.210
smaller than m and n.
00:17:29.210 --> 00:17:38.940
But can I just,
before I leave here
00:17:38.940 --> 00:17:43.750
look a little more at this one?
00:17:43.750 --> 00:17:46.510
At this case of
full column rank?
00:17:46.510 --> 00:17:52.070
So A inverse on the left,
it has this left-inverse
00:17:52.070 --> 00:17:53.930
to give the identity.
00:17:53.930 --> 00:17:56.540
I said if we multiply
it in the other order,
00:17:56.540 --> 00:17:57.970
we wouldn't get the identity.
00:17:57.970 --> 00:18:02.650
But then I just realized that I
should ask you, what do we get?
00:18:02.650 --> 00:18:05.260
So if I put them in
the other order --
00:18:05.260 --> 00:18:19.010
if I continue this down below,
but I write A times A inverse
00:18:19.010 --> 00:18:21.040
left -- so there's A
times the left-inverse,
00:18:21.040 --> 00:18:23.370
but it's not on
the left any more.
00:18:23.370 --> 00:18:26.970
So it's not going to
come out perfectly.
00:18:26.970 --> 00:18:35.990
But everybody in this room
ought to recognize that matrix,
00:18:35.990 --> 00:18:38.150
right?
00:18:38.150 --> 00:18:42.080
Let's see, is that
the guy we know?
00:18:42.080 --> 00:18:43.195
Am I OK, here?
00:18:51.280 --> 00:18:53.000
What is that matrix?
00:18:53.000 --> 00:18:56.130
P. Thanks.
00:18:56.130 --> 00:18:59.230
P. That matrix --
00:18:59.230 --> 00:19:02.070
it's a projection.
00:19:02.070 --> 00:19:07.750
It's the projection
onto the column space.
00:19:07.750 --> 00:19:12.340
It's trying to be the
identity matrix, right?
00:19:12.340 --> 00:19:17.620
A projection matrix tries
to be the identity matrix,
00:19:17.620 --> 00:19:22.140
but you've given it,
an impossible job.
00:19:22.140 --> 00:19:25.240
So it's the identity
matrix where it can be,
00:19:25.240 --> 00:19:27.750
and elsewhere, it's
the zero matrix.
00:19:27.750 --> 00:19:29.820
So this is P, right.
00:19:29.820 --> 00:19:34.190
A projection onto
the column space.
00:19:34.190 --> 00:19:38.190
And if I asked you this one,
and put these in the opposite
00:19:38.190 --> 00:19:41.340
OK. order -- so this
came from up here.
00:19:41.340 --> 00:19:45.610
And similarly, if I try to put
the right inverse on the left
00:19:45.610 --> 00:19:48.380
--
00:19:48.380 --> 00:19:51.840
so that, like, came from above.
00:19:51.840 --> 00:19:53.810
This, coming from
this side, what
00:19:53.810 --> 00:19:56.620
happens if I try to put the
right inverse on the left?
00:19:56.620 --> 00:20:05.090
Then I would have A transpose
A, A transpose inverse A,
00:20:05.090 --> 00:20:08.350
if this matrix is
now on the left, what
00:20:08.350 --> 00:20:09.880
do you figure that matrix is?
00:20:09.880 --> 00:20:17.390
It's going to be a
projection, too, right?
00:20:17.390 --> 00:20:19.090
It looks very much
like this guy,
00:20:19.090 --> 00:20:22.400
except the only difference
is, A and A transpose
00:20:22.400 --> 00:20:24.100
have been reversed.
00:20:24.100 --> 00:20:27.960
So this is a projection,
this is another projection,
00:20:27.960 --> 00:20:29.710
onto the row space.
00:20:33.520 --> 00:20:35.850
Again, it's trying
to be the identity,
00:20:35.850 --> 00:20:39.980
but there's only so
much the matrix can do.
00:20:39.980 --> 00:20:44.580
And this is the projection
onto the column space.
00:20:44.580 --> 00:20:50.600
So let me now go back
to the main picture
00:20:50.600 --> 00:20:55.600
and tell you about the general
case, the pseudo-inverse.
00:20:55.600 --> 00:20:58.060
These are cases we know.
00:20:58.060 --> 00:21:01.500
So this was important review.
00:21:01.500 --> 00:21:08.480
You've got to know the
business about these ranks,
00:21:08.480 --> 00:21:11.350
and the free variables --
00:21:11.350 --> 00:21:14.960
really, this is linear
algebra coming together.
00:21:14.960 --> 00:21:19.130
And, you know, one nice
thing about teaching 18.06,
00:21:19.130 --> 00:21:23.260
It's not trivial.
00:21:23.260 --> 00:21:25.610
But it's --
00:21:25.610 --> 00:21:28.590
I don't know, somehow, it's
nice when it comes out right.
00:21:28.590 --> 00:21:31.360
I mean -- well, I shouldn't say
anything bad about calculus,
00:21:31.360 --> 00:21:33.050
but I will.
00:21:33.050 --> 00:21:35.150
I mean, like, you
know, you have formulas
00:21:35.150 --> 00:21:40.190
for surface area, and other
awful things and, you know,
00:21:40.190 --> 00:21:46.580
they do their best in
calculus, but it's not elegant.
00:21:46.580 --> 00:21:52.300
And, linear algebra just
is -- well, you know,
00:21:52.300 --> 00:21:54.770
linear algebra is about
the nice part of calculus,
00:21:54.770 --> 00:22:00.880
where everything's, like, flat,
and, the formulas come out
00:22:00.880 --> 00:22:01.830
right.
00:22:01.830 --> 00:22:04.040
And you can go into
high dimensions
00:22:04.040 --> 00:22:06.820
where, in calculus,
you're trying
00:22:06.820 --> 00:22:09.780
to visualize these things,
well, two or three dimensions
00:22:09.780 --> 00:22:10.810
is kind of the limit.
00:22:10.810 --> 00:22:12.430
But here, we don't --
00:22:12.430 --> 00:22:16.280
you know, I've stopped
doing two-by-twos,
00:22:16.280 --> 00:22:18.160
I'm just talking about
the general case.
00:22:18.160 --> 00:22:22.430
OK, now I really will speak
about the general case here.
00:22:22.430 --> 00:22:27.810
What could be the inverse --
00:22:27.810 --> 00:22:29.910
what's a kind of
reasonable inverse
00:22:29.910 --> 00:22:34.070
for a matrix for the
completely general matrix where
00:22:34.070 --> 00:22:38.410
there's a rank r, but
it's smaller than n,
00:22:38.410 --> 00:22:41.680
so there's some null space
left, and it's smaller
00:22:41.680 --> 00:22:44.930
than m, so a transpose
has some null space,
00:22:44.930 --> 00:22:48.430
and it's those null spaces
that are screwing up inverses,
00:22:48.430 --> 00:22:49.650
right?
00:22:49.650 --> 00:22:53.090
Because if a matrix
takes a vector to zero,
00:22:53.090 --> 00:23:01.390
well, there's no way an inverse
can, like, bring it back
00:23:01.390 --> 00:23:03.240
to life.
00:23:03.240 --> 00:23:05.810
My topic is now
the pseudo-inverse,
00:23:05.810 --> 00:23:09.190
and let's just by a
picture, see what's
00:23:09.190 --> 00:23:11.170
the best inverse we could have?
00:23:11.170 --> 00:23:15.820
So, here's a vector
x in the row space.
00:23:15.820 --> 00:23:18.170
I multiply by A.
00:23:18.170 --> 00:23:22.050
Now, the one thing everybody
knows is you take a vector,
00:23:22.050 --> 00:23:25.360
you multiply by A,
and you get an output,
00:23:25.360 --> 00:23:28.220
and where is that output?
00:23:28.220 --> 00:23:31.090
Where is Ax?
00:23:31.090 --> 00:23:35.050
Always in the
column space, right?
00:23:35.050 --> 00:23:37.750
Ax is a combination
of the columns.
00:23:37.750 --> 00:23:39.310
So Ax is somewhere here.
00:23:42.800 --> 00:23:46.750
So I could take all the
vectors in the row space.
00:23:46.750 --> 00:23:49.400
I could multiply them all by A.
00:23:49.400 --> 00:23:53.870
I would get a bunch of
vectors in the column space
00:23:53.870 --> 00:23:59.280
and what I think is, I'd get all
the vectors in the column space
00:23:59.280 --> 00:24:00.670
just right.
00:24:00.670 --> 00:24:03.440
I think that this
connection between an x
00:24:03.440 --> 00:24:07.257
in the row space and an Ax
in the column space, this
00:24:07.257 --> 00:24:07.840
is one-to-one.
00:24:12.160 --> 00:24:14.170
We got a chance, because
they have the same
00:24:14.170 --> 00:24:15.150
dimension.
00:24:15.150 --> 00:24:17.480
That's an r-dimensional
space, and that's
00:24:17.480 --> 00:24:20.060
an r-dimensional space.
00:24:20.060 --> 00:24:22.970
And somehow, the matrix A --
00:24:22.970 --> 00:24:27.260
it's got these null
spaces hanging around,
00:24:27.260 --> 00:24:30.820
where it's knocking vectors to
00:24:30.820 --> 00:24:33.510
And then it's got all the
vectors in between, zero.
00:24:33.510 --> 00:24:35.190
which is almost all vectors.
00:24:35.190 --> 00:24:38.460
Almost all vectors have
a row space component
00:24:38.460 --> 00:24:39.250
and a null space
00:24:39.250 --> 00:24:39.950
component.
00:24:39.950 --> 00:24:42.880
And it's killing the
null space component.
00:24:42.880 --> 00:24:45.690
But if I look at the vectors
that are in the row space,
00:24:45.690 --> 00:24:48.740
with no null space component,
just in the row space,
00:24:48.740 --> 00:24:51.110
then they all go into
the column space,
00:24:51.110 --> 00:24:55.340
so if I put another vector,
let's say, y, in the row space,
00:24:55.340 --> 00:25:02.730
I positive that wherever
Ay is, it won't hit Ax.
00:25:02.730 --> 00:25:04.680
Do you see what I'm saying?
00:25:04.680 --> 00:25:05.830
Let's see why.
00:25:09.340 --> 00:25:09.840
All right.
00:25:09.840 --> 00:25:12.600
So here's what I said.
00:25:12.600 --> 00:25:23.430
If x and y are in the
row space, then A x
00:25:23.430 --> 00:25:27.650
is not the same as A y.
00:25:27.650 --> 00:25:30.110
They're both in the
column space, of course,
00:25:30.110 --> 00:25:31.210
but they're different.
00:25:36.690 --> 00:25:39.780
That would be a perfect
question on a final exam,
00:25:39.780 --> 00:25:45.360
because that's what
I'm teaching you
00:25:45.360 --> 00:25:48.690
in that material
of chapter three
00:25:48.690 --> 00:25:53.920
and chapter four,
especially chapter three.
00:25:53.920 --> 00:25:58.430
If x and y are in the row space,
then Ax is different from Ay.
00:25:58.430 --> 00:26:01.280
So what this means --
00:26:01.280 --> 00:26:03.630
and we'll see why --
00:26:03.630 --> 00:26:09.000
is that, in words, from the
row space to the column space,
00:26:09.000 --> 00:26:12.980
A is perfect, it's
an invertible matrix.
00:26:12.980 --> 00:26:16.700
If we, like, limited
it to those spaces.
00:26:16.700 --> 00:26:19.930
And then, its
inverse will be what
00:26:19.930 --> 00:26:21.290
I'll call the pseudo-inverse.
00:26:21.290 --> 00:26:23.770
So that's that the
pseudo-inverse is.
00:26:23.770 --> 00:26:28.940
It's the inverse -- so A goes
this way, from x to y -- sorry,
00:26:28.940 --> 00:26:35.340
x to A x, from y to A y,
that's A, going that way.
00:26:35.340 --> 00:26:38.910
Then in the other direction,
anything in the column space
00:26:38.910 --> 00:26:41.400
comes from somebody
in the row space,
00:26:41.400 --> 00:26:45.010
and the reverse there is what
I'll call the pseudo-inverse,
00:26:45.010 --> 00:26:51.010
and the accepted
notation is A plus.
00:26:51.010 --> 00:26:55.390
So y will be A plus x.
00:26:55.390 --> 00:26:55.890
I'm sorry.
00:26:55.890 --> 00:27:05.900
No, y will be A plus times
whatever it started with, A y.
00:27:05.900 --> 00:27:09.030
Do you see my picture there?
00:27:09.030 --> 00:27:11.340
Same, of course, for x and A x.
00:27:11.340 --> 00:27:15.280
This way, A does it, the other
way is the pseudo-inverse,
00:27:15.280 --> 00:27:18.430
and the pseudo-inverse
just kills this stuff,
00:27:18.430 --> 00:27:20.360
and the matrix just kills this
00:27:20.360 --> 00:27:20.860
stuff.
00:27:20.860 --> 00:27:25.260
So everything that's really
serious here is going
00:27:25.260 --> 00:27:27.840
on in the row space and
the column space, and now,
00:27:27.840 --> 00:27:31.550
tell me --
00:27:31.550 --> 00:27:34.910
this is the fundamental
fact, that between those two
00:27:34.910 --> 00:27:37.560
r-dimensional spaces,
our matrix is perfect.
00:27:40.900 --> 00:27:41.400
Why?
00:27:44.960 --> 00:27:47.180
Suppose they weren't.
00:27:47.180 --> 00:27:49.230
Why do I get into trouble?
00:27:49.230 --> 00:27:51.780
Suppose -- so, proof.
00:27:51.780 --> 00:27:54.090
I haven't written
down proof very much,
00:27:54.090 --> 00:27:57.640
but I'm going to
use that word once.
00:27:57.640 --> 00:28:00.710
Suppose they were the same.
00:28:00.710 --> 00:28:07.990
Suppose these are supposed
to be two different vectors.
00:28:07.990 --> 00:28:10.540
Maybe I'd better make
the statement correctly.
00:28:10.540 --> 00:28:13.930
If x and y are different
vectors in the row space --
00:28:13.930 --> 00:28:20.790
maybe I'll better put if
x is different from y,
00:28:20.790 --> 00:28:23.370
both in the row space --
00:28:23.370 --> 00:28:25.620
so I'm starting with two
different vectors in the row
00:28:25.620 --> 00:28:29.900
space, I'm multiplying by A --
so these guys are in the column
00:28:29.900 --> 00:28:33.520
space, everybody knows
that, and the point is,
00:28:33.520 --> 00:28:36.980
they're different over there.
00:28:36.980 --> 00:28:38.950
So, suppose they weren't.
00:28:38.950 --> 00:28:40.570
Suppose A x=A y.
00:28:44.940 --> 00:28:48.800
Suppose, well, that's the
same as saying A(x-y) is zero.
00:28:54.960 --> 00:28:57.140
So what?
00:28:57.140 --> 00:29:00.090
So, what do I know
now about (x-y),
00:29:00.090 --> 00:29:03.500
what do I know
about this vector?
00:29:03.500 --> 00:29:07.810
Well, I can see right
away, what space is it in?
00:29:07.810 --> 00:29:10.390
It's sitting in the
null space, right?
00:29:10.390 --> 00:29:11.580
So it's in the null space.
00:29:14.520 --> 00:29:17.030
But what else do
I know about it?
00:29:17.030 --> 00:29:20.950
Here it was x in the row
space, y in the row space,
00:29:20.950 --> 00:29:23.170
what about x-y?
00:29:23.170 --> 00:29:29.870
It's also in the
row space, right?
00:29:29.870 --> 00:29:32.030
Heck, that thing
is a vector space,
00:29:32.030 --> 00:29:35.550
and if the vector space
is anything at all,
00:29:35.550 --> 00:29:38.610
if x is in the row space,
and y is in the row space,
00:29:38.610 --> 00:29:43.040
then the difference is also,
so it's also in the row space.
00:29:47.990 --> 00:29:48.850
So what?
00:29:48.850 --> 00:29:52.420
Now I've got a vector x-y
that's in the null space,
00:29:52.420 --> 00:29:56.270
and that's also in the row
space, so what vector is it?
00:29:56.270 --> 00:29:58.380
It's the zero vector.
00:29:58.380 --> 00:30:00.840
So I would conclude
from that that x-y
00:30:00.840 --> 00:30:07.200
had to be the zero vector,
x-y, so, in other words,
00:30:07.200 --> 00:30:09.120
if I start from two
different vectors,
00:30:09.120 --> 00:30:11.480
I get two different vectors.
00:30:11.480 --> 00:30:14.020
If these vectors are the
same, then those vectors
00:30:14.020 --> 00:30:16.050
had to be the same.
00:30:16.050 --> 00:30:21.600
That's like the algebra proof,
which we understand completely
00:30:21.600 --> 00:30:28.230
because we really understand
these subspaces of what
00:30:28.230 --> 00:30:32.480
I said in words, that
a matrix A is really
00:30:32.480 --> 00:30:37.560
a nice, invertible mapping
from row space to columns pace.
00:30:37.560 --> 00:30:40.160
If the null spaces
keep out of the way,
00:30:40.160 --> 00:30:43.200
then we have an inverse.
00:30:43.200 --> 00:30:46.840
And that inverse is
called the pseudo inverse,
00:30:46.840 --> 00:30:51.410
and it's a very, very,
useful in application.
00:30:51.410 --> 00:30:54.360
Statisticians
discovered, oh boy, this
00:30:54.360 --> 00:30:56.580
is the thing that we
needed all our lives,
00:30:56.580 --> 00:30:59.100
and here it finally showed
up, the pseudo-inverse
00:30:59.100 --> 00:31:02.070
is the right thing.
00:31:02.070 --> 00:31:04.300
Why do statisticians need it?
00:31:04.300 --> 00:31:11.360
And because statisticians
are like least-squares-happy.
00:31:11.360 --> 00:31:14.440
I mean they're always
doing least squares.
00:31:14.440 --> 00:31:20.700
And so this is their
central linear regression.
00:31:20.700 --> 00:31:22.920
Statisticians who may
watch this on video,
00:31:22.920 --> 00:31:28.940
please forgive that
description of your interests.
00:31:28.940 --> 00:31:35.170
One of your interests is linear
regression and this problem.
00:31:35.170 --> 00:31:41.150
But this problem is only OK
provided we have full column
00:31:41.150 --> 00:31:42.080
rank.
00:31:42.080 --> 00:31:46.810
And statisticians have to worry
all the time about, oh, God,
00:31:46.810 --> 00:31:49.900
maybe we just repeated
an experiment.
00:31:49.900 --> 00:31:52.220
You know, you're taking
all these measurements,
00:31:52.220 --> 00:31:54.750
maybe you just repeat
them a few times.
00:31:54.750 --> 00:31:56.730
You know, maybe they're
not independent.
00:31:56.730 --> 00:32:00.910
Well, in that case, that
A transpose A matrix
00:32:00.910 --> 00:32:04.230
that they depend on
becomes singular.
00:32:04.230 --> 00:32:06.890
So then that's when they
needed the pseudo-inverse,
00:32:06.890 --> 00:32:09.180
it just arrived at
the right moment,
00:32:09.180 --> 00:32:13.840
and it's the right quantity.
00:32:13.840 --> 00:32:14.440
OK.
00:32:14.440 --> 00:32:21.590
So now that you know what the
pseudo-inverse should do, let
00:32:21.590 --> 00:32:25.080
me see what it is.
00:32:25.080 --> 00:32:27.010
Can we find it?
00:32:27.010 --> 00:32:30.170
So this is my -- to
complete the lecture is --
00:32:30.170 --> 00:32:42.740
how do I find this
pseudo-inverse A plus?
00:32:42.740 --> 00:32:45.310
OK.
00:32:45.310 --> 00:32:46.340
OK.
00:32:46.340 --> 00:32:48.170
Well, here's one way.
00:32:50.710 --> 00:32:53.860
Everything I do today is
to try to review stuff.
00:32:53.860 --> 00:33:00.770
One way would be to
start from the SVD.
00:33:00.770 --> 00:33:02.630
The Singular Value
Decomposition.
00:33:02.630 --> 00:33:05.380
And you remember
that that factored A
00:33:05.380 --> 00:33:10.480
into an orthogonal matrix
times this diagonal matrix
00:33:10.480 --> 00:33:12.500
times this orthogonal matrix.
00:33:12.500 --> 00:33:16.330
But what did that
diagonal guy look like?
00:33:16.330 --> 00:33:25.510
This diagonal guy, sigma,
has some non-zeroes,
00:33:25.510 --> 00:33:28.280
and you remember, they
came from A transpose A,
00:33:28.280 --> 00:33:31.400
and A A transpose, these
are the good guys, and then
00:33:31.400 --> 00:33:34.680
some more zeroes, and all zeroes
there, and all zeroes there.
00:33:38.110 --> 00:33:41.700
So you can guess what
the pseudo-inverse is,
00:33:41.700 --> 00:33:45.030
I just invert stuff that's
nice to invert -- well,
00:33:45.030 --> 00:33:47.280
what's the
pseudo-inverse of this?
00:33:47.280 --> 00:33:50.610
That's what the
problem comes down to.
00:33:50.610 --> 00:33:55.310
What's the pseudo-inverse of
this beautiful diagonal matrix?
00:33:55.310 --> 00:33:58.380
But it's got a
null space, right?
00:33:58.380 --> 00:34:01.480
What's the rank of this matrix?
00:34:01.480 --> 00:34:06.420
What's the rank of
this diagonal matrix?
00:34:06.420 --> 00:34:07.430
r, of course.
00:34:07.430 --> 00:34:09.790
It's got r non-zeroes,
and then it's otherwise,
00:34:09.790 --> 00:34:10.780
zip.
00:34:10.780 --> 00:34:21.000
So it's got n columns, it's got
m rows, and it's got rank r.
00:34:21.000 --> 00:34:23.989
It's the best example, the
simplest example we could ever
00:34:23.989 --> 00:34:28.089
have of our general setup.
00:34:31.179 --> 00:34:31.699
OK?
00:34:31.699 --> 00:34:36.800
So what's the pseudo-inverse?
00:34:36.800 --> 00:34:39.020
What's the matrix --
00:34:39.020 --> 00:34:41.296
so I'll erase our columns,
because right below it,
00:34:41.296 --> 00:34:42.754
I want to write
the pseudo-inverse.
00:34:46.199 --> 00:34:49.179
OK, you can make a
pretty darn good guess.
00:34:49.179 --> 00:34:53.170
If it was a proper diagonal
matrix, invertible,
00:34:53.170 --> 00:34:57.000
if there weren't any zeroes
down here, if it was sigma one
00:34:57.000 --> 00:35:01.010
to sigma n, then everybody
knows what the inverse would be,
00:35:01.010 --> 00:35:08.760
the inverse would be one over
sigma one, down to one over s-
00:35:08.760 --> 00:35:13.340
but of course, I'll
have to stop at sigma r.
00:35:13.340 --> 00:35:17.350
And, it will be the rest,
zeroes again, of course.
00:35:17.350 --> 00:35:22.620
And now this one was
m by n, and this one
00:35:22.620 --> 00:35:25.340
is meant to have a slightly
different, you know,
00:35:25.340 --> 00:35:29.230
transpose shape, n by m.
00:35:29.230 --> 00:35:31.740
They both have that rank r.
00:35:39.820 --> 00:35:45.350
My idea is that the
pseudo-inverse is the best --
00:35:45.350 --> 00:35:48.200
is the closest I can
come to an inverse.
00:35:48.200 --> 00:35:53.220
So what is sigma times
its pseudo-inverse?
00:35:53.220 --> 00:35:56.020
Can you multiply sigma
by its pseudo-inverse?
00:35:56.020 --> 00:35:57.920
Multiply that by that?
00:35:57.920 --> 00:35:59.130
What matrix do you get?
00:36:05.040 --> 00:36:07.280
They're diagonal.
00:36:07.280 --> 00:36:10.200
Rectangular, of course.
00:36:10.200 --> 00:36:18.780
But of course, we're
going to get ones, R ones,
00:36:18.780 --> 00:36:20.590
and all the rest, zeroes.
00:36:20.590 --> 00:36:24.420
And the shape of that, this
whole matrix will be m by
00:36:24.420 --> 00:36:25.780
m.
00:36:25.780 --> 00:36:30.200
And suppose I did it
in the other order.
00:36:30.200 --> 00:36:32.067
Suppose I did sigma plus sigma.
00:36:32.067 --> 00:36:33.525
Why don't I do it
right underneath?
00:36:38.190 --> 00:36:40.500
in the opposite order?
00:36:40.500 --> 00:36:42.640
See, this matrix hasn't
got a left-inverse,
00:36:42.640 --> 00:36:45.170
it hasn't got a right-inverse,
but every matrix
00:36:45.170 --> 00:36:46.590
has got a pseudo-inverse.
00:36:46.590 --> 00:36:51.300
If I do it in the order sigma
plus sigma, what do I get?
00:36:51.300 --> 00:36:55.920
Square matrix, this is
m by n, this is m by m,
00:36:55.920 --> 00:37:01.590
my result is going to m by
m -- is going to be n by n,
00:37:01.590 --> 00:37:03.030
and what is it?
00:37:05.930 --> 00:37:08.260
Those are diagonal
matrices, it's
00:37:08.260 --> 00:37:10.600
going to be ones,
and then zeroes.
00:37:14.670 --> 00:37:19.020
It's not the same as that,
it's a different size --
00:37:19.020 --> 00:37:22.090
it's a projection.
00:37:22.090 --> 00:37:25.560
One is a projection matrix
onto the column space,
00:37:25.560 --> 00:37:30.590
and this one is the projection
matrix onto the row space.
00:37:30.590 --> 00:37:35.470
That's the best that
pseudo-inverse can do.
00:37:35.470 --> 00:37:38.030
So what the
pseudo-inverse does is,
00:37:38.030 --> 00:37:41.050
if you multiply on the left,
you don't get the identity,
00:37:41.050 --> 00:37:42.510
if you multiply
on the right, you
00:37:42.510 --> 00:37:46.080
don't get the identity, what
you get is the projection.
00:37:46.080 --> 00:37:51.300
It brings you into the two
good spaces, the row space
00:37:51.300 --> 00:37:52.760
and column space.
00:37:52.760 --> 00:37:55.240
And it just wipes
out the null space.
00:37:55.240 --> 00:37:57.800
So that's what the
pseudo-inverse of this diagonal
00:37:57.800 --> 00:38:04.710
one is, and then the
pseudo-inverse of A itself --
00:38:04.710 --> 00:38:06.290
this is perfectly invertible.
00:38:06.290 --> 00:38:09.180
What's the inverse
of V transpose?
00:38:09.180 --> 00:38:11.800
Just another tiny bit of review.
00:38:11.800 --> 00:38:18.590
That's an orthogonal matrix,
and its inverse is V, good.
00:38:18.590 --> 00:38:21.360
This guy has got all
the trouble in it,
00:38:21.360 --> 00:38:25.940
all the null space
is responsible for,
00:38:25.940 --> 00:38:28.610
so it doesn't have
a true inverse,
00:38:28.610 --> 00:38:32.010
it has a pseudo-inverse,
and then the inverse of U
00:38:32.010 --> 00:38:37.320
is U transpose, thanks.
00:38:37.320 --> 00:38:40.200
Or, of course, I
could write U inverse.
00:38:40.200 --> 00:38:43.640
So, that's the question of, how
do you find the pseudo-inverse
00:38:43.640 --> 00:38:45.110
--
00:38:45.110 --> 00:38:49.350
so what statisticians do
when they're in this --
00:38:49.350 --> 00:38:54.980
so this is like the case of
where least squares breaks down
00:38:54.980 --> 00:38:58.420
because the rank is --
you don't have full rank,
00:38:58.420 --> 00:39:05.020
and the beauty of the singular
value decomposition is,
00:39:05.020 --> 00:39:09.620
it puts all the problems into
this diagonal matrix where
00:39:09.620 --> 00:39:10.980
it's clear what to do.
00:39:10.980 --> 00:39:13.960
It's the best inverse you
could think of is clear.
00:39:13.960 --> 00:39:16.430
You see there could be other --
00:39:16.430 --> 00:39:18.720
I mean, we could put
some stuff down here,
00:39:18.720 --> 00:39:20.980
it would multiply these zeroes.
00:39:20.980 --> 00:39:27.270
It wouldn't have any effect,
but then the good pseudo-inverse
00:39:27.270 --> 00:39:33.290
is the one with no extra
stuff, it's sort of, like,
00:39:33.290 --> 00:39:36.430
as small as possible.
00:39:36.430 --> 00:39:41.040
It has to have those
to produce the ones.
00:39:41.040 --> 00:39:46.260
If it had other stuff, it
would just be a larger matrix,
00:39:46.260 --> 00:39:51.050
so this pseudo-inverse is kind
of the minimal matrix that
00:39:51.050 --> 00:39:53.960
gives the best result.
00:39:53.960 --> 00:39:57.250
Sigma sigma plus being r ones.
00:39:57.250 --> 00:39:59.710
SK.
00:39:59.710 --> 00:40:03.122
so I guess I'm hoping --
00:40:03.122 --> 00:40:04.830
pseudo-inverse, again,
let me repeat what
00:40:04.830 --> 00:40:06.038
I said at the very beginning.
00:40:09.080 --> 00:40:11.990
This pseudo-inverse,
which appears
00:40:11.990 --> 00:40:17.700
at the end, which is in
section seven point four,
00:40:17.700 --> 00:40:24.090
and probably I did more with
it here than I did in the book.
00:40:24.090 --> 00:40:28.460
The word pseudo-inverse will
not appear on an exam in this
00:40:28.460 --> 00:40:34.900
course, but I think if you
see this all will appear,
00:40:34.900 --> 00:40:40.010
because this is all what the
course was about, chapters one,
00:40:40.010 --> 00:40:41.527
two, three, four --
00:40:44.150 --> 00:40:47.900
but if you see all that,
then you probably see,
00:40:47.900 --> 00:40:51.810
well, OK, the general case
had both null spaces around,
00:40:51.810 --> 00:40:56.470
and this is the
natural thing to do.
00:40:56.470 --> 00:41:01.190
So, this is one way to
find the pseudo-inverse.
00:41:01.190 --> 00:41:03.500
Yes.
00:41:03.500 --> 00:41:07.260
The point of a pseudo-inverse,
of computing a pseudo-inverse
00:41:07.260 --> 00:41:10.220
is to get some factors
where you can find
00:41:10.220 --> 00:41:12.000
the pseudo-inverse quickly.
00:41:12.000 --> 00:41:15.720
And this is, like, the
champion, because this
00:41:15.720 --> 00:41:20.850
is where we can invert
those, and those two, easily,
00:41:20.850 --> 00:41:26.560
just by transposing, and we
know what to do with a diagonal.
00:41:26.560 --> 00:41:33.210
OK, that's as much
review, maybe --
00:41:33.210 --> 00:41:37.920
let's have a five-minute holiday
in 18.06 and, I'll see you
00:41:37.920 --> 00:41:40.060
Wednesday, then,
for the rest of this
00:41:40.060 --> 00:41:40.560
course.
00:41:40.560 --> 00:41:42.110
Thanks.