WEBVTT

00:00:00.000 --> 00:00:01.964
[SQUEAKING]

00:00:01.964 --> 00:00:03.928
[RUSTLING]

00:00:03.928 --> 00:00:05.892
[CLICKING]

00:00:12.770 --> 00:00:15.350
GILBERT STRANG: OK,
this is about finding

00:00:15.350 --> 00:00:17.480
the null space of a matrix A--

00:00:17.480 --> 00:00:20.630
any matrix, square
or rectangular.

00:00:20.630 --> 00:00:22.560
And what does that mean?

00:00:22.560 --> 00:00:25.700
That means, well,
in algebra, we're

00:00:25.700 --> 00:00:29.210
solving the equation
Ax equals 0.

00:00:29.210 --> 00:00:33.830
So A is our matrix, x is a
vector that we're looking for,

00:00:33.830 --> 00:00:38.780
and Ax is a combination
of the columns of A.

00:00:38.780 --> 00:00:40.880
So we're looking
for combinations

00:00:40.880 --> 00:00:44.090
of the columns that
give the zero vector--

00:00:44.090 --> 00:00:46.520
dependent columns,
we'll be saying.

00:00:46.520 --> 00:00:49.500
So that's the goal.

00:00:49.500 --> 00:00:55.730
And the new start for linear
algebra that I've suggested

00:00:55.730 --> 00:00:57.710
solves that problem
for small matrices.

00:00:57.710 --> 00:01:01.370
It kind of just does
it for a small matrix.

00:01:01.370 --> 00:01:05.580
But for any matrix, or a
big one, we need a system.

00:01:05.580 --> 00:01:08.270
So this kind of
completes the idea

00:01:08.270 --> 00:01:15.530
by giving the system,
which uses the algorithm

00:01:15.530 --> 00:01:18.260
of linear algebra,
which is elimination.

00:01:18.260 --> 00:01:20.840
So elimination is
going to be the key

00:01:20.840 --> 00:01:23.990
to solving this problem--

00:01:23.990 --> 00:01:25.800
to finding the null space.

00:01:25.800 --> 00:01:27.980
So that's my picture.

00:01:27.980 --> 00:01:29.390
Oh, these are the--

00:01:29.390 --> 00:01:36.410
I'll just mention-- the three
books which discuss this.

00:01:36.410 --> 00:01:40.190
The Introduction to Linear
Algebra is the main textbook.

00:01:40.190 --> 00:01:44.150
Then the Learning
from Data book--

00:01:44.150 --> 00:01:47.330
actually, that's where
this new idea got started.

00:01:47.330 --> 00:01:49.670
And the Linear
Algebra for Everyone

00:01:49.670 --> 00:01:53.370
has got the idea
more completely.

00:01:53.370 --> 00:01:59.450
So I'm sort of speaking about a
section out of that third book.

00:01:59.450 --> 00:02:00.200
OK.

00:02:00.200 --> 00:02:02.900
And they all have websites.

00:02:02.900 --> 00:02:11.720
Math.mit.edu, the addresses
are there to see more.

00:02:11.720 --> 00:02:13.250
Actually, I should
say, so these are

00:02:13.250 --> 00:02:15.020
the key ideas of this lecture.

00:02:15.020 --> 00:02:20.030
And yesterday, I looked at
my lecture from a few years

00:02:20.030 --> 00:02:22.910
ago-- quite a few years
ago-- lecture seven

00:02:22.910 --> 00:02:29.270
in the OpenCourseWare
series for Math 18.06.

00:02:29.270 --> 00:02:33.830
So that was on the same topic,
elimination and Ax equals 0.

00:02:33.830 --> 00:02:37.670
So a lot of good stuff is
there, but there's a little more

00:02:37.670 --> 00:02:38.690
to say now.

00:02:38.690 --> 00:02:41.700
And that's what
this is about today.

00:02:41.700 --> 00:02:45.450
So OK, here are key
ideas of the lecture.

00:02:45.450 --> 00:02:46.740
So the null space--

00:02:46.740 --> 00:02:50.180
so now you see it in writing--
the null space is all

00:02:50.180 --> 00:02:53.030
the solutions x to Ax equals 0.

00:02:53.030 --> 00:02:55.010
Remember, x is a vector.

00:02:55.010 --> 00:03:00.170
And elimination is the key and
it keeps the same null space.

00:03:00.170 --> 00:03:05.556
And the Matlab
command is rref of A.

00:03:05.556 --> 00:03:09.650
That's the command
that does elimination.

00:03:09.650 --> 00:03:18.050
And we'll see the identity
matrix, we'll see a matrix F,

00:03:18.050 --> 00:03:23.630
and the new idea was to factor
the matrix into a column matrix

00:03:23.630 --> 00:03:27.050
C times a row matrix
R. So this is really

00:03:27.050 --> 00:03:29.120
putting all the ideas together.

00:03:29.120 --> 00:03:33.980
And we learn that if a
matrix is short and wide,

00:03:33.980 --> 00:03:36.110
then there are certainly--

00:03:36.110 --> 00:03:38.120
if we've got lots of
columns, then some

00:03:38.120 --> 00:03:42.200
of them will be dependent
and that means that there'll

00:03:42.200 --> 00:03:44.690
be solutions to Ax equals 0.

00:03:44.690 --> 00:03:47.420
So here we go.

00:03:47.420 --> 00:03:50.680
Oh OK, I'm going to
start with an example.

00:03:50.680 --> 00:03:55.430
And elimination has
already done its job.

00:03:55.430 --> 00:03:57.520
So what elimination
did was to get

00:03:57.520 --> 00:04:00.610
to that-- you see that identity
matrix in the first two

00:04:00.610 --> 00:04:01.300
columns?

00:04:01.300 --> 00:04:03.310
1, 0 and 0, 1.

00:04:03.310 --> 00:04:07.090
So you can't ask for a
simpler matrix than that.

00:04:07.090 --> 00:04:12.310
So once we've got that, we can't
mess with it, so 3, 5, 4, 6,

00:04:12.310 --> 00:04:14.140
we're stuck with--

00:04:14.140 --> 00:04:16.100
those are our other columns.

00:04:16.100 --> 00:04:18.970
So the first two
columns are independent.

00:04:18.970 --> 00:04:23.380
Columns of the identity
matrix are very independent.

00:04:23.380 --> 00:04:29.130
But then 3, 4 is a combination
of those two columns, right?

00:04:29.130 --> 00:04:33.760
3, 4 is 3 times the first
column plus 4 times the second.

00:04:33.760 --> 00:04:38.227
So there is an x
in the null space.

00:04:38.227 --> 00:04:39.810
That's one of the
vectors we're after.

00:04:39.810 --> 00:04:42.930
I'll call it a special
solution because it just

00:04:42.930 --> 00:04:45.030
came specially from one column.

00:04:45.030 --> 00:04:48.330
And you see it down below as s1.

00:04:48.330 --> 00:04:53.220
So if we took minus 3 of
the first column, minus 4

00:04:53.220 --> 00:04:57.060
of the second column, and then
plus 1 of the third column,

00:04:57.060 --> 00:04:58.875
we would have 0.

00:04:58.875 --> 00:05:01.600
Ax equals 0-- what
we're looking for.

00:05:01.600 --> 00:05:04.910
And the second
special solution would

00:05:04.910 --> 00:05:08.870
come from the last column
of the matrix, 5, 6.

00:05:08.870 --> 00:05:12.710
So again, that's 5 of
the 1, 0 column and 6

00:05:12.710 --> 00:05:14.550
of the 0, 1 column.

00:05:14.550 --> 00:05:20.540
And if we put that together
into a special solution s2,

00:05:20.540 --> 00:05:25.280
we want minus 5 of column
1, minus 6 of column 2,

00:05:25.280 --> 00:05:30.440
plus nothing of column 3,
plus column 4, giving 0.

00:05:30.440 --> 00:05:37.280
So in this case, elimination
has produced a simple matrix R--

00:05:37.280 --> 00:05:40.140
simple because it's
got the identity there.

00:05:40.140 --> 00:05:44.510
So can I show you
one more example of R

00:05:44.510 --> 00:05:47.990
before I begin to talk
about how do we get to R?

00:05:47.990 --> 00:05:53.000
OK, so here's another R. A
little different, though.

00:05:53.000 --> 00:05:56.990
It's different because
it's got a row of 0's.

00:05:56.990 --> 00:05:59.180
Well that won't
pose any problem,

00:05:59.180 --> 00:06:02.040
but we just have to
think what to do with it.

00:06:02.040 --> 00:06:04.880
We always move 0
rows to the bottom.

00:06:04.880 --> 00:06:06.320
And there it is.

00:06:06.320 --> 00:06:09.590
And it does have
an identity matrix

00:06:09.590 --> 00:06:16.370
as the first example did, but
you see that the 0, 1 is often

00:06:16.370 --> 00:06:18.230
column 3.

00:06:18.230 --> 00:06:21.440
The identity is in
columns 1 and 3 here

00:06:21.440 --> 00:06:25.190
and that makes a small change.

00:06:25.190 --> 00:06:27.530
But the idea is still the same.

00:06:27.530 --> 00:06:30.320
Those two columns are
the independent ones.

00:06:30.320 --> 00:06:32.390
They're very simple.

00:06:32.390 --> 00:06:37.100
The 1, 0, 0 and 0, 1, 0,
those are totally different

00:06:37.100 --> 00:06:38.330
directions.

00:06:38.330 --> 00:06:44.530
Then the 7, 0, 0 is 7
of the first column.

00:06:44.530 --> 00:06:51.720
So that spotted a special
solution-- an x that I call s1.

00:06:51.720 --> 00:06:54.480
At the bottom line,
you see the minus 7

00:06:54.480 --> 00:06:59.220
of the first column plus 1 of
the second column produces 0.

00:06:59.220 --> 00:07:01.950
If you just look at
those columns, minus 7

00:07:01.950 --> 00:07:05.040
of the first plus 1 of the
second, everything cancels.

00:07:05.040 --> 00:07:10.800
And the other one is going to
come from the 8, 9, 0 column.

00:07:10.800 --> 00:07:14.400
That'll be 8 of the first
column and 9 of the third--

00:07:14.400 --> 00:07:16.470
the two bits of the identity.

00:07:16.470 --> 00:07:25.920
So you see that when we get to
this reduced row echelon form--

00:07:25.920 --> 00:07:28.650
horrible set of words,
but what it means

00:07:28.650 --> 00:07:33.780
is elimination has made
it as simple as it can be.

00:07:33.780 --> 00:07:36.480
OK, I might have
something more to say.

00:07:36.480 --> 00:07:40.090
Yeah, so this is summarizing
what you've just seen.

00:07:40.090 --> 00:07:42.930
So we have the simplest
case where the identity just

00:07:42.930 --> 00:07:47.010
sits there at the left
or the more general case

00:07:47.010 --> 00:07:50.520
where the identity is mixed
in with the other two columns.

00:07:50.520 --> 00:07:52.170
We can live with both.

00:07:52.170 --> 00:07:55.980
And I'm using F for
the other columns--

00:07:55.980 --> 00:07:58.510
the columns that are not
part of the identity.

00:07:58.510 --> 00:08:01.650
And then it has
that extra 0 row.

00:08:01.650 --> 00:08:02.310
OK.

00:08:02.310 --> 00:08:06.330
And now, I want to write it--

00:08:06.330 --> 00:08:15.660
a key part of this lecture is to
see the result this matrix R--

00:08:15.660 --> 00:08:20.010
see it in matrix form instead
of a bunch of numbers.

00:08:20.010 --> 00:08:24.870
Often, in the computing, you're
just pages full of numbers

00:08:24.870 --> 00:08:26.650
and you don't see
what's happening.

00:08:26.650 --> 00:08:31.980
So what we're looking for
is the identity matrix

00:08:31.980 --> 00:08:36.450
and the non-zero matrix F.

00:08:36.450 --> 00:08:40.650
And now, in that R0 case,
that second example,

00:08:40.650 --> 00:08:44.039
there's a P. What is that
matrix P doing there?

00:08:44.039 --> 00:08:47.340
Well, it's a
permutation matrix--

00:08:47.340 --> 00:08:49.140
an exchange matrix.

00:08:49.140 --> 00:08:53.490
Because the identity
in this second example

00:08:53.490 --> 00:08:55.380
is not in columns 1 and 2.

00:08:55.380 --> 00:08:56.640
It's in 1 and 3.

00:08:56.640 --> 00:08:59.070
So that P has to put it there.

00:08:59.070 --> 00:09:04.990
So that P is-- and
because P is on the right,

00:09:04.990 --> 00:09:06.480
it'll move columns.

00:09:06.480 --> 00:09:10.440
And there I wrote
down what that P is.

00:09:10.440 --> 00:09:15.360
That exchanges column 2
and 3 and puts the identity

00:09:15.360 --> 00:09:18.150
in where we want it.

00:09:18.150 --> 00:09:18.720
OK.

00:09:18.720 --> 00:09:23.880
So that's the linear algebra--
the matrix notation part.

00:09:23.880 --> 00:09:31.420
Oh, and this is-- here, now
you're seeing the new start,

00:09:31.420 --> 00:09:39.790
this C times R. So that's what
I'm suggesting that if you give

00:09:39.790 --> 00:09:43.360
me a matrix, the first thing
I would do would be to find

00:09:43.360 --> 00:09:44.710
independent columns--

00:09:44.710 --> 00:09:48.940
those would go into C. And
then the matrix R would--

00:09:48.940 --> 00:09:52.240
well, we've seen the matrix
R and that would tell me

00:09:52.240 --> 00:09:55.390
the combinations
of those C columns

00:09:55.390 --> 00:09:57.790
to get A. Yeah, you'll see it.

00:09:57.790 --> 00:10:03.460
Well, this box is around
all the matrix formulas.

00:10:03.460 --> 00:10:08.170
So we're sorting every matrix
into independent columns

00:10:08.170 --> 00:10:12.190
followed by dependent columns
and then this permutation--

00:10:12.190 --> 00:10:14.230
this exchange matrix P--

00:10:14.230 --> 00:10:17.090
if they don't really
come in that order.

00:10:17.090 --> 00:10:18.530
If we need a new order.

00:10:18.530 --> 00:10:23.770
So all this is about
A equals CR, the new--

00:10:23.770 --> 00:10:24.790
well, not new.

00:10:24.790 --> 00:10:31.420
Not completely new, I'm sure,
but it's the factorization

00:10:31.420 --> 00:10:33.580
of any matrix.

00:10:33.580 --> 00:10:34.080
OK.

00:10:34.080 --> 00:10:36.990
And now how do we actually
do it if the matrix is big

00:10:36.990 --> 00:10:40.920
and we need the
computer to help?

00:10:40.920 --> 00:10:44.050
Well, we do it by elimination.

00:10:44.050 --> 00:10:48.410
OK, this is what
elimination is about.

00:10:48.410 --> 00:10:52.160
What are we allowed
to do in elimination?

00:10:52.160 --> 00:10:55.730
We don't want to mess up
the equation Ax equals 0.

00:10:55.730 --> 00:11:00.200
I'm operating on A, but
I'm not creating or losing

00:11:00.200 --> 00:11:02.130
any of those solutions.

00:11:02.130 --> 00:11:09.140
So if I subtract one
row from another row,

00:11:09.140 --> 00:11:11.960
or any multiple of one
row from another row,

00:11:11.960 --> 00:11:14.120
that's the main
operation of elimination.

00:11:14.120 --> 00:11:17.330
And obviously, the
equations are still true.

00:11:17.330 --> 00:11:22.010
If I take 3 of equation
one away from equation two,

00:11:22.010 --> 00:11:24.650
I still have a correct equation.

00:11:24.650 --> 00:11:29.750
And I could multiply a row
by 15 or any non-zero number.

00:11:29.750 --> 00:11:30.770
No change.

00:11:30.770 --> 00:11:33.560
And I can switch any two rows.

00:11:33.560 --> 00:11:37.460
That third one of switching
rows, exchanging rows,

00:11:37.460 --> 00:11:41.160
is just to move the 0
row down to the bottom.

00:11:41.160 --> 00:11:44.180
This is the example.

00:11:44.180 --> 00:11:45.330
Can you see those numbers?

00:11:45.330 --> 00:11:49.340
So I'm giving an example of a
matrix that leads to that very

00:11:49.340 --> 00:11:52.610
same R that we worked out.

00:11:52.610 --> 00:11:54.500
So you see what I'm doing now.

00:11:54.500 --> 00:11:55.910
I'm backing up.

00:11:55.910 --> 00:12:01.180
I know the R I want, but
I'm given some matrix A,

00:12:01.180 --> 00:12:06.360
and it's these elimination steps
that will lead me to R. OK.

00:12:06.360 --> 00:12:09.540
So you see A over on the left?

00:12:09.540 --> 00:12:13.410
So it's a full matrix
A. No identity there.

00:12:13.410 --> 00:12:16.530
But I'm allowed
to do elimination.

00:12:16.530 --> 00:12:19.920
I'm allowed to do these
three steps when I want to.

00:12:19.920 --> 00:12:21.540
So what shall I do?

00:12:21.540 --> 00:12:25.080
Probably, at this point,
you've seen elimination

00:12:25.080 --> 00:12:26.820
as much as you want
to, but allow me

00:12:26.820 --> 00:12:29.190
to say it once or twice more.

00:12:29.190 --> 00:12:34.050
So I would take 3 of
row 1 away from row 2.

00:12:34.050 --> 00:12:38.640
That would produce-- that 3
that's in the original matrix

00:12:38.640 --> 00:12:40.500
would turn into a 0.

00:12:40.500 --> 00:12:44.290
So when you take 3
of that first row,

00:12:44.290 --> 00:12:47.910
that would be 3, 6, 33, 51.

00:12:47.910 --> 00:12:52.080
I subtract it and I get
just that 0, 1, 4, 6.

00:12:52.080 --> 00:12:55.020
So I've got a much
better second row.

00:12:55.020 --> 00:12:59.370
Now, I'm going to use the
second row going upwards.

00:12:59.370 --> 00:13:03.240
So I'll take 2 of the second
row away from the first row.

00:13:03.240 --> 00:13:06.390
That'll knock out the
2 that I don't want

00:13:06.390 --> 00:13:09.720
and it'll produce the
identity that I do want.

00:13:09.720 --> 00:13:10.800
So do you see that?

00:13:10.800 --> 00:13:15.900
If I take 2 of that last
row, that 0, 2, 8, 12,

00:13:15.900 --> 00:13:19.170
when I take 0, 2, 8, 12
away from the top row,

00:13:19.170 --> 00:13:21.120
I'm left with 1 and 0.

00:13:21.120 --> 00:13:25.200
And 3 and 5 are the numbers
that happen to be left.

00:13:25.200 --> 00:13:27.360
OK.

00:13:27.360 --> 00:13:30.130
So really, what has
elimination done?

00:13:30.130 --> 00:13:32.280
This is an important
idea that I think

00:13:32.280 --> 00:13:35.970
I never caught onto when I
was learning linear algebra.

00:13:35.970 --> 00:13:41.595
Elimination has found the
inverse of this 1, 2, 3,

00:13:41.595 --> 00:13:44.220
7-- of this 2 by 2 matrix.

00:13:44.220 --> 00:13:47.340
Because it's started
with 1, 2, 3, 7 and it's

00:13:47.340 --> 00:13:49.320
ended with the identity.

00:13:49.320 --> 00:13:51.810
So that must have
inverted that matrix.

00:13:51.810 --> 00:13:55.470
And then it had to
apply the inverse

00:13:55.470 --> 00:14:00.810
to the second half of the matrix
and that produced the 3, 4, 5,

00:14:00.810 --> 00:14:02.520
6.

00:14:02.520 --> 00:14:04.830
So you see, this
is-- oh, I think

00:14:04.830 --> 00:14:09.090
there's probably an example
that leads to number two.

00:14:09.090 --> 00:14:11.280
Here, I must have thought ahead.

00:14:11.280 --> 00:14:15.090
So it says on the
screen what I just

00:14:15.090 --> 00:14:19.590
said, that elimination has
inverted the leading matrix.

00:14:19.590 --> 00:14:25.650
And then it's
written out the step

00:14:25.650 --> 00:14:32.280
H equal WF that connects the
original dependent columns

00:14:32.280 --> 00:14:37.680
to the final dependent
columns in F, 3, 4, 5, 6.

00:14:37.680 --> 00:14:38.520
OK.

00:14:38.520 --> 00:14:42.270
I think elimination is
just a matter of practice

00:14:42.270 --> 00:14:44.550
and you probably have done it.

00:14:44.550 --> 00:14:48.740
And here is an important
point about elimination,

00:14:48.740 --> 00:14:52.830
that you could do things
in different orders.

00:14:52.830 --> 00:14:56.210
I spoke and I did them in
a sort of natural order,

00:14:56.210 --> 00:14:59.130
but you could do it other ways.

00:14:59.130 --> 00:15:02.910
But you wouldn't get
a different outcome.

00:15:02.910 --> 00:15:05.430
You'd get the same matrix R.

00:15:05.430 --> 00:15:09.960
Well, because the equations
are still true at every point.

00:15:09.960 --> 00:15:13.020
The x's, the null
space of a matrix,

00:15:13.020 --> 00:15:17.970
is not changing by
my elimination steps.

00:15:17.970 --> 00:15:19.500
Yeah.

00:15:19.500 --> 00:15:21.240
So let's just repeat this.

00:15:21.240 --> 00:15:28.290
When you finally get to R, when
you've done the elimination

00:15:28.290 --> 00:15:31.110
and you get to R with
its identity matrix,

00:15:31.110 --> 00:15:35.910
that identity matrix in R tells
you which columns at the start

00:15:35.910 --> 00:15:38.290
were independent.

00:15:38.290 --> 00:15:44.850
So you need to have a way to
do that and now we have a way.

00:15:44.850 --> 00:15:49.960
In the other opening
talks in this series--

00:15:49.960 --> 00:15:52.990
in this series about
the new start--

00:15:52.990 --> 00:15:55.720
the matrices were
nice and small so I

00:15:55.720 --> 00:15:57.160
didn't need much of a system.

00:15:57.160 --> 00:15:58.900
I could practically wing it.

00:15:58.900 --> 00:16:01.730
But now, we need a system.

00:16:01.730 --> 00:16:08.080
And of course, if we want to
use the Matlab or other code,

00:16:08.080 --> 00:16:09.740
it has to have a system.

00:16:09.740 --> 00:16:16.120
So anyway, we now have a
system of row elimination steps

00:16:16.120 --> 00:16:19.990
and we know what it ends with.

00:16:19.990 --> 00:16:25.000
It ends with an identity matrix
of the right size, a matrix F--

00:16:25.000 --> 00:16:26.920
that's the other columns--

00:16:26.920 --> 00:16:30.910
and then possibly a permutation
to tell us the order.

00:16:30.910 --> 00:16:34.450
That's elimination.

00:16:34.450 --> 00:16:37.410
OK and here's the
second example.

00:16:37.410 --> 00:16:39.470
So we're moving along.

00:16:39.470 --> 00:16:42.230
You've got to pay
attention because this

00:16:42.230 --> 00:16:46.040
is all there is is
two examples and then

00:16:46.040 --> 00:16:48.270
put the ideas together.

00:16:48.270 --> 00:16:50.420
So here is an
example of a matrix A

00:16:50.420 --> 00:16:52.970
with numbers as big as 97.

00:16:52.970 --> 00:16:54.680
I don't know how I got there.

00:16:54.680 --> 00:16:55.610
OK.

00:16:55.610 --> 00:17:00.770
So there are no 0's in
the original matrix A

00:17:00.770 --> 00:17:04.260
but elimination aims to get 0's.

00:17:04.260 --> 00:17:04.760
Oh, here.

00:17:04.760 --> 00:17:08.990
It got 0's very quickly
because the second row

00:17:08.990 --> 00:17:15.200
of that matrix, 2,
14, 6, 70, that's

00:17:15.200 --> 00:17:16.859
just twice the first row.

00:17:16.859 --> 00:17:20.599
So when I subtract 2 times the
first row from the second row,

00:17:20.599 --> 00:17:22.050
I get that 0 row.

00:17:22.050 --> 00:17:25.310
So I'm now one step
along with a row of 0's.

00:17:25.310 --> 00:17:26.690
That's great.

00:17:26.690 --> 00:17:31.250
And the third row
improved too because I

00:17:31.250 --> 00:17:34.340
took 2 of the first row
away from the third row

00:17:34.340 --> 00:17:40.590
and that gave me a 0, 0 at
the start and then a 3, 27.

00:17:40.590 --> 00:17:41.720
So are we OK?

00:17:41.720 --> 00:17:44.120
We're at the second matrix now.

00:17:44.120 --> 00:17:45.350
It's not finished.

00:17:45.350 --> 00:17:50.030
It's not in its R form,
in its echelon form,

00:17:50.030 --> 00:17:52.190
but it's a lot closer.

00:17:52.190 --> 00:17:52.730
OK.

00:17:52.730 --> 00:17:55.100
So what do we still have to do?

00:17:55.100 --> 00:17:59.870
Well I look at that 3, and I'm
looking to get the identity

00:17:59.870 --> 00:18:02.390
matrix, so I'll divide that--

00:18:02.390 --> 00:18:05.900
sooner or later, I'm going
to divide that last row by 3

00:18:05.900 --> 00:18:08.370
and get 0, 0, 1, 9.

00:18:08.370 --> 00:18:12.360
But another thing I want to
do is clean up the first row.

00:18:12.360 --> 00:18:16.130
So I subtract that last
row from the first row.

00:18:16.130 --> 00:18:17.330
You see that?

00:18:17.330 --> 00:18:20.220
See, I'm much closer
to the identity.

00:18:20.220 --> 00:18:23.840
So I'm now moved on
to the third matrix.

00:18:23.840 --> 00:18:26.900
And if you've done a lot of
these, you know what I'm doing.

00:18:29.870 --> 00:18:32.300
I'm working a column at a time.

00:18:32.300 --> 00:18:37.970
And notice, column 2, I
can't do anything with.

00:18:37.970 --> 00:18:39.950
Column 2 is a dependent column.

00:18:39.950 --> 00:18:41.930
It's seven times column 1.

00:18:41.930 --> 00:18:43.880
I can't improve column 2.

00:18:43.880 --> 00:18:47.720
But I can improve column
3 and that's what we do.

00:18:47.720 --> 00:18:55.730
Divide that third, that 0, 0,
3, 27, by 3 and exchange rows.

00:18:55.730 --> 00:18:59.930
Then you see that
the result is the R0.

00:18:59.930 --> 00:19:04.110
So it's the R0 with the 0 row.

00:19:04.110 --> 00:19:04.610
Yeah.

00:19:04.610 --> 00:19:10.310
So just to repeat,
this is a matrix A

00:19:10.310 --> 00:19:13.400
that leads through the
steps of elimination,

00:19:13.400 --> 00:19:17.360
which we're remembering to R0.

00:19:17.360 --> 00:19:21.980
So we're really finished
with this matrix A

00:19:21.980 --> 00:19:24.950
because this shows
how to get it to R0,

00:19:24.950 --> 00:19:28.280
and then we've
already seen with R0,

00:19:28.280 --> 00:19:32.540
we've figured out which
were the special solutions--

00:19:32.540 --> 00:19:35.910
the vectors in the null space.

00:19:35.910 --> 00:19:37.370
So we're good.

00:19:37.370 --> 00:19:39.980
We really have done the job.

00:19:39.980 --> 00:19:46.400
And it remains to just see
a little bit, what did we do

00:19:46.400 --> 00:19:55.040
and what's going on with
these other two columns?

00:19:55.040 --> 00:19:58.280
The F columns that
are not the identity.

00:19:58.280 --> 00:20:01.880
And you see, what
we've done is--

00:20:01.880 --> 00:20:10.490
well, we started
with columns 1 and 3.

00:20:10.490 --> 00:20:13.130
Yeah, so that's our matrix
C. Those are the two

00:20:13.130 --> 00:20:15.330
independent columns.

00:20:15.330 --> 00:20:16.590
Well how do I know?

00:20:16.590 --> 00:20:18.720
Where did C come from?

00:20:18.720 --> 00:20:20.520
Because that's part
of this search,

00:20:20.520 --> 00:20:22.410
where are the
independent columns.

00:20:22.410 --> 00:20:25.410
They're the ones that
end up with the identity

00:20:25.410 --> 00:20:27.660
because the identity
is the way to go

00:20:27.660 --> 00:20:30.180
to the right
independent columns.

00:20:30.180 --> 00:20:34.150
So they're columns 1 and
3 of the original matrix.

00:20:34.150 --> 00:20:35.550
So you see C there?

00:20:35.550 --> 00:20:37.920
1, 2, 2 and 3, 6, 9.

00:20:37.920 --> 00:20:44.660
Then F is the part of R
from the dependent column.

00:20:44.660 --> 00:20:46.830
So that's the 7, 8, 0, 9.

00:20:46.830 --> 00:20:50.340
And then I'm seeing
the dependent columns.

00:20:50.340 --> 00:20:50.840
Yeah.

00:20:50.840 --> 00:20:57.770
You have to do this a few times,
but all the ideas now are--

00:20:57.770 --> 00:21:00.860
and there I've put it together.

00:21:00.860 --> 00:21:04.550
So the idea is,
with a small matrix,

00:21:04.550 --> 00:21:08.450
like only three columns
and maybe only two rows,

00:21:08.450 --> 00:21:13.380
we could find C and
R almost by sight.

00:21:13.380 --> 00:21:19.220
But now, even where we're up
to four columns and three rows,

00:21:19.220 --> 00:21:20.510
we need a method.

00:21:20.510 --> 00:21:23.630
And that's the point of
this talk and it was also

00:21:23.630 --> 00:21:28.520
the point of lecture seven
in the original 18.06--

00:21:28.520 --> 00:21:31.940
Math 18.06 on OpenCourseWare.

00:21:31.940 --> 00:21:35.000
OK, so we've found
C and R. Good.

00:21:35.000 --> 00:21:36.170
Good.

00:21:36.170 --> 00:21:40.430
I think it's just congratulating
ourselves from here in.

00:21:40.430 --> 00:21:43.370
These are remembering
the special solutions.

00:21:43.370 --> 00:21:46.080
Oh, and then why not--

00:21:46.080 --> 00:21:48.110
you remember those?

00:21:48.110 --> 00:21:51.240
That second column was
7 of the first column,

00:21:51.240 --> 00:21:55.160
so that gave that special
solution, that combination,

00:21:55.160 --> 00:21:56.450
to give 0.

00:21:56.450 --> 00:22:00.140
And then the other
combination gave the 0 vector.

00:22:00.140 --> 00:22:03.060
So we've got two
special solutions.

00:22:03.060 --> 00:22:05.630
Oh, and here is the
general principle.

00:22:05.630 --> 00:22:06.330
Yeah.

00:22:06.330 --> 00:22:06.830
Yeah.

00:22:06.830 --> 00:22:07.730
It's very simple.

00:22:07.730 --> 00:22:12.850
After you've got this matrix
down to identity and F,

00:22:12.850 --> 00:22:19.520
then the solutions x
should have a minus FI.

00:22:19.520 --> 00:22:22.430
You remember the minus
signs that we saw?

00:22:22.430 --> 00:22:24.830
So all I'm saying
here on the first line

00:22:24.830 --> 00:22:32.340
is that the matrix IF times
minus FI gives the 0 matrix.

00:22:32.340 --> 00:22:33.750
Of course it does.

00:22:33.750 --> 00:22:38.400
And then the second line is
the little bit of special trick

00:22:38.400 --> 00:22:43.380
that you have to do if
there's a P, a permutation,

00:22:43.380 --> 00:22:45.840
an exchange involved here.

00:22:45.840 --> 00:22:49.140
When the identity matrix is
not sitting where you want it,

00:22:49.140 --> 00:22:51.660
you have to have a
P to put it there.

00:22:51.660 --> 00:22:56.610
Then P transpose has to
come into the solutions.

00:22:56.610 --> 00:22:57.330
Yeah.

00:22:57.330 --> 00:22:59.940
Yeah, because P
times P transpose.

00:22:59.940 --> 00:23:03.330
So remember about
permutation matrices.

00:23:03.330 --> 00:23:08.580
Linear algebra is about learning
different special character

00:23:08.580 --> 00:23:09.900
of different matrices.

00:23:09.900 --> 00:23:16.230
And permutations just take the
identity matrix and mess around

00:23:16.230 --> 00:23:21.810
with the rows or mess around
with the columns, same thing.

00:23:21.810 --> 00:23:25.740
And then P transpose
unmesses it.

00:23:25.740 --> 00:23:27.940
Brings it back to the identity.

00:23:27.940 --> 00:23:31.020
So P times P transpose
is the identity.

00:23:31.020 --> 00:23:34.380
OK, so we're just about there.

00:23:34.380 --> 00:23:38.130
Oh, then this is the
little point I made--

00:23:38.130 --> 00:23:39.960
well, it's an important point.

00:23:39.960 --> 00:23:45.980
Suppose I have five columns
and only three rows.

00:23:45.980 --> 00:23:49.430
That means I've got
five unknowns and only

00:23:49.430 --> 00:23:51.150
three equations.

00:23:51.150 --> 00:23:53.360
So there are going to
be solutions to that.

00:23:53.360 --> 00:23:58.520
There are going to be solutions
other than the 0 solution.

00:23:58.520 --> 00:24:02.510
The 0 combination
will surely give

00:24:02.510 --> 00:24:09.020
0, but if I have five unknowns,
five columns in there,

00:24:09.020 --> 00:24:13.260
and I have only three
equations, three rows--

00:24:13.260 --> 00:24:16.460
so five greater than
three, n greater than m--

00:24:16.460 --> 00:24:20.640
then there will be
some columns in F.

00:24:20.640 --> 00:24:22.850
There'll be some
non-zero solutions.

00:24:22.850 --> 00:24:25.590
And these examples show.

00:24:25.590 --> 00:24:30.020
So take that matrix
M at the end there.

00:24:30.020 --> 00:24:36.020
You see there's a matrix with
four columns and only two rows.

00:24:36.020 --> 00:24:38.320
So I've got four
vectors in the plane.

00:24:38.320 --> 00:24:40.630
If I have four
vectors in the plane,

00:24:40.630 --> 00:24:44.470
there are combinations
that give 0.

00:24:44.470 --> 00:24:47.210
They can complete triangles.

00:24:47.210 --> 00:24:51.850
So in that case,
rank would be 2.

00:24:51.850 --> 00:24:58.000
So we're really seeing this
part of linear algebra,

00:24:58.000 --> 00:25:02.950
to simplify the matrix by
elimination so that all

00:25:02.950 --> 00:25:07.250
the main facts are clear.

00:25:07.250 --> 00:25:07.750
Yeah.

00:25:07.750 --> 00:25:14.020
And oh, if you really want
to see it in shorthand--

00:25:14.020 --> 00:25:18.010
I'm not necessarily
pushing this last idea,

00:25:18.010 --> 00:25:20.620
but if you want to
see it in a shorthand,

00:25:20.620 --> 00:25:25.570
we could think of the matrix
as having just four blocks.

00:25:25.570 --> 00:25:28.060
So it's a 2 by 2 matrix,
but unfortunately,

00:25:28.060 --> 00:25:31.510
each of those guys is a block.

00:25:31.510 --> 00:25:35.770
So W is the one in the corner
that we talked about earlier

00:25:35.770 --> 00:25:38.230
that gets inverted.

00:25:38.230 --> 00:25:41.200
So if we look at
the last line, that

00:25:41.200 --> 00:25:47.130
W in the corner, that
ended up as the identity.

00:25:47.130 --> 00:25:53.550
The second block row, the
J, K, ended up as 0's.

00:25:53.550 --> 00:26:00.180
And the one remaining guy,
it tells us the combinations

00:26:00.180 --> 00:26:02.130
that we are looking for--

00:26:02.130 --> 00:26:05.880
is this W inverse H.

00:26:05.880 --> 00:26:09.530
So let me look at that
equation in the box

00:26:09.530 --> 00:26:13.520
W, H, J, K, that's the
matrix I'm starting with.

00:26:13.520 --> 00:26:17.900
I'm going to invert W
and I know that J, K is

00:26:17.900 --> 00:26:24.470
some combination of W, H.
So when I invert W, I get--

00:26:24.470 --> 00:26:29.000
the first elimination
produces the first row,

00:26:29.000 --> 00:26:30.630
starting with the identity.

00:26:30.630 --> 00:26:34.850
And when I apply that to
the second, the J, K row,

00:26:34.850 --> 00:26:36.620
those go to 0.

00:26:36.620 --> 00:26:42.380
So that's what elimination
is doing in real shorthand.

00:26:42.380 --> 00:26:46.850
It's taking a 2 by 2 block
matrix to that matrix R

00:26:46.850 --> 00:26:51.080
at the end, with a 0
block row and an identity

00:26:51.080 --> 00:26:53.360
block in the upper corner.

00:26:53.360 --> 00:26:58.700
So that is elimination
and the solution of--

00:26:58.700 --> 00:27:00.600
and finding the null space.

00:27:00.600 --> 00:27:04.670
So really, this
completes the job

00:27:04.670 --> 00:27:10.800
of the first topic in linear
algebra, is identifying--

00:27:10.800 --> 00:27:15.120
understanding these four
fundamental subspaces.

00:27:15.120 --> 00:27:16.980
The null space
being one of them.

00:27:16.980 --> 00:27:18.900
The column space being another.

00:27:18.900 --> 00:27:22.320
We understand now which
columns are independent.

00:27:22.320 --> 00:27:25.440
And the row space being another.

00:27:25.440 --> 00:27:30.330
And so we're really
understanding

00:27:30.330 --> 00:27:35.100
the idea of these four
fundamental subspaces that

00:27:35.100 --> 00:27:38.160
go into the big picture
of linear algebra.

00:27:38.160 --> 00:27:45.920
So this completes the first
major stage of a linear algebra

00:27:45.920 --> 00:27:46.820
course.

00:27:46.820 --> 00:27:50.960
And then what's to
follow will come.

00:27:50.960 --> 00:27:55.790
Eigenvalues, singular values,
applications of all kinds.

00:27:55.790 --> 00:27:57.050
Least squares.

00:27:57.050 --> 00:27:57.560
Good.

00:27:57.560 --> 00:27:58.220
Good.

00:27:58.220 --> 00:27:59.300
Good math.

00:27:59.300 --> 00:28:01.220
And thank you very much.

00:28:01.220 --> 00:28:07.790
So that's my summary of
finding the null space.

00:28:07.790 --> 00:28:09.340
Good.