WEBVTT
00:00:11.825 --> 00:00:12.325
OK.
00:00:14.830 --> 00:00:15.900
cameras are rolling.
00:00:15.900 --> 00:00:21.840
This is lecture fourteen,
starting a new chapter.
00:00:21.840 --> 00:00:24.660
Chapter about orthogonality.
00:00:24.660 --> 00:00:29.480
What it means for
vectors to be orthogonal.
00:00:29.480 --> 00:00:33.070
What it means for
subspaces to be orthogonal.
00:00:33.070 --> 00:00:37.370
What it means for
bases to be orthogonal.
00:00:37.370 --> 00:00:44.460
So ninety degrees, this is
a ninety-degree chapter.
00:00:44.460 --> 00:00:47.800
So what does it mean --
00:00:47.800 --> 00:00:50.890
let me jump to subspaces.
00:00:50.890 --> 00:00:54.830
Because I've drawn
here the big picture.
00:00:54.830 --> 00:00:59.910
This is the 18.06 picture here.
00:00:59.910 --> 00:01:04.800
And hold it down, guys.
00:01:04.800 --> 00:01:09.280
So this is the picture and we
know a lot about that picture
00:01:09.280 --> 00:01:10.550
already.
00:01:10.550 --> 00:01:14.570
We know the dimension
of every subspace.
00:01:14.570 --> 00:01:22.040
We know that these
dimensions are r and n-r.
00:01:22.040 --> 00:01:26.320
We know that these
dimensions are r and m-r.
00:01:29.680 --> 00:01:33.780
What I want to show now is
what this figure is saying,
00:01:33.780 --> 00:01:38.140
that the angle --
00:01:38.140 --> 00:01:43.520
the figure is just my attempt to
draw what I'm now going to say,
00:01:43.520 --> 00:01:50.510
that the angle between these
subspaces is ninety degrees.
00:01:50.510 --> 00:01:53.890
And the angle between these
subspaces is ninety degrees.
00:01:53.890 --> 00:01:55.670
Now I have to say
what does that mean?
00:01:55.670 --> 00:02:01.080
What does it mean for
subspaces to be orthogonal?
00:02:01.080 --> 00:02:06.540
But I hope you appreciate
the beauty of this picture,
00:02:06.540 --> 00:02:11.340
that that those
subspaces are going
00:02:11.340 --> 00:02:14.170
to come out to be orthogonal.
00:02:14.170 --> 00:02:18.140
Those two and also those two.
00:02:18.140 --> 00:02:24.870
So that's like one point,
one important point
00:02:24.870 --> 00:02:28.960
to step forward in
understanding those subspaces.
00:02:28.960 --> 00:02:31.140
We knew what each
subspace was like,
00:02:31.140 --> 00:02:33.500
we could compute bases for them.
00:02:33.500 --> 00:02:35.440
Now we know more.
00:02:35.440 --> 00:02:38.240
Or we will in a few minutes.
00:02:38.240 --> 00:02:38.750
OK.
00:02:38.750 --> 00:02:43.110
I have to say first of all what
does it mean for two vectors
00:02:43.110 --> 00:02:44.920
to be orthogonal?
00:02:44.920 --> 00:02:46.470
So let me start with that.
00:02:49.560 --> 00:02:50.560
Orthogonal vectors.
00:02:57.010 --> 00:03:00.610
The word orthogonal is
-- is just another word
00:03:00.610 --> 00:03:02.620
for perpendicular.
00:03:02.620 --> 00:03:05.510
It means that in
n-dimensional space
00:03:05.510 --> 00:03:10.290
the angle between those
vectors is ninety degrees.
00:03:10.290 --> 00:03:13.480
It means that they
form a right triangle.
00:03:13.480 --> 00:03:20.940
It even means that the going
way back to the Greeks that this
00:03:20.940 --> 00:03:29.010
angle that this triangle
a vector x, a vector x,
00:03:29.010 --> 00:03:36.560
and a vector x+y -- of course
that'll be the hypotenuse,
00:03:36.560 --> 00:03:42.750
so what was it the Greeks
figured out and it's neat.
00:03:42.750 --> 00:03:45.860
It's the fact that the --
00:03:45.860 --> 00:03:50.980
so these are orthogonal,
this is a right angle, if --
00:03:50.980 --> 00:03:57.660
so let me put the great
name down, Pythagoras,
00:03:57.660 --> 00:04:00.540
I'm looking for --
what I looking for?
00:04:00.540 --> 00:04:05.980
I'm looking for the condition if
you give me two vectors, how do
00:04:05.980 --> 00:04:08.540
I know if they're orthogonal?
00:04:08.540 --> 00:04:11.490
How can I tell two
perpendicular vectors?
00:04:11.490 --> 00:04:13.830
And actually you
probably know the answer.
00:04:13.830 --> 00:04:15.770
Let me write the answer down.
00:04:15.770 --> 00:04:20.550
Orthogonal vectors, what's
the test for orthogonality?
00:04:20.550 --> 00:04:29.430
I take the dot product which I
tend to write as x transpose y,
00:04:29.430 --> 00:04:33.060
because that's a
row times a column,
00:04:33.060 --> 00:04:38.390
and that matrix multiplication
just gives me the right thing,
00:04:38.390 --> 00:04:45.340
that x1y1+x2y2 and so on, so
these vectors are orthogonal
00:04:45.340 --> 00:04:52.560
if this result x
transpose y is zero.
00:04:52.560 --> 00:04:54.130
That's the test.
00:04:54.130 --> 00:04:55.340
OK.
00:04:55.340 --> 00:05:00.850
Can I connect that
to other things?
00:05:00.850 --> 00:05:06.480
I mean -- it's just beautiful
that here we have we're in n
00:05:06.480 --> 00:05:09.390
dimensions, we've got
a couple of vectors,
00:05:09.390 --> 00:05:11.620
we want to know the
angle between them,
00:05:11.620 --> 00:05:16.750
and the right thing to look at
is the simplest thing that you
00:05:16.750 --> 00:05:18.930
could imagine, the dot product.
00:05:18.930 --> 00:05:19.600
OK.
00:05:19.600 --> 00:05:20.890
Now why?
00:05:20.890 --> 00:05:24.430
So I'm answering the
question now why --
00:05:24.430 --> 00:05:33.500
let's add some justification
to this fact, that's the test.
00:05:33.500 --> 00:05:38.050
OK, so Pythagoras would say
we've got a right triangle,
00:05:38.050 --> 00:05:41.400
if that length squared plus
that length squared equals
00:05:41.400 --> 00:05:43.170
that length squared.
00:05:43.170 --> 00:05:51.800
OK, can I write it as x
squared plus y squared equals
00:05:51.800 --> 00:05:54.890
x plus y squared?
00:05:54.890 --> 00:05:59.120
Now don't, please don't think
that this is always true.
00:05:59.120 --> 00:06:03.360
This is only going
to be true in this --
00:06:03.360 --> 00:06:07.600
it's going to be equivalent
to orthogonality.
00:06:07.600 --> 00:06:11.180
For other triangles of
course, it's not true.
00:06:11.180 --> 00:06:12.540
For other triangles it's not.
00:06:12.540 --> 00:06:16.860
But for a right triangle
somehow that fact
00:06:16.860 --> 00:06:18.810
should connect to that fact.
00:06:18.810 --> 00:06:20.240
Can we just make
that connection?
00:06:23.580 --> 00:06:28.090
What's the connection between
this test for orthogonality
00:06:28.090 --> 00:06:31.040
and this statement
of orthogonality?
00:06:31.040 --> 00:06:35.520
Well, I guess I have to say
what is the length squared?
00:06:35.520 --> 00:06:42.370
So let's continue on the board
underneath with that equation.
00:06:42.370 --> 00:06:47.800
Give me another way to express
the length squared of a vector.
00:06:47.800 --> 00:06:49.960
And let me just
give you a vector.
00:06:49.960 --> 00:06:53.420
The vector one, two, three.
00:06:53.420 --> 00:06:55.730
That's in three dimensions.
00:06:55.730 --> 00:07:00.720
What is the length squared of
the vector x equal one, two,
00:07:00.720 --> 00:07:03.410
three?
00:07:03.410 --> 00:07:06.820
So how do you find
the length squared?
00:07:06.820 --> 00:07:10.710
Well, really you just, you want
the length of that vector that
00:07:10.710 --> 00:07:15.220
goes along one -- up
two, and out three --
00:07:15.220 --> 00:07:19.910
and we'll come back to
that right triangle stuff.
00:07:19.910 --> 00:07:25.928
The length squared is
exactly x transpose x.
00:07:28.870 --> 00:07:31.230
Whenever I see x
transpose x, I know
00:07:31.230 --> 00:07:35.040
I've got a number
that's positive.
00:07:35.040 --> 00:07:37.750
It's a length
squared unless it --
00:07:37.750 --> 00:07:40.440
unless x happens to
be the zero vector,
00:07:40.440 --> 00:07:44.500
that's the one case
where the length is zero.
00:07:44.500 --> 00:07:49.770
So right -- this is just x1
squared plus x2 squared plus
00:07:49.770 --> 00:07:51.920
so on, plus xn squared.
00:07:51.920 --> 00:07:55.830
So one -- in the example I gave
you what was the length squared
00:07:55.830 --> 00:07:59.230
of that vector one, two, three?
00:07:59.230 --> 00:08:02.070
So you square those, you
get one, four and nine,
00:08:02.070 --> 00:08:04.310
you add, you get fourteen.
00:08:04.310 --> 00:08:08.590
So the vector one, two,
three has length fourteen.
00:08:08.590 --> 00:08:13.000
So let me just put
down a vector here.
00:08:13.000 --> 00:08:15.995
Let x be the vector
one, two, three.
00:08:19.220 --> 00:08:22.060
Let me cook up a vector
that's orthogonal to it.
00:08:25.550 --> 00:08:27.590
So what's the vector
that's orthogonal?
00:08:27.590 --> 00:08:33.270
So right down here, x squared
is one plus four plus nine,
00:08:33.270 --> 00:08:38.480
fourteen, let me cook up a
vector that's orthogonal to it,
00:08:38.480 --> 00:08:47.170
we'll get right that that
-- those two vectors are
00:08:47.170 --> 00:08:55.320
orthogonal, the length
of y squared is five,
00:08:55.320 --> 00:09:02.520
and x plus y is one
and two making three,
00:09:02.520 --> 00:09:06.290
two and minus one making one,
three and zero making three,
00:09:06.290 --> 00:09:13.010
and the length of this squared
is nine plus one plus nine,
00:09:13.010 --> 00:09:15.700
nineteen.
00:09:15.700 --> 00:09:20.810
And sure enough, I
haven't proved anything.
00:09:20.810 --> 00:09:28.900
I've just like checked to
see that my test x transpose
00:09:28.900 --> 00:09:31.530
y equals zero, which
is true, right?
00:09:31.530 --> 00:09:35.370
Everybody sees that x
transpose y is zero here?
00:09:35.370 --> 00:09:37.740
That's maybe the main point.
00:09:37.740 --> 00:09:41.910
That you should get really
quick at doing x transpose y,
00:09:41.910 --> 00:09:45.980
so it's just this plus this
plus this and that's zero.
00:09:45.980 --> 00:09:51.730
And sure enough, that clicks
with fourteen plus five
00:09:51.730 --> 00:09:53.190
agreeing with nineteen.
00:09:53.190 --> 00:09:56.780
Now let me just do
that in letters.
00:09:56.780 --> 00:10:01.110
So that's y transpose y.
00:10:01.110 --> 00:10:08.150
And this is x plus y
transpose x plus y.
00:10:08.150 --> 00:10:09.290
OK.
00:10:09.290 --> 00:10:12.530
So I'm looking, again,
this isn't always true.
00:10:12.530 --> 00:10:14.110
I repeat.
00:10:14.110 --> 00:10:18.020
This is going to be true
when we have a right angle.
00:10:18.020 --> 00:10:20.210
And let's just --
well, of course,
00:10:20.210 --> 00:10:22.850
I'm just going to
simplify this stuff here.
00:10:22.850 --> 00:10:25.960
There's an x transpose x there.
00:10:25.960 --> 00:10:29.850
And there's a y
transpose y there.
00:10:29.850 --> 00:10:34.450
And there's an x transpose y.
00:10:34.450 --> 00:10:36.610
And there's a y transpose x.
00:10:41.700 --> 00:10:46.990
I knew I could do that
simplification because I'm just
00:10:46.990 --> 00:10:50.150
doing matrix multiplication and
I've just followed the rules.
00:10:50.150 --> 00:10:50.880
OK.
00:10:50.880 --> 00:10:53.650
So x transpose x-s cancel.
00:10:53.650 --> 00:10:55.910
Y transpose y-s cancel.
00:10:55.910 --> 00:10:57.950
And what about these guys?
00:10:57.950 --> 00:11:03.400
What can you tell me about
the inner product of x with y
00:11:03.400 --> 00:11:06.890
and the inner
product of y with x?
00:11:06.890 --> 00:11:09.620
Is there a difference?
00:11:09.620 --> 00:11:15.150
I think if we -- while
we're doing real vectors,
00:11:15.150 --> 00:11:18.310
which is all we're doing now,
there isn't a difference,
00:11:18.310 --> 00:11:19.660
there's no difference.
00:11:19.660 --> 00:11:23.690
If I take x transpose
y, that'll give me zero,
00:11:23.690 --> 00:11:28.840
if I took y transpose x I would
have the same x1y1 and x2y2
00:11:28.840 --> 00:11:35.280
and x3y3, it would be
the same, so this is --
00:11:35.280 --> 00:11:37.820
this is the same as
that, this is really
00:11:37.820 --> 00:11:40.070
I'll knock that guy out
and say two of these.
00:11:43.700 --> 00:11:47.360
So actually that's the --
00:11:47.360 --> 00:11:51.960
this equation boiled down
to this thing being zero.
00:11:51.960 --> 00:11:52.670
Right?
00:11:52.670 --> 00:11:56.010
Everything else canceled
and this equation
00:11:56.010 --> 00:11:57.250
boiled down to that one.
00:11:57.250 --> 00:12:00.450
So that's really all I wanted.
00:12:00.450 --> 00:12:08.170
I just wanted to check that
Pythagoras for a right triangle
00:12:08.170 --> 00:12:13.250
led me to this -- of course
I cancel the two now.
00:12:13.250 --> 00:12:14.680
No problem.
00:12:14.680 --> 00:12:18.880
To x transpose y equals
zero as the test.
00:12:18.880 --> 00:12:19.610
Fair enough.
00:12:19.610 --> 00:12:21.190
OK.
00:12:21.190 --> 00:12:23.150
You knew it was coming.
00:12:23.150 --> 00:12:27.010
The dot product of
orthogonal vectors is zero.
00:12:27.010 --> 00:12:31.010
It's just -- I just want
to say that's really neat.
00:12:31.010 --> 00:12:32.760
That it comes out so well.
00:12:32.760 --> 00:12:34.830
All right.
00:12:34.830 --> 00:12:38.760
Now what about -- so
now I know if two --
00:12:38.760 --> 00:12:40.700
when two vectors are orthogonal.
00:12:40.700 --> 00:12:45.860
By the way, what about if one of
these guys is the zero vector?
00:12:45.860 --> 00:12:51.660
Suppose x is the zero
vector, and y is whatever.
00:12:51.660 --> 00:12:55.010
Are they orthogonal?
00:12:55.010 --> 00:12:57.190
Sure.
00:12:57.190 --> 00:12:59.310
In math the one thing
about math is you're
00:12:59.310 --> 00:13:01.910
supposed to follow the rules.
00:13:01.910 --> 00:13:05.450
So you're supposed to --
if x is the zero vector,
00:13:05.450 --> 00:13:09.330
you're supposed to take the
zero vector dotted with y
00:13:09.330 --> 00:13:12.180
and of course you
always get zero.
00:13:12.180 --> 00:13:17.340
So just so we're all
sure, the zero vector
00:13:17.340 --> 00:13:20.730
is orthogonal to everybody.
00:13:20.730 --> 00:13:24.770
But what I want to --
00:13:24.770 --> 00:13:32.240
what I now want to think
about is subspaces.
00:13:32.240 --> 00:13:37.530
What does it mean for me to
say that some subspace is
00:13:37.530 --> 00:13:39.320
orthogonal to some
other subspace?
00:13:39.320 --> 00:13:41.110
So OK.
00:13:41.110 --> 00:13:42.590
Now I've got to write this down.
00:13:42.590 --> 00:13:47.960
So because we're defining
definition of subspace
00:13:47.960 --> 00:14:02.390
S is orthogonal so to
subspace let's say T,
00:14:02.390 --> 00:14:04.130
so I've got a
couple of subspaces.
00:14:07.350 --> 00:14:11.480
And what should it mean for
those guys to be orthogonal?
00:14:11.480 --> 00:14:14.780
It's just sort of what's
the natural extension
00:14:14.780 --> 00:14:19.990
from orthogonal vectors
to orthogonal subspaces?
00:14:19.990 --> 00:14:26.820
Well, and in
particular, let's think
00:14:26.820 --> 00:14:34.410
of some orthogonal
subspaces, like this wall.
00:14:34.410 --> 00:14:37.300
Let's say in three dimensions.
00:14:37.300 --> 00:14:41.320
So the blackboard extended
to infinity, right, is a --
00:14:41.320 --> 00:14:44.950
is a subspace, a plane, a
two-dimensional subspace.
00:14:47.680 --> 00:14:51.310
It's a little bumpy
but anyway, it's a --
00:14:51.310 --> 00:14:57.001
think of it as a subspace, let
me take the floor as another
00:14:57.001 --> 00:14:57.500
subspace.
00:15:01.850 --> 00:15:07.840
Again, it's not
a great subspace,
00:15:07.840 --> 00:15:15.400
MIT only built it
like so-so, but I'll
00:15:15.400 --> 00:15:18.840
put the origin right here.
00:15:18.840 --> 00:15:23.040
So the origin of the
world is right there.
00:15:23.040 --> 00:15:23.540
OK.
00:15:26.730 --> 00:15:30.340
Thereby giving linear algebra
its proper importance in this.
00:15:30.340 --> 00:15:31.190
OK.
00:15:31.190 --> 00:15:34.290
So there is one subspace,
there's another one.
00:15:34.290 --> 00:15:35.940
The floor.
00:15:35.940 --> 00:15:42.200
And are they orthogonal?
00:15:42.200 --> 00:15:44.130
What does it mean
for two subspaces
00:15:44.130 --> 00:15:46.460
to be orthogonal and
in that special case
00:15:46.460 --> 00:15:47.650
are they orthogonal?
00:15:47.650 --> 00:15:48.290
All right.
00:15:48.290 --> 00:15:50.950
Let's finish this sentence.
00:15:50.950 --> 00:15:58.501
What does it mean
means we have to know
00:15:58.501 --> 00:15:59.500
what we're talking about
00:15:59.500 --> 00:16:00.040
here.
00:16:00.040 --> 00:16:05.310
So what would be a reasonable
idea of orthogonal?
00:16:05.310 --> 00:16:08.740
Well, let me put
the right thing up.
00:16:08.740 --> 00:16:17.930
It means that every vector
in S, every vector in S,
00:16:17.930 --> 00:16:23.565
is orthogonal to --
00:16:27.290 --> 00:16:30.080
what I going to say?
00:16:30.080 --> 00:16:38.000
Every vector in T.
00:16:38.000 --> 00:16:42.990
That's a reasonable
and it's a good
00:16:42.990 --> 00:16:45.560
and it's the right
definition for two subspaces
00:16:45.560 --> 00:16:47.010
to be orthogonal.
00:16:47.010 --> 00:16:50.740
But I just want you to see,
hey, what does that mean?
00:16:50.740 --> 00:16:53.620
So answer the
question about the --
00:16:53.620 --> 00:16:57.440
the blackboard and the floor.
00:16:57.440 --> 00:17:01.440
Are those two subspaces,
they're two-dimensional, right,
00:17:01.440 --> 00:17:03.830
and we're in R^3.
00:17:03.830 --> 00:17:09.665
It's like a xz plane or
something and a xy plane.
00:17:15.089 --> 00:17:17.109
Are they orthogonal?
00:17:17.109 --> 00:17:19.560
Is every vector in the
blackboard orthogonal
00:17:19.560 --> 00:17:23.349
to every vector in the floor,
starting from the origin
00:17:23.349 --> 00:17:24.140
right there?
00:17:27.640 --> 00:17:28.900
Yes or no?
00:17:28.900 --> 00:17:31.010
I could take a vote.
00:17:31.010 --> 00:17:35.040
Well we get some
yeses and some noes.
00:17:35.040 --> 00:17:36.170
No is the answer.
00:17:36.170 --> 00:17:37.810
They're not.
00:17:37.810 --> 00:17:41.340
You can tell me a
vector in the blackboard
00:17:41.340 --> 00:17:43.995
and a vector in the floor
that are not orthogonal.
00:17:49.360 --> 00:17:54.960
Well you can tell me
quite a few, I guess.
00:17:54.960 --> 00:17:59.140
Maybe like I could take
some forty-five-degree guy
00:17:59.140 --> 00:18:05.730
in the blackboard, and
something in the floor,
00:18:05.730 --> 00:18:08.580
they're not at ninety
degrees, right?
00:18:08.580 --> 00:18:10.780
In fact, even more,
you could tell me
00:18:10.780 --> 00:18:17.140
a vector that's in both the
blackboard plane and the floor
00:18:17.140 --> 00:18:20.550
plane, so it's certainly
not orthogonal to itself.
00:18:20.550 --> 00:18:25.600
So for sure, those two
planes aren't orthogonal.
00:18:25.600 --> 00:18:26.950
What would that be?
00:18:26.950 --> 00:18:29.350
So what's a vector that's --
00:18:29.350 --> 00:18:32.320
in both of those planes?
00:18:32.320 --> 00:18:35.170
It's this guy running
along the crack
00:18:35.170 --> 00:18:39.540
there, in the intersection,
the intersection.
00:18:39.540 --> 00:18:41.360
A vector, you know --
00:18:41.360 --> 00:18:45.460
if two subspaces meet at some
vector, well then for sure
00:18:45.460 --> 00:18:51.170
they're not orthogonal,
because that vector is in one
00:18:51.170 --> 00:18:54.240
and it's in the other, and
it's not orthogonal to itself
00:18:54.240 --> 00:18:55.770
unless it's zero.
00:18:55.770 --> 00:19:01.000
So the only I mean
so orthogonal is
00:19:01.000 --> 00:19:04.220
for me to say these two
subspaces are orthogonal first
00:19:04.220 --> 00:19:09.080
of all I'm certainly saying
that they don't intersect
00:19:09.080 --> 00:19:14.550
in any nonzero vector.
00:19:14.550 --> 00:19:18.430
But also I mean more than
that just not intersecting
00:19:18.430 --> 00:19:18.990
isn't good
00:19:18.990 --> 00:19:19.490
enough.
00:19:19.490 --> 00:19:26.150
So give me an example, oh,
let's say in the plane, oh well,
00:19:26.150 --> 00:19:30.090
when do we have orthogonal
subspaces in the plane?
00:19:30.090 --> 00:19:32.100
Yeah, tell me in the
plane, so we don't --
00:19:32.100 --> 00:19:34.960
there aren't that many different
subspaces in the plane.
00:19:34.960 --> 00:19:38.385
What what have we got in the
plane as possible subspaces?
00:19:40.900 --> 00:19:44.420
The zero vector, real small.
00:19:44.420 --> 00:19:47.330
A line through the origin.
00:19:47.330 --> 00:19:49.570
Or the whole plane.
00:19:49.570 --> 00:19:50.420
OK.
00:19:50.420 --> 00:19:55.769
Now so when is a line
through the origin orthogonal
00:19:55.769 --> 00:19:56.560
to the whole plane?
00:19:59.870 --> 00:20:02.070
Never, right, never.
00:20:02.070 --> 00:20:05.620
When is a line through the
origin orthogonal to the zero
00:20:05.620 --> 00:20:06.120
subspace?
00:20:09.310 --> 00:20:10.180
Always.
00:20:10.180 --> 00:20:10.910
Right.
00:20:10.910 --> 00:20:13.480
When is a line through
the origin orthogonal
00:20:13.480 --> 00:20:15.920
to a different line
through the origin?
00:20:15.920 --> 00:20:19.880
Well, that's the case that
we all have a clear picture
00:20:19.880 --> 00:20:21.070
of, they --
00:20:21.070 --> 00:20:23.610
the two lines have to
meet at ninety degrees.
00:20:23.610 --> 00:20:28.340
They have only the -- so
that's like this simple case
00:20:28.340 --> 00:20:29.090
I'm talking about.
00:20:29.090 --> 00:20:31.970
There's one subspace,
there's the other subspace.
00:20:31.970 --> 00:20:33.770
They only meet at zero.
00:20:33.770 --> 00:20:35.570
And they're orthogonal.
00:20:35.570 --> 00:20:36.340
OK.
00:20:36.340 --> 00:20:37.930
Now.
00:20:37.930 --> 00:20:43.540
So we now know what it means for
two subspaces to be orthogonal.
00:20:43.540 --> 00:20:47.240
And now I want to say that
this is true for the row
00:20:47.240 --> 00:20:49.080
space and the null space.
00:20:49.080 --> 00:20:49.740
OK.
00:20:49.740 --> 00:20:53.880
So that's the neat fact.
00:20:53.880 --> 00:21:05.720
So row space is orthogonal
to the null space.
00:21:05.720 --> 00:21:07.435
Now how did I come up with that?
00:21:12.540 --> 00:21:16.500
But you see the rank it's
great, that means that these --
00:21:16.500 --> 00:21:19.020
that these subspaces are
just the right things,
00:21:19.020 --> 00:21:23.220
they're just cutting
the whole space up
00:21:23.220 --> 00:21:27.560
into two perpendicular
subspaces.
00:21:27.560 --> 00:21:28.060
OK.
00:21:28.060 --> 00:21:28.560
So why?
00:21:33.710 --> 00:21:38.040
Well, what have I
got to work with?
00:21:38.040 --> 00:21:41.230
All I know is the null space.
00:21:41.230 --> 00:21:46.360
The null space has vectors
that solve Ax equals zero.
00:21:46.360 --> 00:21:49.780
So this is a guy x.
00:21:49.780 --> 00:21:53.940
x is in the null space.
00:21:53.940 --> 00:21:57.350
Then Ax is zero.
00:21:57.350 --> 00:22:03.250
So why is it orthogonal
to the rows of A?
00:22:03.250 --> 00:22:05.340
If I write down Ax
equals zero, which
00:22:05.340 --> 00:22:07.720
is all I know about
the null space,
00:22:07.720 --> 00:22:13.130
then I guess I want you to
see that that's telling me,
00:22:13.130 --> 00:22:16.430
just that equation right
there is telling me
00:22:16.430 --> 00:22:19.670
that the rows of A,
let me write it out.
00:22:19.670 --> 00:22:24.000
There's row one of A.
00:22:24.000 --> 00:22:24.540
Row two.
00:22:27.290 --> 00:22:32.300
Row m of A. that's A.
00:22:32.300 --> 00:22:34.530
And it's multiplying X.
00:22:34.530 --> 00:22:36.560
And it's producing zero.
00:22:36.560 --> 00:22:37.060
OK.
00:22:41.550 --> 00:22:47.200
Written out that
way you'll see it.
00:22:47.200 --> 00:22:50.200
So I'm saying that a
vector in the row space
00:22:50.200 --> 00:22:54.250
is perpendicular to this
guy X in the null space.
00:22:54.250 --> 00:22:57.310
And you see why?
00:22:57.310 --> 00:22:59.550
Because this equation
is telling you
00:22:59.550 --> 00:23:06.330
that row one of A multiplying
that's a dot product, right?
00:23:06.330 --> 00:23:11.480
Row one of A dot product with
this x is producing this zero.
00:23:11.480 --> 00:23:15.970
So x is orthogonal
to the first row.
00:23:15.970 --> 00:23:17.670
And to the second row.
00:23:17.670 --> 00:23:20.630
Row two of A, x is
giving that zero.
00:23:20.630 --> 00:23:23.590
Row m of A times x
is giving that zero.
00:23:23.590 --> 00:23:25.020
So x is --
00:23:25.020 --> 00:23:27.380
the equation is
telling me that x
00:23:27.380 --> 00:23:31.309
is orthogonal to all the rows.
00:23:31.309 --> 00:23:32.600
Right, it's just sitting there.
00:23:32.600 --> 00:23:36.380
That's all we -- it had to
be sitting there because we
00:23:36.380 --> 00:23:40.000
didn't know anything more
about the null space than this.
00:23:40.000 --> 00:23:46.120
And now I guess to be
totally complete here
00:23:46.120 --> 00:23:49.350
I'd now check that
x is orthogonal
00:23:49.350 --> 00:23:51.720
to each separate row.
00:23:51.720 --> 00:23:55.710
But what else strictly
speaking do I have to do?
00:23:58.670 --> 00:24:02.340
To show that those
subspaces are orthogonal,
00:24:02.340 --> 00:24:05.580
I have to take this x in
the null space and show that
00:24:05.580 --> 00:24:10.380
it's orthogonal to every
vector in the row space,
00:24:10.380 --> 00:24:12.740
every vector in the
row space, so what --
00:24:12.740 --> 00:24:15.330
what else is in the row space?
00:24:15.330 --> 00:24:18.970
This row is in the row space,
that row is in the row space,
00:24:18.970 --> 00:24:22.870
they're all there, but it's
not the whole row space,
00:24:22.870 --> 00:24:24.990
right, we just have to
like remember, what does it
00:24:24.990 --> 00:24:29.610
mean, what does that
word space telling us?
00:24:29.610 --> 00:24:34.470
And what else is
in the row space?
00:24:34.470 --> 00:24:37.810
Besides the rows?
00:24:37.810 --> 00:24:41.990
All their combinations.
00:24:41.990 --> 00:24:44.680
So I really have to
check that sure enough
00:24:44.680 --> 00:24:46.770
if x is perpendicular
to row one,
00:24:46.770 --> 00:24:49.720
row two, all the
different separate rows,
00:24:49.720 --> 00:24:54.110
then also x is perpendicular
to a combination of the rows.
00:24:54.110 --> 00:24:57.000
And that's just matrix
multiplication again.
00:24:57.000 --> 00:25:03.300
You know, I have row
one transpose x is zero,
00:25:03.300 --> 00:25:11.000
so on, row two
transpose x is zero,
00:25:11.000 --> 00:25:16.800
so I'm entitled to multiply that
by some c1, this by some c2,
00:25:16.800 --> 00:25:20.700
I still have zeroes,
I'm entitled to add,
00:25:20.700 --> 00:25:24.760
so I have c1 row one so --
so all this when I put that
00:25:24.760 --> 00:25:33.290
together that's big parentheses
c1 row one plus c2 row two
00:25:33.290 --> 00:25:34.780
and so on.
00:25:34.780 --> 00:25:38.220
Transpose x is zero.
00:25:38.220 --> 00:25:38.720
Right?
00:25:38.720 --> 00:25:41.380
I just added the
zeroes and got zero,
00:25:41.380 --> 00:25:43.760
and I just added these
following the rule.
00:25:46.550 --> 00:25:48.490
No big deal.
00:25:48.490 --> 00:25:51.890
The whole point was
right sitting in that.
00:25:51.890 --> 00:25:54.610
OK.
00:25:54.610 --> 00:26:02.480
So if I come back to this figure
now I'm like a happier person.
00:26:02.480 --> 00:26:05.090
Because I have this --
00:26:05.090 --> 00:26:10.730
I now see how those
subspaces are oriented.
00:26:10.730 --> 00:26:14.090
And these subspaces
are also oriented.
00:26:14.090 --> 00:26:20.200
Well, actually why is
that orthogonality?
00:26:20.200 --> 00:26:23.660
Well, it's the same
statement for A transpose
00:26:23.660 --> 00:26:25.330
that that one was for A.
00:26:25.330 --> 00:26:27.670
So I won't take time
to prove it again
00:26:27.670 --> 00:26:32.500
because we've checked
it for every matrix
00:26:32.500 --> 00:26:35.960
and A transpose is just
as good a matrix as A.
00:26:35.960 --> 00:26:39.520
So we're orthogonal over there.
00:26:39.520 --> 00:26:46.180
So we really have
carved up this --
00:26:46.180 --> 00:26:50.070
this was like carving
up m-dimensional space
00:26:50.070 --> 00:26:56.070
into two subspaces and
this one was carving up
00:26:56.070 --> 00:27:01.810
n-dimensional space
into two subspaces.
00:27:01.810 --> 00:27:06.500
And well, one more thing here.
00:27:06.500 --> 00:27:07.550
One more important thing.
00:27:11.110 --> 00:27:13.330
Let me move into
three dimensions.
00:27:15.990 --> 00:27:22.780
Tell me a couple of orthogonal
subspaces in three dimensions
00:27:22.780 --> 00:27:27.450
that somehow don't carve
up the whole space,
00:27:27.450 --> 00:27:30.330
there's stuff left there.
00:27:30.330 --> 00:27:34.830
I'm thinking of a couple
of orthogonal lines.
00:27:34.830 --> 00:27:38.550
If I -- suppose I'm in
three dimensions, R^3.
00:27:38.550 --> 00:27:43.510
And I have one line, one
one-dimensional subspace,
00:27:43.510 --> 00:27:46.030
and a perpendicular one.
00:27:46.030 --> 00:27:51.170
Could those be the row
space and the null space?
00:27:51.170 --> 00:27:54.590
Could those be the row
space and the null space?
00:27:54.590 --> 00:28:00.520
Could I be in three
dimensions and have
00:28:00.520 --> 00:28:05.930
a row space that's a line and
a null space that's a line?
00:28:05.930 --> 00:28:07.270
No.
00:28:07.270 --> 00:28:10.180
Why not?
00:28:10.180 --> 00:28:11.840
Because the dimensions
aren't right.
00:28:11.840 --> 00:28:12.340
Right?
00:28:12.340 --> 00:28:14.080
The dimensions are no good.
00:28:14.080 --> 00:28:19.050
The dimensions here, r and
n-r, they add up to three,
00:28:19.050 --> 00:28:21.490
they add up to n.
00:28:21.490 --> 00:28:23.220
If I take --
00:28:23.220 --> 00:28:26.800
just follow that example --
00:28:26.800 --> 00:28:30.910
if the row space
is one-dimensional,
00:28:30.910 --> 00:28:36.030
suppose A is what's
a good in R^3,
00:28:36.030 --> 00:28:39.560
I want a one-dimensional row
space, let me take one, two,
00:28:39.560 --> 00:28:43.470
five, two, four, ten.
00:28:43.470 --> 00:28:45.330
What's the dimension
of that row space?
00:28:48.590 --> 00:28:50.220
One.
00:28:50.220 --> 00:28:52.260
What's the dimension
of the null space?
00:28:56.160 --> 00:28:59.130
Tell what's the null space
look like in that case?
00:28:59.130 --> 00:29:01.670
The row space is a line, right?
00:29:01.670 --> 00:29:06.440
One-dimensional, it's just a
line through one, two, five.
00:29:06.440 --> 00:29:08.370
Geometrically what's
the row space look like?
00:29:13.950 --> 00:29:15.660
What's its dimension?
00:29:15.660 --> 00:29:20.920
So here r here n
is three, the rank
00:29:20.920 --> 00:29:26.610
is one, so the dimension
of the null space,
00:29:26.610 --> 00:29:32.180
so I'm looking at
this x, x1, x2, x3.
00:29:32.180 --> 00:29:33.680
To give zero.
00:29:33.680 --> 00:29:43.300
So the dimension of the null
space is we all know is two.
00:29:43.300 --> 00:29:43.800
Right.
00:29:43.800 --> 00:29:45.440
It's a plane.
00:29:45.440 --> 00:29:49.770
And now actually we know, we
see better, what plane is it?
00:29:49.770 --> 00:29:52.660
What plane is it?
00:29:52.660 --> 00:29:57.500
It's the plane that's
perpendicular to one,
00:29:57.500 --> 00:29:59.950
two, five.
00:29:59.950 --> 00:30:00.450
Right?
00:30:00.450 --> 00:30:01.350
We now see.
00:30:01.350 --> 00:30:04.710
In fact the two, four,
ten didn't actually
00:30:04.710 --> 00:30:07.000
have any effect at all.
00:30:07.000 --> 00:30:09.750
I could have just ignored that.
00:30:09.750 --> 00:30:14.340
That didn't change the row
space or the null space.
00:30:14.340 --> 00:30:17.051
I'll just make
that one equation.
00:30:17.051 --> 00:30:17.550
Yeah.
00:30:17.550 --> 00:30:18.000
OK.
00:30:18.000 --> 00:30:18.499
Sure.
00:30:18.499 --> 00:30:21.210
That's the easiest to deal with.
00:30:21.210 --> 00:30:22.130
One equation.
00:30:22.130 --> 00:30:24.440
Three unknowns.
00:30:24.440 --> 00:30:32.670
And I want to ask --
00:30:32.670 --> 00:30:37.180
what would the equation
give me the null space,
00:30:37.180 --> 00:30:41.380
and you would have
said back in September
00:30:41.380 --> 00:30:43.410
you would have said
it gives you a plane,
00:30:43.410 --> 00:30:46.530
and we're completely right.
00:30:46.530 --> 00:30:49.530
And the plane it gives
you, the normal vector,
00:30:49.530 --> 00:30:53.610
you remember in calculus, there
was this dumb normal vector
00:30:53.610 --> 00:30:54.760
called N.
00:30:54.760 --> 00:30:55.496
Well there it is.
00:30:55.496 --> 00:30:56.120
One, two, five.
00:30:56.120 --> 00:30:56.620
OK.
00:30:56.620 --> 00:31:08.820
What is the what's the
point I want to make here?
00:31:08.820 --> 00:31:10.010
I want to make --
00:31:10.010 --> 00:31:14.300
I want to emphasize
that not only are the --
00:31:14.300 --> 00:31:15.410
let me write it in words.
00:31:19.930 --> 00:31:33.900
So I want to write the null
space and the row space are
00:31:33.900 --> 00:31:40.290
orthogonal, that's this
neat fact, which we've --
00:31:40.290 --> 00:31:43.240
we've just checked
from Ax equals zero,
00:31:43.240 --> 00:31:48.750
but now I want to say more
because there's a little more
00:31:48.750 --> 00:31:51.190
that's true.
00:31:51.190 --> 00:31:54.940
Their dimensions add
to the whole space.
00:31:54.940 --> 00:31:57.660
So that's like a little
extra information.
00:31:57.660 --> 00:31:59.860
That it's not like
I could have --
00:31:59.860 --> 00:32:03.030
I couldn't have a line and
a line in three dimensions.
00:32:03.030 --> 00:32:07.130
Those don't add up one and
one don't add to three.
00:32:07.130 --> 00:32:17.820
So I used the word orthogonal
complements in R^n.
00:32:17.820 --> 00:32:20.000
And the idea of
this word complement
00:32:20.000 --> 00:32:28.720
is that the orthogonal
complement of a row space
00:32:28.720 --> 00:32:32.950
contains not just some vectors
that are orthogonal to it,
00:32:32.950 --> 00:32:34.400
but all.
00:32:34.400 --> 00:32:36.050
So what does that mean?
00:32:36.050 --> 00:32:42.260
That means that the null
space contains all, not just
00:32:42.260 --> 00:32:51.270
some but all, vectors that are
perpendicular to the row space.
00:32:51.270 --> 00:32:52.260
OK.
00:32:52.260 --> 00:33:04.190
Really what I've done in this
half of the lecture is just
00:33:04.190 --> 00:33:09.100
notice some of the
nice geometry that --
00:33:09.100 --> 00:33:12.030
that we didn't pick up before
because we didn't discuss
00:33:12.030 --> 00:33:15.010
perpendicular vectors before.
00:33:15.010 --> 00:33:16.800
But it was all sitting there.
00:33:16.800 --> 00:33:18.440
And now we picked it up.
00:33:18.440 --> 00:33:21.370
That these vectors are
orthogonal complements.
00:33:21.370 --> 00:33:23.740
And I guess I even
call this part
00:33:23.740 --> 00:33:26.730
one of the fundamental
theorem of linear algebra.
00:33:26.730 --> 00:33:32.330
The fundamental theorem
of linear algebra
00:33:32.330 --> 00:33:37.230
is about these four
subspaces, so part one
00:33:37.230 --> 00:33:41.660
is about their dimension, maybe
I should call it part two now.
00:33:41.660 --> 00:33:44.600
Their dimensions we got.
00:33:44.600 --> 00:33:49.010
Now we're getting their
orthogonality, that's part two.
00:33:49.010 --> 00:33:54.640
And part three will be
about bases for them.
00:33:54.640 --> 00:33:57.290
Orthogonal bases.
00:33:57.290 --> 00:34:00.520
So that's coming up.
00:34:00.520 --> 00:34:01.570
OK.
00:34:01.570 --> 00:34:10.870
So I'm happy with that
geometry right now.
00:34:10.870 --> 00:34:11.980
OK.
00:34:11.980 --> 00:34:12.949
OK.
00:34:12.949 --> 00:34:16.239
Now what's my next
goal in this chapter?
00:34:16.239 --> 00:34:19.087
Here's the main
problem of the chapter.
00:34:19.087 --> 00:34:21.420
The main problem of the chapter
is -- so this is coming.
00:34:21.420 --> 00:34:22.378
It's coming attraction.
00:34:22.378 --> 00:34:37.160
This is the very last
chapter that's about Ax=b.
00:34:44.030 --> 00:34:48.690
I would like to solve
that system of equations
00:34:48.690 --> 00:34:50.710
when there is no solution.
00:34:54.560 --> 00:34:56.880
You may say what a
ridiculous thing to do.
00:34:56.880 --> 00:35:00.670
But I have to say it's
done all the time.
00:35:00.670 --> 00:35:02.440
In fact it has to be done.
00:35:02.440 --> 00:35:07.510
You get -- so the
problem is solve --
00:35:07.510 --> 00:35:20.626
the best possible solve I'll
put quote Ax=b when there is no
00:35:20.626 --> 00:35:21.125
solution.
00:35:24.470 --> 00:35:25.930
And of course what
does that mean?
00:35:25.930 --> 00:35:29.550
b isn't in the column space.
00:35:29.550 --> 00:35:35.720
And it's quite typical if this
matrix A is rectangular if I --
00:35:35.720 --> 00:35:40.370
maybe I have m equations and
that's bigger than the number
00:35:40.370 --> 00:35:47.840
of unknowns, then for
sure the rank is not m,
00:35:47.840 --> 00:35:51.910
the rank couldn't be m now, so
there'll be a lot of right-hand
00:35:51.910 --> 00:36:00.300
sides with no solution,
but here's an example.
00:36:00.300 --> 00:36:02.760
Some satellite is buzzing along.
00:36:02.760 --> 00:36:06.196
You measure its position.
00:36:06.196 --> 00:36:07.570
You make a thousand
measurements.
00:36:10.190 --> 00:36:13.940
So that gives you a thousand
equations for the --
00:36:13.940 --> 00:36:18.340
for the parameters that
-- that give the position.
00:36:18.340 --> 00:36:20.350
But there aren't a
thousand parameters,
00:36:20.350 --> 00:36:23.310
there's just maybe
six or something.
00:36:23.310 --> 00:36:26.930
Or you're measuring the --
you're doing questionnaires.
00:36:29.690 --> 00:36:36.210
You're measuring resistances.
00:36:36.210 --> 00:36:37.650
You're taking pulses.
00:36:37.650 --> 00:36:41.120
You're measuring
somebody's pulse rate.
00:36:41.120 --> 00:36:42.190
There's just one unknown.
00:36:42.190 --> 00:36:42.690
OK.
00:36:42.690 --> 00:36:44.890
The pulse rate.
00:36:44.890 --> 00:36:46.950
So you measure it
once, OK, fine,
00:36:46.950 --> 00:36:49.510
but if you really
want to know it,
00:36:49.510 --> 00:36:54.800
you measure it multiple times,
but then the measurements have
00:36:54.800 --> 00:36:57.120
noise in them, so there's --
00:36:57.120 --> 00:36:59.820
the problem is that
in many many problems
00:36:59.820 --> 00:37:03.740
we've got too many
equations and they've got
00:37:03.740 --> 00:37:05.240
noise in the right-hand side.
00:37:05.240 --> 00:37:11.780
So Ax=b I can't expect to
solve it exactly right,
00:37:11.780 --> 00:37:13.230
because I don't
even know what --
00:37:13.230 --> 00:37:18.680
there's a measurement
mistake in b.
00:37:18.680 --> 00:37:21.790
But there's information too.
00:37:21.790 --> 00:37:25.120
There's a lot of information
about x in there.
00:37:25.120 --> 00:37:30.070
And what I want to do is like
separate the noise, the junk,
00:37:30.070 --> 00:37:33.860
from the information.
00:37:33.860 --> 00:37:38.680
And so this is a straightforward
linear algebra problem.
00:37:38.680 --> 00:37:41.600
How do I solve, what's
the best solution?
00:37:41.600 --> 00:37:43.600
OK.
00:37:43.600 --> 00:37:45.070
Now.
00:37:45.070 --> 00:37:52.190
I want to say so that's
like describes the problem
00:37:52.190 --> 00:37:55.010
in an algebraic way.
00:37:55.010 --> 00:37:58.820
I got some equations, I'm
looking for the best solution.
00:37:58.820 --> 00:38:02.020
Well, one way to find it
is -- one way to start,
00:38:02.020 --> 00:38:09.580
one way to find a solution
is throw away equations until
00:38:09.580 --> 00:38:12.340
you've got a nice, square
invertible system and solve
00:38:12.340 --> 00:38:14.760
that.
00:38:14.760 --> 00:38:18.620
That's not satisfactory.
00:38:18.620 --> 00:38:21.410
There's no reason in
these measurements
00:38:21.410 --> 00:38:23.300
to say these
measurements are perfect
00:38:23.300 --> 00:38:25.970
and these measurements
are useless.
00:38:25.970 --> 00:38:27.990
We want to use all
the measurements
00:38:27.990 --> 00:38:32.140
to get the best information,
to get the maximum information.
00:38:32.140 --> 00:38:33.170
But how?
00:38:33.170 --> 00:38:34.390
OK.
00:38:34.390 --> 00:38:40.020
Let me anticipate a matrix
that's going to show up.
00:38:40.020 --> 00:38:43.720
This A is typically rectangular.
00:38:43.720 --> 00:38:47.390
But a matrix that shows
up whenever you have --
00:38:47.390 --> 00:38:52.260
and we chapter three was all
about rectangular matrices.
00:38:52.260 --> 00:38:54.840
And we know when
this is solvable,
00:38:54.840 --> 00:38:58.020
you could do elimination
on it, right?
00:38:58.020 --> 00:39:00.600
But I'm thinking hey,
you do elimination
00:39:00.600 --> 00:39:03.299
and you get equation zero
equal other non-zeroes.
00:39:03.299 --> 00:39:05.590
I'm thinking we really --
elimination is going to fail.
00:39:05.590 --> 00:39:05.630
So that's our question.
00:39:05.630 --> 00:39:07.046
Elimination will
get us down to --
00:39:16.320 --> 00:39:18.700
will tell us if there
is a solution or not.
00:39:18.700 --> 00:39:22.600
But I'm now thinking not.
00:39:22.600 --> 00:39:24.130
So what are we going to do?
00:39:24.130 --> 00:39:24.720
OK.
00:39:24.720 --> 00:39:25.320
All right.
00:39:25.320 --> 00:39:30.810
I want to tell you to jump
ahead to the matrix that
00:39:30.810 --> 00:39:32.310
will play a key role.
00:39:32.310 --> 00:39:35.780
So this is the matrix that
you want to understand
00:39:35.780 --> 00:39:38.950
for this chapter four.
00:39:38.950 --> 00:39:47.740
And it's the matrix
A transpose A.
00:39:47.740 --> 00:39:53.130
What's -- tell me some
things about that matrix.
00:39:53.130 --> 00:39:58.640
So A is this m by n matrix,
rectangular, but now
00:39:58.640 --> 00:40:03.100
I'm saying that the good
matrix that shows up in the end
00:40:03.100 --> 00:40:05.130
is A transpose A.
00:40:05.130 --> 00:40:09.120
So tell me something about that.
00:40:09.120 --> 00:40:14.010
Is it -- what's the first thing
you know about A transpose A.
00:40:14.010 --> 00:40:15.390
It's square.
00:40:15.390 --> 00:40:16.780
Right?
00:40:16.780 --> 00:40:21.740
Square because this is m
by n and this is n by m.
00:40:21.740 --> 00:40:24.090
So this is the result is n by n.
00:40:24.090 --> 00:40:25.240
Good.
00:40:25.240 --> 00:40:26.230
Square.
00:40:26.230 --> 00:40:27.990
What else?
00:40:27.990 --> 00:40:28.970
It's symmetric.
00:40:28.970 --> 00:40:29.535
Good.
00:40:29.535 --> 00:40:30.160
It's symmetric.
00:40:35.920 --> 00:40:39.650
Because you remember
how to do that.
00:40:39.650 --> 00:40:44.270
If we transpose that
matrix let's transpose it,
00:40:44.270 --> 00:40:48.510
A transpose A, if
I transpose it,
00:40:48.510 --> 00:40:55.230
then that comes first
transposed, this comes second,
00:40:55.230 --> 00:41:01.990
transposed, and then transposing
twice is leaves it --
00:41:01.990 --> 00:41:05.100
brings it back to the
same so it's symmetric.
00:41:05.100 --> 00:41:06.620
Good.
00:41:06.620 --> 00:41:11.340
Now we now know how to
ask more about a matrix.
00:41:14.050 --> 00:41:20.110
I'm interested in
is it invertible?
00:41:20.110 --> 00:41:23.810
If not, what's its null space?
00:41:23.810 --> 00:41:26.940
So I want to know about --
because you're going to see,
00:41:26.940 --> 00:41:32.260
well, let me -- let me even,
well I shouldn't do this,
00:41:32.260 --> 00:41:33.350
but I will.
00:41:33.350 --> 00:41:37.460
Let me tell you what
equation to solve
00:41:37.460 --> 00:41:42.300
when you can't solve that one.
00:41:42.300 --> 00:41:47.450
The good equation comes
from multiplying both sides
00:41:47.450 --> 00:41:53.150
by A transpose, so the good
equation that you get to
00:41:53.150 --> 00:41:54.820
is this one.
00:41:54.820 --> 00:42:00.790
A transpose Ax
equals A transpose b.
00:42:04.960 --> 00:42:08.700
That will be the central
equation in the chapter.
00:42:08.700 --> 00:42:10.950
So I think why not
tell it to you.
00:42:10.950 --> 00:42:13.290
Why not admit it right away.
00:42:13.290 --> 00:42:14.440
OK.
00:42:14.440 --> 00:42:15.260
I have to --
00:42:15.260 --> 00:42:17.960
I should really give x.
00:42:17.960 --> 00:42:27.750
I want to sort of indicate that
this x isn't I mean this x was
00:42:27.750 --> 00:42:30.580
the solution to that
equation if it existed,
00:42:30.580 --> 00:42:33.440
but probably didn't.
00:42:33.440 --> 00:42:39.020
Now let me give this a
different name, x hat.
00:42:39.020 --> 00:42:45.740
Because I'm hoping this
one will have a solution.
00:42:45.740 --> 00:42:49.680
And I'm saying that
it's my best solution.
00:42:49.680 --> 00:42:52.400
I'll have to say
what does best mean.
00:42:52.400 --> 00:42:55.720
But that's going to
be my -- my plan.
00:42:55.720 --> 00:42:59.650
I'm going to say that the best
solution solves this equation.
00:42:59.650 --> 00:43:03.770
So you see right away why I'm
so interested in this matrix
00:43:03.770 --> 00:43:05.620
A transpose A.
00:43:05.620 --> 00:43:07.130
And in its invertibility.
00:43:07.130 --> 00:43:07.630
OK.
00:43:10.210 --> 00:43:11.720
Now, when is it invertible?
00:43:14.290 --> 00:43:14.900
OK.
00:43:14.900 --> 00:43:22.040
Let me take a case, let me
just do an example and then --
00:43:22.040 --> 00:43:25.910
I'll just pick a matrix here.
00:43:25.910 --> 00:43:28.510
Just so we see what A
transpose A looks like.
00:43:28.510 --> 00:43:35.010
So let me take a matrix A
one, one, one, one, two, five.
00:43:35.010 --> 00:43:37.190
Just to invent a matrix.
00:43:37.190 --> 00:43:40.600
So there's a matrix A.
00:43:40.600 --> 00:43:48.390
Notice that it has M equal three
rows and N equal two columns.
00:43:48.390 --> 00:43:50.950
Its rank is --
00:43:50.950 --> 00:43:55.200
the rank of that matrix is two.
00:43:55.200 --> 00:43:58.460
Right, yeah, the
columns are independent.
00:43:58.460 --> 00:44:01.200
Does Ax equal b?
00:44:01.200 --> 00:44:09.086
If I look at Ax=b, so x is
just x1 x2, and b is b1 b2 b3.
00:44:12.700 --> 00:44:15.300
Do I expect to solve Ax=b?
00:44:15.300 --> 00:44:17.760
What's -- no way, right?
00:44:17.760 --> 00:44:21.440
I mean linear algebra's
great, but solving
00:44:21.440 --> 00:44:24.040
three equations with
only two unknowns usually
00:44:24.040 --> 00:44:26.020
we can't do it.
00:44:26.020 --> 00:44:30.050
We can only solve it if
this vector is b is what?
00:44:33.230 --> 00:44:37.780
I can solve that equation
if that vector b1 b2 b3
00:44:37.780 --> 00:44:41.700
is in the column space.
00:44:41.700 --> 00:44:44.650
If it's a combination of
those columns then fine.
00:44:44.650 --> 00:44:47.030
But usually it won't be.
00:44:47.030 --> 00:44:49.580
The combinations
just fill up a plane
00:44:49.580 --> 00:44:52.780
and most vectors
aren't on that plane.
00:44:52.780 --> 00:44:56.630
So what I'm saying is
that I'm going to work
00:44:56.630 --> 00:44:59.990
with the matrix A transpose A.
00:44:59.990 --> 00:45:03.600
And I just want to figure
out in this example what
00:45:03.600 --> 00:45:07.780
A transpose A is.
00:45:07.780 --> 00:45:09.390
So it's two by two.
00:45:09.390 --> 00:45:13.320
The first entry is a three,
the next entry is an eight,
00:45:13.320 --> 00:45:15.160
this entry is --
00:45:18.040 --> 00:45:20.910
what's that entry?
00:45:20.910 --> 00:45:23.110
Eight, for sure.
00:45:23.110 --> 00:45:25.270
We knew it had to
be, and this entry
00:45:25.270 --> 00:45:32.630
is, what's that now, getting out
my trusty calculator, thirty,
00:45:32.630 --> 00:45:35.880
is that right?
00:45:35.880 --> 00:45:37.580
And is that matrix invertible?
00:45:37.580 --> 00:45:38.080
Thirty.
00:45:38.080 --> 00:45:40.960
There's an A transpose A.
00:45:40.960 --> 00:45:42.620
And it is invertible, right?
00:45:42.620 --> 00:45:46.040
Three, eight is not a
multiple of eight, thirty --
00:45:46.040 --> 00:45:48.050
and it's invertible.
00:45:48.050 --> 00:45:51.420
And that's the normal,
that's what I expect.
00:45:51.420 --> 00:45:56.730
So this is I want to show.
00:45:56.730 --> 00:45:59.740
So here's the final --
here's the key point.
00:45:59.740 --> 00:46:04.660
The null space of
A transpose A --
00:46:04.660 --> 00:46:06.645
it's not going to be
always invertible.
00:46:09.650 --> 00:46:11.450
Tell me a matrix --
00:46:11.450 --> 00:46:15.640
I have to say that I can't
say A transpose A is always
00:46:15.640 --> 00:46:16.770
invertible.
00:46:16.770 --> 00:46:19.590
Because that's asking too much.
00:46:19.590 --> 00:46:22.860
I mean what could the
matrix A be, for example,
00:46:22.860 --> 00:46:27.330
so that A transpose
A was not invertible?
00:46:27.330 --> 00:46:29.110
Well, it even could
be the zero matrix.
00:46:29.110 --> 00:46:32.250
I mean that's like extreme case.
00:46:32.250 --> 00:46:38.960
Suppose I make this rank --
00:46:38.960 --> 00:46:46.090
suppose I change to that A.
00:46:46.090 --> 00:46:49.240
Now I figure out A transpose
A again and I get --
00:46:49.240 --> 00:46:49.840
what do I get?
00:46:53.890 --> 00:46:58.360
I get nine, I get nine
of course and here I
00:46:58.360 --> 00:47:02.434
get what's that entry?
00:47:02.434 --> 00:47:02.975
Twenty-seven.
00:47:06.900 --> 00:47:09.100
And is that matrix invertible?
00:47:09.100 --> 00:47:09.600
No.
00:47:12.110 --> 00:47:13.470
And why do I --
00:47:13.470 --> 00:47:16.590
I knew it wouldn't
be invertible anyway.
00:47:16.590 --> 00:47:23.190
Because this matrix
only has rank one.
00:47:23.190 --> 00:47:26.100
And if I have a product
of matrices of rank one,
00:47:26.100 --> 00:47:30.340
the product is not going to
have a rank bigger than one.
00:47:30.340 --> 00:47:33.810
So I'm not surprised that
the answer only has rank one.
00:47:33.810 --> 00:47:36.730
And that's what I --
00:47:36.730 --> 00:47:41.200
always happens, that the
rank of A transpose A
00:47:41.200 --> 00:47:44.460
comes out to equal
the rank of A.
00:47:44.460 --> 00:47:49.580
So, yes, so the null
space of A transpose A
00:47:49.580 --> 00:47:57.580
equals the null space of A,
the rank of A transpose A
00:47:57.580 --> 00:48:01.840
equals the rank of A.
00:48:01.840 --> 00:48:11.460
So let's -- as soon as
I can why that's true.
00:48:11.460 --> 00:48:18.920
But let's draw from that
what the fact that I want.
00:48:18.920 --> 00:48:24.810
This tells me that this square
symmetric matrix is invertible
00:48:24.810 --> 00:48:26.650
if --
00:48:26.650 --> 00:48:30.460
so here's my conclusion.
00:48:30.460 --> 00:48:40.400
A transpose A is invertible
if exactly when --
00:48:40.400 --> 00:48:46.390
exactly if this null space
is only got the zero vector.
00:48:46.390 --> 00:48:51.700
Which means the columns
of A are independent.
00:48:51.700 --> 00:48:54.342
So A transpose A is
invertible exactly
00:48:54.342 --> 00:48:55.550
if A has independent columns.
00:48:55.550 --> 00:49:12.290
That's the fact that I
need about A transpose A.
00:49:12.290 --> 00:49:17.160
And then you'll see next
time how A transpose A
00:49:17.160 --> 00:49:18.780
enters everything.
00:49:18.780 --> 00:49:22.130
Next lecture is
actually a crucial one.
00:49:22.130 --> 00:49:26.280
Here I'm preparing
for it by getting us
00:49:26.280 --> 00:49:29.000
thinking about A transpose A.
00:49:29.000 --> 00:49:31.970
And its rank is the
same as the rank of A,
00:49:31.970 --> 00:49:34.310
and we can decide
when it's invertible.
00:49:34.310 --> 00:49:34.980
OK.
00:49:34.980 --> 00:49:35.938
So I'll see you Friday.
00:49:35.938 --> 00:49:37.140
Thanks.