1
00:00:12.79 --> 00:00:15
OK.
cameras are rolling.
2
00:00:15 --> 00:00:20
This is lecture fourteen,
starting a new chapter.
3
00:00:20 --> 00:00:24
Chapter about orthogonality.
4
00:00:24 --> 00:00:29.01
What it means for vectors to be
orthogonal.
5
00:00:29.01 --> 00:00:34
What it means for subspaces to
be orthogonal.
6
00:00:34 --> 00:00:39
What it means for bases to be
orthogonal.
7
00:00:39 --> 00:00:45
So ninety degrees,
this is a ninety-degree
8
00:00:45 --> 00:00:46
chapter.
9
00:00:46 --> 00:00:52
So what does it mean -- let me
jump to subspaces.
10
00:00:52 --> 00:00:58
Because I've drawn here the big
picture.
11
00:00:58 --> 00:01:02.32
This is the 18.06 picture here.
12
00:01:02.32 --> 00:01:06
And hold it down,
guys.
13
00:01:06 --> 00:01:13
So this is the picture and we
know a lot about that picture
14
00:01:13 --> 00:01:14
already.
15
00:01:14 --> 00:01:19
We know the dimension of every
subspace.
16
00:01:19 --> 00:01:24
We know that these dimensions
are r and n-r.
17
00:01:24 --> 00:01:30
We know that these dimensions
are r and m-r.
18
00:01:30 --> 00:01:36
What I want to show now is what
this figure is saying,
19
00:01:36 --> 00:01:42
that the angle --
the figure is just my attempt
20
00:01:42 --> 00:01:47
to draw what I'm now going to
say, that the angle between
21
00:01:47 --> 00:01:50
these subspaces is ninety
degrees.
22
00:01:50 --> 00:01:54
And the angle between these
subspaces is ninety degrees.
23
00:01:54 --> 00:01:58
Now I have to say what does
that mean?
24
00:01:58 --> 00:02:03
What does it mean for subspaces
to be orthogonal?
25
00:02:03 --> 00:02:07
But I hope you appreciate the
beauty of this picture,
26
00:02:07 --> 00:02:12
that that those subspaces are
going to come out to be
27
00:02:12 --> 00:02:13
orthogonal.
28
00:02:13 --> 00:02:16
Those two and also those two.
29
00:02:16 --> 00:02:21
So that's like one point,
one important point to step
30
00:02:21 --> 00:02:24
forward in understanding those
subspaces.
31
00:02:24 --> 00:02:30
We knew what each subspace was
like, we could compute bases for
32
00:02:30 --> 00:02:31.98
them.
33
00:02:31.98 --> 00:02:34.17
Now we know more.
34
00:02:34.17 --> 00:02:37
Or we will in a few minutes.
35
00:02:37 --> 00:02:38
OK.
36
00:02:38 --> 00:02:46
I have to say first of all what
does it mean for two vectors to
37
00:02:46 --> 00:02:47
be orthogonal?
38
00:02:47 --> 00:02:51
So let me start with that.
39
00:02:51 --> 00:02:53
Orthogonal vectors.
40
00:02:53 --> 00:03:00
The word orthogonal is -- is
just another word for
41
00:03:00 --> 00:03:03
perpendicular.
42
00:03:03 --> 00:03:10
It means that in n-dimensional
space the angle between those
43
00:03:10 --> 00:03:12.97
vectors is ninety degrees.
44
00:03:12.97 --> 00:03:17
It means that they form a right
triangle.
45
00:03:17 --> 00:03:24
It even means that the going
way back to the Greeks that this
46
00:03:24 --> 00:03:29
angle that this triangle a
vector x, a vector x,
47
00:03:29 --> 00:03:35
and a vector x+y --
of course that'll be the
48
00:03:35 --> 00:03:42
hypotenuse, so what was it the
Greeks figured out and it's
49
00:03:42 --> 00:03:43
neat.
50
00:03:43 --> 00:03:48
It's the fact that the -- so
these are orthogonal,
51
00:03:48 --> 00:03:54
this is a right angle,
if -- so let me put the great
52
00:03:54 --> 00:04:00.08
name down, Pythagoras,
I'm looking for -- what I
53
00:04:00.08 --> 00:04:02
looking for?
54
00:04:02 --> 00:04:07
I'm looking for the condition
if you give me two vectors,
55
00:04:07 --> 00:04:10
how do I know if they're
orthogonal?
56
00:04:10 --> 00:04:13
How can I tell two
perpendicular vectors?
57
00:04:13 --> 00:04:17
And actually you probably know
the answer.
58
00:04:17 --> 00:04:19
Let me write the answer down.
59
00:04:19 --> 00:04:23
Orthogonal vectors,
what's the test for
60
00:04:23 --> 00:04:25
orthogonality?
61
00:04:25 --> 00:04:32
I take the dot product which I
tend to write as x transpose y,
62
00:04:32 --> 00:04:38
because that's a row times a
column, and that matrix
63
00:04:38 --> 00:04:45
multiplication just gives me the
right thing, that x1y1+x2y2 and
64
00:04:45 --> 00:04:51
so on, so these vectors are
orthogonal if this result x
65
00:04:51 --> 00:04:54
transpose y is zero.
66
00:04:54 --> 00:04:55
That's the test.
67
00:04:55 --> 00:04:56
OK.
68
00:04:56 --> 00:05:01
Can I connect that to other
things?
69
00:05:01 --> 00:05:06
I mean -- it's just beautiful
that here we have we're in n
70
00:05:06 --> 00:05:11
dimensions, we've got a couple
of vectors, we want to know the
71
00:05:11 --> 00:05:15.43
angle between them,
and the right thing to look at
72
00:05:15.43 --> 00:05:19.05
is the simplest thing that you
could imagine,
73
00:05:19.05 --> 00:05:20
the dot product.
74
00:05:20 --> 00:05:20
OK.
75
00:05:20 --> 00:05:21
Now why?
76
00:05:21 --> 00:05:25.55
So I'm answering the question
now why --
77
00:05:25.55 --> 00:05:31
let's add some justification to
this fact, that's the test.
78
00:05:31 --> 00:05:36
OK, so Pythagoras would say
we've got a right triangle,
79
00:05:36 --> 00:05:42
if that length squared plus
that length squared equals that
80
00:05:42 --> 00:05:44
length squared.
81
00:05:44 --> 00:05:50
OK, can I write it as x squared
plus y squared equals x plus y
82
00:05:50 --> 00:05:51
squared?
83
00:05:51 --> 00:05:56
Now don't, please don't think
that this is always true.
84
00:05:56 --> 00:06:01
This is only going to be true
in this -- it's going to be
85
00:06:01 --> 00:06:04
equivalent to orthogonality.
86
00:06:04 --> 00:06:08
For other triangles of course,
it's not true.
87
00:06:08 --> 00:06:11
For other triangles it's not.
88
00:06:11 --> 00:06:16
But for a right triangle
somehow that fact should connect
89
00:06:16 --> 00:06:18
to that fact.
90
00:06:18 --> 00:06:21
Can we just make that
connection?
91
00:06:21 --> 00:06:26
What's the connection between
this test for orthogonality and
92
00:06:26 --> 00:06:29
this statement of orthogonality?
93
00:06:29 --> 00:06:34
Well, I guess I have to say
what is the length squared?
94
00:06:34 --> 00:06:39
So let's continue on the board
underneath with that equation.
95
00:06:39 --> 00:06:46
Give me another way to express
the length squared of a vector.
96
00:06:46 --> 00:06:49
And let me just give you a
vector.
97
00:06:49 --> 00:06:51
The vector one,
two, three.
98
00:06:51 --> 00:06:53.81
That's in three dimensions.
99
00:06:53.81 --> 00:06:58
What is the length squared of
the vector x equal one,
100
00:06:58 --> 00:06:59
two, three?
101
00:06:59 --> 00:07:02
So how do you find the length
squared?
102
00:07:02 --> 00:07:07
Well, really you just,
you want the length of that
103
00:07:07 --> 00:07:14
vector that goes along one --
up two, and out three -- and
104
00:07:14 --> 00:07:19
we'll come back to that right
triangle stuff.
105
00:07:19 --> 00:07:24
The length squared is exactly x
transpose x.
106
00:07:24 --> 00:07:30
Whenever I see x transpose x,
I know I've got a number that's
107
00:07:30 --> 00:07:31.83
positive.
108
00:07:31.83 --> 00:07:36
It's a length squared unless it
--
109
00:07:36 --> 00:07:41
unless x happens to be the zero
vector, that's the one case
110
00:07:41 --> 00:07:44.07
where the length is zero.
111
00:07:44.07 --> 00:07:49
So right -- this is just x1
squared plus x2 squared plus so
112
00:07:49 --> 00:07:51
on, plus xn squared.
113
00:07:51 --> 00:07:56.52
So one -- in the example I gave
you what was the length squared
114
00:07:56.52 --> 00:08:00
of that vector one,
two, three?
115
00:08:00 --> 00:08:02.89
So you square those,
you get one,
116
00:08:02.89 --> 00:08:06
four and nine,
you add, you get fourteen.
117
00:08:06 --> 00:08:10
So the vector one,
two, three has length fourteen.
118
00:08:10 --> 00:08:13
So let me just put down a
vector here.
119
00:08:13 --> 00:08:17
Let x be the vector one,
two, three.
120
00:08:17 --> 00:08:22
Let me cook up a vector that's
orthogonal to it.
121
00:08:22 --> 00:08:27
So what's the vector that's
orthogonal?
122
00:08:27 --> 00:08:35.45
So right down here,
x squared is one plus four plus
123
00:08:35.45 --> 00:08:42
nine, fourteen,
let me cook up a vector that's
124
00:08:42 --> 00:08:50
orthogonal to it,
we'll get right that that --
125
00:08:50 --> 00:08:56
those two vectors are
orthogonal, the length of y
126
00:08:56 --> 00:09:01
squared is five,
and x plus y is one and two
127
00:09:01 --> 00:09:06
making three,
two and minus one making one,
128
00:09:06 --> 00:09:13
three and zero making three,
and the length of this squared
129
00:09:13 --> 00:09:18
is nine plus one plus nine,
nineteen.
130
00:09:18 --> 00:09:24
And sure enough,
I haven't proved anything.
131
00:09:24 --> 00:09:29
I've just like checked to see
that my test x transpose y
132
00:09:29 --> 00:09:32
equals zero, which is true,
right?
133
00:09:32 --> 00:09:37
Everybody sees that x transpose
y is zero here?
134
00:09:37 --> 00:09:39
That's maybe the main point.
135
00:09:39 --> 00:09:44
That you should get really
quick at doing x transpose y,
136
00:09:44 --> 00:09:50
so it's just this plus this
plus this and that's zero.
137
00:09:50 --> 00:09:54
And sure enough,
that clicks with fourteen plus
138
00:09:54 --> 00:09:57
five agreeing with nineteen.
139
00:09:57 --> 00:10:00.5
Now let me just do that in
letters.
140
00:10:00.5 --> 00:10:02
So that's y transpose y.
141
00:10:02 --> 00:10:06
And this is x plus y transpose
x plus y.
142
00:10:06 --> 00:10:06
OK.
143
00:10:06 --> 00:10:10
So I'm looking,
again, this isn't always true.
144
00:10:10 --> 00:10:11
I repeat.
145
00:10:11 --> 00:10:17
This is going to be true when
we have a right angle.
146
00:10:17 --> 00:10:22
And let's just -- well,
of course, I'm just going to
147
00:10:22 --> 00:10:24
simplify this stuff here.
148
00:10:24 --> 00:10:27
There's an x transpose x there.
149
00:10:27 --> 00:10:30
And there's a y transpose y
there.
150
00:10:30 --> 00:10:32
And there's an x transpose y.
151
00:10:32 --> 00:10:35.49
And there's a y transpose x.
152
00:10:35.49 --> 00:10:40.31
I knew I could do that
simplification because I'm just
153
00:10:40.31 --> 00:10:46
doing matrix multiplication and
I've just followed the rules.
154
00:10:46 --> 00:10:47
OK.
155
00:10:47 --> 00:10:50.2
So x transpose x-s cancel.
156
00:10:50.2 --> 00:10:52
Y transpose y-s cancel.
157
00:10:52 --> 00:10:55
And what about these guys?
158
00:10:55 --> 00:11:02.65
What can you tell me about the
inner product of x with y and
159
00:11:02.65 --> 00:11:06
the inner product of y with x?
160
00:11:06 --> 00:11:08
Is there a difference?
161
00:11:08 --> 00:11:14
I think if we --
while we're doing real vectors,
162
00:11:14 --> 00:11:18
which is all we're doing now,
there isn't a difference,
163
00:11:18 --> 00:11:20
there's no difference.
164
00:11:20 --> 00:11:24.42
If I take x transpose y,
that'll give me zero,
165
00:11:24.42 --> 00:11:29.68
if I took y transpose x I would
have the same x1y1 and x2y2 and
166
00:11:29.68 --> 00:11:33
x3y3, it would be the same,
so this is --
167
00:11:33 --> 00:11:39
this is the same as that,
this is really I'll knock that
168
00:11:39 --> 00:11:42
guy out and say two of these.
169
00:11:42 --> 00:11:48
So actually that's the -- this
equation boiled down to this
170
00:11:48 --> 00:11:50
thing being zero.
171
00:11:50 --> 00:11:50
Right?
172
00:11:50 --> 00:11:56.67
Everything else canceled and
this equation boiled down to
173
00:11:56.67 --> 00:11:58
that one.
174
00:11:58 --> 00:12:01
So that's really all I wanted.
175
00:12:01 --> 00:12:07
I just wanted to check that
Pythagoras for a right triangle
176
00:12:07 --> 00:12:11.85
led me to this -- of course I
cancel the two now.
177
00:12:11.85 --> 00:12:12.92
No problem.
178
00:12:12.92 --> 00:12:16
To x transpose y equals zero as
the test.
179
00:12:16 --> 00:12:17
Fair enough.
180
00:12:17 --> 00:12:18
OK.
181
00:12:18 --> 00:12:20
You knew it was coming.
182
00:12:20 --> 00:12:26
The dot product of orthogonal
vectors is zero.
183
00:12:26 --> 00:12:30
It's just -- I just want to say
that's really neat.
184
00:12:30 --> 00:12:33.43
That it comes out so well.
185
00:12:33.43 --> 00:12:34
All right.
186
00:12:34 --> 00:12:40
Now what about -- so now I know
if two -- when two vectors are
187
00:12:40 --> 00:12:41.29
orthogonal.
188
00:12:41.29 --> 00:12:46
By the way, what about if one
of these guys is the zero
189
00:12:46 --> 00:12:48
vector?
190
00:12:48 --> 00:12:52
Suppose x is the zero vector,
and y is whatever.
191
00:12:52 --> 00:12:53
Are they orthogonal?
192
00:12:53 --> 00:12:54.19
Sure.
193
00:12:54.19 --> 00:12:58
In math the one thing about
math is you're supposed to
194
00:12:58 --> 00:13:00
follow the rules.
195
00:13:00 --> 00:13:04
So you're supposed to -- if x
is the zero vector,
196
00:13:04 --> 00:13:09
you're supposed to take the
zero vector dotted with y and of
197
00:13:09 --> 00:13:12
course you always get zero.
198
00:13:12 --> 00:13:19
So just so we're all sure,
the zero vector is orthogonal
199
00:13:19 --> 00:13:21
to everybody.
200
00:13:21 --> 00:13:28
But what I want to -- what I
now want to think about is
201
00:13:28 --> 00:13:29
subspaces.
202
00:13:29 --> 00:13:37
What does it mean for me to say
that some subspace is orthogonal
203
00:13:37 --> 00:13:41
to some other subspace?
204
00:13:41 --> 00:13:42
So OK.
205
00:13:42 --> 00:13:45.64
Now I've got to write this
down.
206
00:13:45.64 --> 00:13:51
So because we're defining
definition of subspace S is
207
00:13:51 --> 00:13:57
orthogonal so to subspace let's
say T, so I've got a couple of
208
00:13:57 --> 00:13:58
subspaces.
209
00:13:58 --> 00:14:04
And what should it mean for
those guys to be orthogonal?
210
00:14:04 --> 00:14:10
It's just sort of what's the
natural extension from
211
00:14:10 --> 00:14:16
orthogonal vectors to orthogonal
subspaces?
212
00:14:16 --> 00:14:25
Well, and in particular,
let's think of some orthogonal
213
00:14:25 --> 00:14:30
subspaces, like this wall.
214
00:14:30 --> 00:14:35
Let's say in three dimensions.
215
00:14:35 --> 00:14:43
So the blackboard extended to
infinity, right,
216
00:14:43 --> 00:14:47
is a --
is a subspace,
217
00:14:47 --> 00:14:52
a plane, a two-dimensional
subspace.
218
00:14:52 --> 00:14:58
It's a little bumpy but anyway,
it's a -- think of it as a
219
00:14:58 --> 00:15:05
subspace, let me take the floor
as another subspace.
220
00:15:05 --> 00:15:11
Again, it's not a great
subspace, MIT only built it like
221
00:15:11 --> 00:15:16.42
so-so, but I'll put the origin
right here.
222
00:15:16.42 --> 00:15:22
So the origin of the world is
right there.
223
00:15:22 --> 00:15:23
OK.
224
00:15:23 --> 00:15:29
Thereby giving linear algebra
its proper importance in this.
225
00:15:29 --> 00:15:29
OK.
226
00:15:29 --> 00:15:34
So there is one subspace,
there's another one.
227
00:15:34 --> 00:15:35
The floor.
228
00:15:35 --> 00:15:38
And are they orthogonal?
229
00:15:38 --> 00:15:44
What does it mean for two
subspaces to be orthogonal and
230
00:15:44 --> 00:15:49
in that special case are they
orthogonal?
231
00:15:49 --> 00:15:51
All right.
232
00:15:51 --> 00:15:54
Let's finish this sentence.
233
00:15:54 --> 00:16:01
What does it mean means we have
to know what we're talking about
234
00:16:01 --> 00:16:01
here.
235
00:16:01 --> 00:16:06
So what would be a reasonable
idea of orthogonal?
236
00:16:06 --> 00:16:10
Well, let me put the right
thing up.
237
00:16:10 --> 00:16:16
It means that every vector in
S, every vector in S,
238
00:16:16 --> 00:16:22
is orthogonal to --
what I going to say?
239
00:16:22 --> 00:16:25
Every vector in T.
240
00:16:25 --> 00:16:32
That's a reasonable and it's a
good and it's the right
241
00:16:32 --> 00:16:39
definition for two subspaces to
be orthogonal.
242
00:16:39 --> 00:16:47
But I just want you to see,
hey, what does that mean?
243
00:16:47 --> 00:16:55
So answer the question about
the -- the blackboard and the
244
00:16:55 --> 00:16:57
floor.
245
00:16:57 --> 00:17:03
Are those two subspaces,
they're two-dimensional,
246
00:17:03 --> 00:17:06
right, and we're in R^3.
247
00:17:06 --> 00:17:12
It's like a xz plane or
something and a xy plane.
248
00:17:12 --> 00:17:14
Are they orthogonal?
249
00:17:14 --> 00:17:21
Is every vector in the
blackboard orthogonal to every
250
00:17:21 --> 00:17:27
vector in the floor,
starting from the origin right
251
00:17:27 --> 00:17:29
there?
252
00:17:29 --> 00:17:30
Yes or no?
253
00:17:30 --> 00:17:32
I could take a vote.
254
00:17:32 --> 00:17:37
Well we get some yeses and some
noes.
255
00:17:37 --> 00:17:39
No is the answer.
256
00:17:39 --> 00:17:40
They're not.
257
00:17:40 --> 00:17:47
You can tell me a vector in the
blackboard and a vector in the
258
00:17:47 --> 00:17:51.57
floor that are not orthogonal.
259
00:17:51.57 --> 00:17:57
Well you can tell me quite a
few, I guess.
260
00:17:57 --> 00:18:02
Maybe like I could take some
forty-five-degree guy in the
261
00:18:02 --> 00:18:08
blackboard, and something in the
floor, they're not at ninety
262
00:18:08 --> 00:18:09
degrees, right?
263
00:18:09 --> 00:18:13.49
In fact, even more,
you could tell me a vector
264
00:18:13.49 --> 00:18:18
that's in both the blackboard
plane and the floor plane,
265
00:18:18 --> 00:18:23
so it's certainly not
orthogonal to itself.
266
00:18:23 --> 00:18:26
So for sure,
those two planes aren't
267
00:18:26 --> 00:18:27
orthogonal.
268
00:18:27 --> 00:18:29
What would that be?
269
00:18:29 --> 00:18:34
So what's a vector that's -- in
both of those planes?
270
00:18:34 --> 00:18:39
It's this guy running along the
crack there, in the
271
00:18:39 --> 00:18:42
intersection,
the intersection.
272
00:18:42 --> 00:18:47
A vector, you know --
if two subspaces meet at some
273
00:18:47 --> 00:18:51
vector, well then for sure
they're not orthogonal,
274
00:18:51 --> 00:18:54
because that vector is in one
and it's in the other,
275
00:18:54 --> 00:18:58
and it's not orthogonal to
itself unless it's zero.
276
00:18:58 --> 00:19:02
So the only I mean so
orthogonal is for me to say
277
00:19:02 --> 00:19:05
these two subspaces are
orthogonal first of all I'm
278
00:19:05 --> 00:19:11
certainly saying that they don't
intersect in any nonzero vector.
279
00:19:11 --> 00:19:17
But also I mean more than that
just not intersecting isn't good
280
00:19:17 --> 00:19:17
enough.
281
00:19:17 --> 00:19:22
So give me an example,
oh, let's say in the plane,
282
00:19:22 --> 00:19:26
oh well, when do we have
orthogonal subspaces in the
283
00:19:26 --> 00:19:27.37
plane?
284
00:19:27.37 --> 00:19:32
Yeah, tell me in the plane,
so we don't -- there aren't
285
00:19:32 --> 00:19:37
that many different subspaces in
the plane.
286
00:19:37 --> 00:19:42
What what have we got in the
plane as possible subspaces?
287
00:19:42 --> 00:19:45
The zero vector,
real small.
288
00:19:45 --> 00:19:47
A line through the origin.
289
00:19:47 --> 00:19:49
Or the whole plane.
290
00:19:49 --> 00:19:49
OK.
291
00:19:49 --> 00:19:55
Now so when is a line through
the origin orthogonal to the
292
00:19:55 --> 00:19:56
whole plane?
293
00:19:56 --> 00:19:58
Never, right,
never.
294
00:19:58 --> 00:20:03
When is a line through the
origin orthogonal to the zero
295
00:20:03 --> 00:20:05
subspace?
296
00:20:05 --> 00:20:06
Always.
297
00:20:06 --> 00:20:06
Right.
298
00:20:06 --> 00:20:12
When is a line through the
origin orthogonal to a different
299
00:20:12 --> 00:20:14.65
line through the origin?
300
00:20:14.65 --> 00:20:20
Well, that's the case that we
all have a clear picture of,
301
00:20:20 --> 00:20:25
they -- the two lines have to
meet at ninety degrees.
302
00:20:25 --> 00:20:30
They have only the -- so that's
like this simple case I'm
303
00:20:30 --> 00:20:31
talking about.
304
00:20:31 --> 00:20:35.55
There's one subspace,
there's the other subspace.
305
00:20:35.55 --> 00:20:37
They only meet at zero.
306
00:20:37 --> 00:20:39
And they're orthogonal.
307
00:20:39 --> 00:20:39
OK.
308
00:20:39 --> 00:20:39
Now.
309
00:20:39 --> 00:20:44
So we now know what it means
for two subspaces to be
310
00:20:44 --> 00:20:45
orthogonal.
311
00:20:45 --> 00:20:53
And now I want to say that this
is true for the row space and
312
00:20:53 --> 00:20:55
the null space.
313
00:20:55 --> 00:20:55
OK.
314
00:20:55 --> 00:20:58.55
So that's the neat fact.
315
00:20:58.55 --> 00:21:04
So row space is orthogonal to
the null space.
316
00:21:04 --> 00:21:07
Now how did I come up with
that?
317
00:21:07 --> 00:21:16
But you see the rank it's
great, that means that these --
318
00:21:16 --> 00:21:22
that these subspaces are just
the right things,
319
00:21:22 --> 00:21:30
they're just cutting the whole
space up into two perpendicular
320
00:21:30 --> 00:21:31
subspaces.
321
00:21:31 --> 00:21:32
OK.
322
00:21:32 --> 00:21:32
So why?
323
00:21:32 --> 00:21:37
Well, what have I got to work
with?
324
00:21:37 --> 00:21:41.3
All I know is the null space.
325
00:21:41.3 --> 00:21:49
The null space has vectors that
solve Ax equals zero.
326
00:21:49 --> 00:21:53
So this is a guy x.
x is in the null space.
327
00:21:53 --> 00:21:55
Then Ax is zero.
328
00:21:55 --> 00:21:59
So why is it orthogonal to the
rows of A?
329
00:21:59 --> 00:22:04
If I write down Ax equals zero,
which is all I know about the
330
00:22:04 --> 00:22:10
null space, then I guess I want
you to see that that's telling
331
00:22:10 --> 00:22:16
me, just that equation right
there is telling me that the
332
00:22:16 --> 00:22:20
rows of A, let me write it out.
333
00:22:20 --> 00:22:23
There's row one of A.
334
00:22:23 --> 00:22:24
Row two.
335
00:22:24 --> 00:22:27
Row m of A.
that's A.
336
00:22:27 --> 00:22:30
And it's multiplying X.
337
00:22:30 --> 00:22:34
And it's producing zero.
338
00:22:34 --> 00:22:34
OK.
339
00:22:34 --> 00:22:39
Written out that way you'll see
it.
340
00:22:39 --> 00:22:48
So I'm saying that a vector in
the row space is perpendicular
341
00:22:48 --> 00:22:52
to this guy X in the null space.
342
00:22:52 --> 00:22:57
And you see why?
343
00:22:57 --> 00:23:02
Because this equation is
telling you that row one of A
344
00:23:02 --> 00:23:06
multiplying that's a dot
product, right?
345
00:23:06 --> 00:23:12
Row one of A dot product with
this x is producing this zero.
346
00:23:12 --> 00:23:15
So x is orthogonal to the first
row.
347
00:23:15 --> 00:23:18
And to the second row.
348
00:23:18 --> 00:23:22
Row two of A,
x is giving that zero.
349
00:23:22 --> 00:23:26
Row m of A times x is giving
that zero.
350
00:23:26 --> 00:23:31
So x is -- the equation is
telling me that x is orthogonal
351
00:23:31 --> 00:23:32
to all the rows.
352
00:23:32 --> 00:23:35
Right, it's just sitting there.
353
00:23:35 --> 00:23:41
That's all we -- it had to be
sitting there because we didn't
354
00:23:41 --> 00:23:46
know anything more about the
null space than this.
355
00:23:46 --> 00:23:50
And now I guess to be totally
complete here I'd now check that
356
00:23:50 --> 00:23:53
x is orthogonal to each separate
row.
357
00:23:53 --> 00:23:56
But what else strictly speaking
do I have to do?
358
00:23:56 --> 00:23:59
To show that those subspaces
are orthogonal,
359
00:23:59 --> 00:24:03
I have to take this x in the
null space and show that it's
360
00:24:03 --> 00:24:07
orthogonal to every vector in
the row space,
361
00:24:07 --> 00:24:10
every vector in the row space,
so what --
362
00:24:10 --> 00:24:13
what else is in the row space?
363
00:24:13 --> 00:24:19
This row is in the row space,
that row is in the row space,
364
00:24:19 --> 00:24:23
they're all there,
but it's not the whole row
365
00:24:23 --> 00:24:27.15
space, right,
we just have to like remember,
366
00:24:27.15 --> 00:24:31
what does it mean,
what does that word space
367
00:24:31 --> 00:24:33
telling us?
368
00:24:33 --> 00:24:35
And what else is in the row
space?
369
00:24:35 --> 00:24:37
Besides the rows?
370
00:24:37 --> 00:24:38
All their combinations.
371
00:24:38 --> 00:24:42
So I really have to check that
sure enough if x is
372
00:24:42 --> 00:24:46
perpendicular to row one,
row two, all the different
373
00:24:46 --> 00:24:50
separate rows,
then also x is perpendicular to
374
00:24:50 --> 00:24:53
a combination of the rows.
375
00:24:53 --> 00:24:57
And that's just matrix
multiplication again.
376
00:24:57 --> 00:25:02
You know, I have row one
transpose x is zero,
377
00:25:02 --> 00:25:08
so on, row two transpose x is
zero, so I'm entitled to
378
00:25:08 --> 00:25:12
multiply that by some c1,
this by some c2,
379
00:25:12 --> 00:25:16
I still have zeroes,
I'm entitled to add,
380
00:25:16 --> 00:25:24
so I have c1 row one so --
so all this when I put that
381
00:25:24 --> 00:25:31
together that's big parentheses
c1 row one plus c2 row two and
382
00:25:31 --> 00:25:32
so on.
383
00:25:32 --> 00:25:34
Transpose x is zero.
384
00:25:34 --> 00:25:35
Right?
385
00:25:35 --> 00:25:42
I just added the zeroes and got
zero, and I just added these
386
00:25:42 --> 00:25:44
following the rule.
387
00:25:44 --> 00:25:46
No big deal.
388
00:25:46 --> 00:25:52
The whole point was right
sitting in that.
389
00:25:52 --> 00:25:53
OK.
390
00:25:53 --> 00:26:00
So if I come back to this
figure now I'm like a happier
391
00:26:00 --> 00:26:01
person.
392
00:26:01 --> 00:26:08
Because I have this -- I now
see how those subspaces are
393
00:26:08 --> 00:26:09
oriented.
394
00:26:09 --> 00:26:14
And these subspaces are also
oriented.
395
00:26:14 --> 00:26:21
Well, actually why is that
orthogonality?
396
00:26:21 --> 00:26:27
Well, it's the same statement
for A transpose that that one
397
00:26:27 --> 00:26:28.27
was for A.
398
00:26:28.27 --> 00:26:34
So I won't take time to prove
it again because we've checked
399
00:26:34 --> 00:26:39
it for every matrix and A
transpose is just as good a
400
00:26:39 --> 00:26:41
matrix as A.
401
00:26:41 --> 00:26:44
So we're orthogonal over there.
402
00:26:44 --> 00:26:48
So we really have carved up
this --
403
00:26:48 --> 00:26:53
this was like carving up
m-dimensional space into two
404
00:26:53 --> 00:26:59.26
subspaces and this one was
carving up n-dimensional space
405
00:26:59.26 --> 00:27:01
into two subspaces.
406
00:27:01 --> 00:27:03
And well, one more thing here.
407
00:27:03 --> 00:27:06
One more important thing.
408
00:27:06 --> 00:27:09
Let me move into three
dimensions.
409
00:27:09 --> 00:27:15
Tell me a couple of orthogonal
subspaces in three dimensions
410
00:27:15 --> 00:27:21
that somehow don't carve up the
whole space, there's stuff left
411
00:27:21 --> 00:27:23.08
there.
412
00:27:23.08 --> 00:27:28
I'm thinking of a couple of
orthogonal lines.
413
00:27:28 --> 00:27:33
If I -- suppose I'm in three
dimensions, R^3.
414
00:27:33 --> 00:27:39
And I have one line,
one one-dimensional subspace,
415
00:27:39 --> 00:27:42
and a perpendicular one.
416
00:27:42 --> 00:27:49
Could those be the row space
and the null space?
417
00:27:49 --> 00:27:55
Could those be the row space
and the null space?
418
00:27:55 --> 00:28:02
Could I be in three dimensions
and have a row space that's a
419
00:28:02 --> 00:28:06
line and a null space that's a
line?
420
00:28:06 --> 00:28:07
No.
421
00:28:07 --> 00:28:08
Why not?
422
00:28:08 --> 00:28:12
Because the dimensions aren't
right.
423
00:28:12 --> 00:28:12
Right?
424
00:28:12 --> 00:28:17
The dimensions are no good.
425
00:28:17 --> 00:28:21
The dimensions here,
r and n-r, they add up to
426
00:28:21 --> 00:28:23
three, they add up to n.
427
00:28:23 --> 00:28:28
If I take -- just follow that
example -- if the row space is
428
00:28:28 --> 00:28:32
one-dimensional,
suppose A is what's a good in
429
00:28:32 --> 00:28:37
R^3, I want a one-dimensional
row space, let me take one,
430
00:28:37 --> 00:28:40
two, five, two,
four, ten.
431
00:28:40 --> 00:28:45
What's the dimension of that
row space?
432
00:28:45 --> 00:28:45.97
One.
433
00:28:45.97 --> 00:28:50
What's the dimension of the
null space?
434
00:28:50 --> 00:28:57
Tell what's the null space look
like in that case?
435
00:28:57 --> 00:29:00
The row space is a line,
right?
436
00:29:00 --> 00:29:06.51
One-dimensional,
it's just a line through one,
437
00:29:06.51 --> 00:29:09
two, five.
438
00:29:09 --> 00:29:14
Geometrically what's the row
space look like?
439
00:29:14 --> 00:29:17
What's its dimension?
440
00:29:17 --> 00:29:22
So here r here n is three,
the rank is one,
441
00:29:22 --> 00:29:29
so the dimension of the null
space, so I'm looking at this x,
442
00:29:29 --> 00:29:30
x1, x2, x3.
443
00:29:30 --> 00:29:32
To give zero.
444
00:29:32 --> 00:29:40
So the dimension of the null
space is we all know is two.
445
00:29:40 --> 00:29:41
Right.
446
00:29:41 --> 00:29:42
It's a plane.
447
00:29:42 --> 00:29:47
And now actually we know,
we see better,
448
00:29:47 --> 00:29:49.07
what plane is it?
449
00:29:49.07 --> 00:29:51
What plane is it?
450
00:29:51 --> 00:29:55
It's the plane that's
perpendicular to one,
451
00:29:55 --> 00:29:56
two, five.
452
00:29:56 --> 00:29:57
Right?
453
00:29:57 --> 00:29:58
We now see.
454
00:29:58 --> 00:30:04.08
In fact the two,
four, ten didn't actually have
455
00:30:04.08 --> 00:30:07
any effect at all.
456
00:30:07 --> 00:30:11
I could have just ignored that.
457
00:30:11 --> 00:30:17
That didn't change the row
space or the null space.
458
00:30:17 --> 00:30:21
I'll just make that one
equation.
459
00:30:21 --> 00:30:21
Yeah.
460
00:30:21 --> 00:30:22
OK.
461
00:30:22 --> 00:30:22
Sure.
462
00:30:22 --> 00:30:26
That's the easiest to deal
with.
463
00:30:26 --> 00:30:28
One equation.
464
00:30:28 --> 00:30:30
Three unknowns.
465
00:30:30 --> 00:30:36
And I want to ask --
what would the equation give me
466
00:30:36 --> 00:30:40
the null space,
and you would have said back in
467
00:30:40 --> 00:30:45
September you would have said it
gives you a plane,
468
00:30:45 --> 00:30:47
and we're completely right.
469
00:30:47 --> 00:30:51
And the plane it gives you,
the normal vector,
470
00:30:51 --> 00:30:55.68
you remember in calculus,
there was this dumb normal
471
00:30:55.68 --> 00:30:57
vector called N.
472
00:30:57 --> 00:30:59
Well there it is.
473
00:30:59 --> 00:31:02
One, two, five.
474
00:31:02 --> 00:31:02
OK.
475
00:31:02 --> 00:31:10
What is the what's the point I
want to make here?
476
00:31:10 --> 00:31:20
I want to make -- I want to
emphasize that not only are the
477
00:31:20 --> 00:31:24
-- let me write it in words.
478
00:31:24 --> 00:31:33
So I want to write the null
space and the row space are
479
00:31:33 --> 00:31:43
orthogonal, that's this neat
fact, which we've --
480
00:31:43 --> 00:31:47
we've just checked from Ax
equals zero, but now I want to
481
00:31:47 --> 00:31:51
say more because there's a
little more that's true.
482
00:31:51 --> 00:31:55
Their dimensions add to the
whole space.
483
00:31:55 --> 00:31:58
So that's like a little extra
information.
484
00:31:58 --> 00:32:03
That it's not like I could have
-- I couldn't have a line and a
485
00:32:03 --> 00:32:06
line in three dimensions.
486
00:32:06 --> 00:32:11
Those don't add up one and one
don't add to three.
487
00:32:11 --> 00:32:16
So I used the word orthogonal
complements in R^n.
488
00:32:16 --> 00:32:21
And the idea of this word
complement is that the
489
00:32:21 --> 00:32:27
orthogonal complement of a row
space contains not just some
490
00:32:27 --> 00:32:31
vectors that are orthogonal to
it, but all.
491
00:32:31 --> 00:32:35.53
So what does that mean?
492
00:32:35.53 --> 00:32:42
That means that the null space
contains all,
493
00:32:42 --> 00:32:50
not just some but all,
vectors that are perpendicular
494
00:32:50 --> 00:32:52
to the row space.
495
00:32:52 --> 00:32:53.11
OK.
496
00:32:53.11 --> 00:33:01.67
Really what I've done in this
half of the lecture is just
497
00:33:01.67 --> 00:33:09
notice some of the nice geometry
that --
498
00:33:09 --> 00:33:13
that we didn't pick up before
because we didn't discuss
499
00:33:13 --> 00:33:16
perpendicular vectors before.
500
00:33:16 --> 00:33:19
But it was all sitting there.
501
00:33:19 --> 00:33:21
And now we picked it up.
502
00:33:21 --> 00:33:25
That these vectors are
orthogonal complements.
503
00:33:25 --> 00:33:30
And I guess I even call this
part one of the fundamental
504
00:33:30 --> 00:33:33
theorem of linear algebra.
505
00:33:33 --> 00:33:38
The fundamental theorem of
linear algebra is about these
506
00:33:38 --> 00:33:42.41
four subspaces,
so part one is about their
507
00:33:42.41 --> 00:33:46
dimension, maybe I should call
it part two now.
508
00:33:46 --> 00:33:48
Their dimensions we got.
509
00:33:48 --> 00:33:52
Now we're getting their
orthogonality,
510
00:33:52 --> 00:33:53
that's part two.
511
00:33:53 --> 00:33:57
And part three will be about
bases for them.
512
00:33:57 --> 00:33:59
Orthogonal bases.
513
00:33:59 --> 00:34:02
So that's coming up.
514
00:34:02 --> 00:34:03
OK.
515
00:34:03 --> 00:34:09
So I'm happy with that geometry
right now.
516
00:34:09 --> 00:34:09
OK.
517
00:34:09 --> 00:34:10
OK.
518
00:34:10 --> 00:34:16
Now what's my next goal in this
chapter?
519
00:34:16 --> 00:34:21
Here's the main problem of the
chapter.
520
00:34:21 --> 00:34:30
The main problem of the chapter
is -- so this is coming.
521
00:34:30 --> 00:34:35
It's coming attraction.
522
00:34:35 --> 00:34:42
This is the very last chapter
that's about Ax=b.
523
00:34:42 --> 00:34:50.03
I would like to solve that
system of equations when there
524
00:34:50.03 --> 00:34:52
is no solution.
525
00:34:52 --> 00:34:58
You may say what a ridiculous
thing to do.
526
00:34:58 --> 00:35:03
But I have to say it's done all
the time.
527
00:35:03 --> 00:35:07
In fact it has to be done.
528
00:35:07 --> 00:35:14
You get -- so the problem is
solve --
529
00:35:14 --> 00:35:20
the best possible solve I'll
put quote Ax=b when there is no
530
00:35:20 --> 00:35:21.65
solution.
531
00:35:21.65 --> 00:35:25
And of course what does that
mean?
532
00:35:25 --> 00:35:28
b isn't in the column space.
533
00:35:28 --> 00:35:34
And it's quite typical if this
matrix A is rectangular if I --
534
00:35:34 --> 00:35:41
maybe I have m equations and
that's bigger than the number of
535
00:35:41 --> 00:35:47
unknowns, then for sure the rank
is not m,
536
00:35:47 --> 00:35:52
the rank couldn't be m now,
so there'll be a lot of
537
00:35:52 --> 00:35:57
right-hand sides with no
solution, but here's an example.
538
00:35:57 --> 00:36:01
Some satellite is buzzing
along.
539
00:36:01 --> 00:36:03
You measure its position.
540
00:36:03 --> 00:36:06.98
You make a thousand
measurements.
541
00:36:06.98 --> 00:36:12
So that gives you a thousand
equations for the -- for the
542
00:36:12 --> 00:36:18
parameters that --
that give the position.
543
00:36:18 --> 00:36:23
But there aren't a thousand
parameters, there's just maybe
544
00:36:23 --> 00:36:25
six or something.
545
00:36:25 --> 00:36:30
Or you're measuring the --
you're doing questionnaires.
546
00:36:30 --> 00:36:33
You're measuring resistances.
547
00:36:33 --> 00:36:35
You're taking pulses.
548
00:36:35 --> 00:36:40.69
You're measuring somebody's
pulse rate.
549
00:36:40.69 --> 00:36:40
OK.
550
00:36:40 --> 00:36:42
There's just one unknown.
551
00:36:42 --> 00:36:43
The pulse rate.
552
00:36:43 --> 00:36:47
So you measure it once,
OK, fine, but if you really
553
00:36:47 --> 00:36:50
want to know it,
you measure it multiple times,
554
00:36:50 --> 00:36:53
but then the measurements have
noise in them,
555
00:36:53 --> 00:36:57
so there's -- the problem is
that in many many problems we've
556
00:36:57 --> 00:37:00
got too many equations and
they've got noise in the
557
00:37:00 --> 00:37:02
right-hand side.
558
00:37:02 --> 00:37:08
So Ax=b I can't expect to solve
it exactly right,
559
00:37:08 --> 00:37:15
because I don't even know what
-- there's a measurement mistake
560
00:37:15 --> 00:37:16
in b.
561
00:37:16 --> 00:37:19.27
But there's information too.
562
00:37:19.27 --> 00:37:25
There's a lot of information
about x in there.
563
00:37:25 --> 00:37:30
And what I want to do is like
separate the noise,
564
00:37:30 --> 00:37:33
the junk, from the information.
565
00:37:33 --> 00:37:37
And so this is a
straightforward linear algebra
566
00:37:37 --> 00:37:38
problem.
567
00:37:38 --> 00:37:42
How do I solve,
what's the best solution?
568
00:37:42 --> 00:37:42.57
OK.
569
00:37:42.57 --> 00:37:42.96
Now.
570
00:37:42.96 --> 00:37:48
I want to say so that's like
describes the problem in an
571
00:37:48 --> 00:37:50
algebraic way.
572
00:37:50 --> 00:37:56
I got some equations,
I'm looking for the best
573
00:37:56 --> 00:37:57
solution.
574
00:37:57 --> 00:38:03
Well, one way to find it is --
one way to start,
575
00:38:03 --> 00:38:10
one way to find a solution is
throw away equations until
576
00:38:10 --> 00:38:15.99
you've got a nice,
square invertible system and
577
00:38:15.99 --> 00:38:17
solve that.
578
00:38:17 --> 00:38:21
That's not satisfactory.
579
00:38:21 --> 00:38:25
There's no reason in these
measurements to say these
580
00:38:25 --> 00:38:30
measurements are perfect and
these measurements are useless.
581
00:38:30 --> 00:38:34
We want to use all the
measurements to get the best
582
00:38:34 --> 00:38:37
information, to get the maximum
information.
583
00:38:37 --> 00:38:38
But how?
584
00:38:38 --> 00:38:38
OK.
585
00:38:38 --> 00:38:42
Let me anticipate a matrix
that's going to show up.
586
00:38:42 --> 00:38:45
This A is typically
rectangular.
587
00:38:45 --> 00:38:50
But a matrix that shows up
whenever you have -- and we
588
00:38:50 --> 00:38:53
chapter three was all about
rectangular matrices.
589
00:38:53 --> 00:38:57
And we know when this is
solvable, you could do
590
00:38:57 --> 00:38:59
elimination on it,
right?
591
00:38:59 --> 00:39:03
But I'm thinking hey,
you do elimination and you get
592
00:39:03 --> 00:39:06
equation zero equal other
non-zeroes.
593
00:39:06 --> 00:39:11
I'm thinking we really --
elimination is going to fail.
594
00:39:11 --> 00:39:14
So that's our question.
595
00:39:14 --> 00:39:19
Elimination will get us down to
-- will tell us if there is a
596
00:39:19 --> 00:39:21
solution or not.
597
00:39:21 --> 00:39:23.35
But I'm now thinking not.
598
00:39:23.35 --> 00:39:23
OK.
599
00:39:23 --> 00:39:26.1
So what are we going to do?
600
00:39:26.1 --> 00:39:27
All right.
601
00:39:27 --> 00:39:32
I want to tell you to jump
ahead to the matrix that will
602
00:39:32 --> 00:39:34
play a key role.
603
00:39:34 --> 00:39:41
So this is the matrix that you
want to understand for this
604
00:39:41 --> 00:39:42.54
chapter four.
605
00:39:42.54 --> 00:39:46
And it's the matrix A transpose
A.
606
00:39:46 --> 00:39:51
What's -- tell me some things
about that matrix.
607
00:39:51 --> 00:39:55
So A is this m by n matrix,
rectangular,
608
00:39:55 --> 00:40:02
but now I'm saying that the
good matrix that shows up in the
609
00:40:02 --> 00:40:06
end is A transpose A.
610
00:40:06 --> 00:40:09
So tell me something about
that.
611
00:40:09 --> 00:40:16
Is it -- what's the first thing
you know about A transpose A.
612
00:40:16 --> 00:40:17
It's square.
613
00:40:17 --> 00:40:18
Right?
614
00:40:18 --> 00:40:23.49
Square because this is m by n
and this is n by m.
615
00:40:23.49 --> 00:40:26
So this is the result is n by
n.
616
00:40:26 --> 00:40:27.41
Good.
617
00:40:27.41 --> 00:40:28
Square.
618
00:40:28 --> 00:40:30
What else?
619
00:40:30 --> 00:40:32
It's symmetric.
620
00:40:32 --> 00:40:32
Good.
621
00:40:32 --> 00:40:34
It's symmetric.
622
00:40:34 --> 00:40:37
Because you remember how to do
that.
623
00:40:37 --> 00:40:42.61
If we transpose that matrix
let's transpose it,
624
00:40:42.61 --> 00:40:45
A transpose A,
if I transpose it,
625
00:40:45 --> 00:40:51.15
then that comes first
transposed, this comes second,
626
00:40:51.15 --> 00:40:56
transposed, and then
transposing twice is leaves it
627
00:40:56 --> 00:41:01
--
brings it back to the same so
628
00:41:01 --> 00:41:03
it's symmetric.
629
00:41:03 --> 00:41:04
Good.
630
00:41:04 --> 00:41:10
Now we now know how to ask more
about a matrix.
631
00:41:10 --> 00:41:14
I'm interested in is it
invertible?
632
00:41:14 --> 00:41:18
If not, what's its null space?
633
00:41:18 --> 00:41:25.39
So I want to know about --
because you're going to see,
634
00:41:25.39 --> 00:41:30.18
well, let me --
let me even,
635
00:41:30.18 --> 00:41:34
well I shouldn't do this,
but I will.
636
00:41:34 --> 00:41:41
Let me tell you what equation
to solve when you can't solve
637
00:41:41 --> 00:41:42
that one.
638
00:41:42 --> 00:41:49
The good equation comes from
multiplying both sides by A
639
00:41:49 --> 00:41:57
transpose, so the good equation
that you get to is this one.
640
00:41:57 --> 00:42:00
A transpose Ax equals A
transpose b.
641
00:42:00 --> 00:42:05
That will be the central
equation in the chapter.
642
00:42:05 --> 00:42:08
So I think why not tell it to
you.
643
00:42:08 --> 00:42:10
Why not admit it right away.
644
00:42:10 --> 00:42:11
OK.
645
00:42:11 --> 00:42:14
I have to -- I should really
give x.
646
00:42:14 --> 00:42:20
I want to sort of indicate that
this x isn't I mean this x was
647
00:42:20 --> 00:42:27
the solution to that equation if
it existed, but probably didn't.
648
00:42:27 --> 00:42:32
Now let me give this a
different name,
649
00:42:32 --> 00:42:33
x hat.
650
00:42:33 --> 00:42:39
Because I'm hoping this one
will have a solution.
651
00:42:39 --> 00:42:45.3
And I'm saying that it's my
best solution.
652
00:42:45.3 --> 00:42:50
I'll have to say what does best
mean.
653
00:42:50 --> 00:42:55
But that's going to be my --
my plan.
654
00:42:55 --> 00:43:01
I'm going to say that the best
solution solves this equation.
655
00:43:01 --> 00:43:07
So you see right away why I'm
so interested in this matrix A
656
00:43:07 --> 00:43:08
transpose A.
657
00:43:08 --> 00:43:11
And in its invertibility.
658
00:43:11 --> 00:43:11
OK.
659
00:43:11 --> 00:43:14
Now, when is it invertible?
660
00:43:14 --> 00:43:14
OK.
661
00:43:14 --> 00:43:19
Let me take a case,
let me just do an example and
662
00:43:19 --> 00:43:24
then --
I'll just pick a matrix here.
663
00:43:24 --> 00:43:29
Just so we see what A transpose
A looks like.
664
00:43:29 --> 00:43:34
So let me take a matrix A one,
one, one, one,
665
00:43:34 --> 00:43:35
two, five.
666
00:43:35 --> 00:43:38
Just to invent a matrix.
667
00:43:38 --> 00:43:40
So there's a matrix A.
668
00:43:40 --> 00:43:46.44
Notice that it has M equal
three rows and N equal two
669
00:43:46.44 --> 00:43:48.59
columns.
670
00:43:48.59 --> 00:43:55
Its rank is -- the rank of that
matrix is two.
671
00:43:55 --> 00:44:01
Right, yeah,
the columns are independent.
672
00:44:01 --> 00:44:03
Does Ax equal b?
673
00:44:03 --> 00:44:08.94
If I look at Ax=b,
so x is just x1 x2,
674
00:44:08.94 --> 00:44:11
and b is b1 b2 b3.
675
00:44:11 --> 00:44:15
Do I expect to solve Ax=b?
676
00:44:15 --> 00:44:20
What's -- no way,
right?
677
00:44:20 --> 00:44:26
I mean linear algebra's great,
but solving three equations
678
00:44:26 --> 00:44:30
with only two unknowns usually
we can't do it.
679
00:44:30 --> 00:44:35
We can only solve it if this
vector is b is what?
680
00:44:35 --> 00:44:40
I can solve that equation if
that vector b1 b2 b3 is in the
681
00:44:40 --> 00:44:42
column space.
682
00:44:42 --> 00:44:47
If it's a combination of those
columns then fine.
683
00:44:47 --> 00:44:50
But usually it won't be.
684
00:44:50 --> 00:44:55.96
The combinations just fill up a
plane and most vectors aren't on
685
00:44:55.96 --> 00:44:56
that plane.
686
00:44:56 --> 00:45:02
So what I'm saying is that I'm
going to work with the matrix A
687
00:45:02 --> 00:45:03
transpose A.
688
00:45:03 --> 00:45:09
And I just want to figure out
in this example what A transpose
689
00:45:09 --> 00:45:09.93
A is.
690
00:45:09.93 --> 00:45:12
So it's two by two.
691
00:45:12 --> 00:45:18
The first entry is a three,
the next entry is an eight,
692
00:45:18 --> 00:45:21.96
this entry is -- what's that
entry?
693
00:45:21.96 --> 00:45:23
Eight, for sure.
694
00:45:23 --> 00:45:27
We knew it had to be,
and this entry is,
695
00:45:27 --> 00:45:31
what's that now,
getting out my trusty
696
00:45:31 --> 00:45:35.08
calculator, thirty,
is that right?
697
00:45:35.08 --> 00:45:35
Thirty.
698
00:45:35 --> 00:45:40
And is that matrix invertible?
699
00:45:40 --> 00:45:42
There's an A transpose A.
700
00:45:42 --> 00:45:44
And it is invertible,
right?
701
00:45:44 --> 00:45:49
Three, eight is not a multiple
of eight, thirty -- and it's
702
00:45:49 --> 00:45:50
invertible.
703
00:45:50 --> 00:45:54
And that's the normal,
that's what I expect.
704
00:45:54 --> 00:45:56
So this is I want to show.
705
00:45:56 --> 00:46:00
So here's the final -- here's
the key point.
706
00:46:00 --> 00:46:06.64
The null space of A transpose A
-- it's not going to be always
707
00:46:06.64 --> 00:46:07
invertible.
708
00:46:07 --> 00:46:13
Tell me a matrix -- I have to
say that I can't say A transpose
709
00:46:13 --> 00:46:15
A is always invertible.
710
00:46:15 --> 00:46:18.31
Because that's asking too much.
711
00:46:18.31 --> 00:46:22
I mean what could the matrix A
be, for example,
712
00:46:22 --> 00:46:27
so that A transpose A was not
invertible?
713
00:46:27 --> 00:46:33
Well, it even could be the zero
matrix.
714
00:46:33 --> 00:46:37
I mean that's like extreme
case.
715
00:46:37 --> 00:46:45
Suppose I make this rank --
suppose I change to that A.
716
00:46:45 --> 00:46:54
Now I figure out A transpose A
again and I get -- what do I
717
00:46:54 --> 00:46:55
get?
718
00:46:55 --> 00:47:03
I get nine, I get nine of
course and here I get what's
719
00:47:03 --> 00:47:06
that entry?
720
00:47:06 --> 00:47:07
Twenty-seven.
721
00:47:07 --> 00:47:10.76
And is that matrix invertible?
722
00:47:10.76 --> 00:47:11
No.
723
00:47:11 --> 00:47:16
And why do I -- I knew it
wouldn't be invertible anyway.
724
00:47:16 --> 00:47:20
Because this matrix only has
rank one.
725
00:47:20 --> 00:47:24
And if I have a product of
matrices of rank one,
726
00:47:24 --> 00:47:31
the product is not going to
have a rank bigger than one.
727
00:47:31 --> 00:47:37
So I'm not surprised that the
answer only has rank one.
728
00:47:37 --> 00:47:43
And that's what I -- always
happens, that the rank of A
729
00:47:43 --> 00:47:48
transpose A comes out to equal
the rank of A.
730
00:47:48 --> 00:47:54
So, yes, so the null space of A
transpose A equals the null
731
00:47:54 --> 00:48:01
space of A, the rank of A
transpose A equals the rank of
732
00:48:01 --> 00:48:02
A.
733
00:48:02 --> 00:48:10.13
So let's -- as soon as I can
why that's true.
734
00:48:10.13 --> 00:48:18
But let's draw from that what
the fact that I want.
735
00:48:18 --> 00:48:28.95
This tells me that this square
symmetric matrix is invertible
736
00:48:28.95 --> 00:48:34
if -- so here's my conclusion.
737
00:48:34 --> 00:48:43
A transpose A is invertible if
exactly when --
738
00:48:43 --> 00:48:51
exactly if this null space is
only got the zero vector.
739
00:48:51 --> 00:48:58
Which means the columns of A
are independent.
740
00:48:58 --> 00:49:07
So A transpose A is invertible
exactly if A has independent
741
00:49:07 --> 00:49:08
columns.
742
00:49:08 --> 00:49:16
That's the fact that I need
about A transpose A.
743
00:49:16 --> 00:49:20
And then you'll see next time
how A transpose A enters
744
00:49:20 --> 00:49:21
everything.
745
00:49:21 --> 00:49:24
Next lecture is actually a
crucial one.
746
00:49:24 --> 00:49:28
Here I'm preparing for it by
getting us thinking about A
747
00:49:28 --> 00:49:29
transpose A.
748
00:49:29 --> 00:49:33
And its rank is the same as the
rank of A, and we can decide
749
00:49:33 --> 00:49:35
when it's invertible.
750
00:49:35 --> 00:49:36
OK.
751
00:49:36 --> 00:49:38
So I'll see you Friday.
752
00:49:38 --> 00:49:41
Thanks.