1
00:00:00 --> 00:00:01
2
00:00:01 --> 00:00:02
The following content is
provided under a creative
3
00:00:03 --> 00:00:03
commons license.
4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to
5
00:00:06 --> 00:00:09
offer high-quality educational
resources for free.
6
00:00:09 --> 00:00:12
To make a donation or to view
additional materials from
7
00:00:12 --> 00:00:16
hundreds of MIT courses visit
MIT OpenCourseWare
8
00:00:16 --> 00:00:19
at ocw.mit.edu.
9
00:00:19 --> 00:00:24
PROFESSOR STRANG: Starting
today on section 1.3, actually
10
00:00:24 --> 00:00:26
we'll finish that today.
11
00:00:26 --> 00:00:31
And that's about the big
problem in, or the most common
12
00:00:31 --> 00:00:33
problem in scientific
computing, solving
13
00:00:33 --> 00:00:35
a linear system.
14
00:00:35 --> 00:00:35
Au=b.
15
00:00:36 --> 00:00:37
How would you do that?
16
00:00:37 --> 00:00:43
So we will certainly do
it for some examples.
17
00:00:43 --> 00:00:49
Like our free-fixed or our
fixed-fixed matrices.
18
00:00:49 --> 00:00:51
But I thought I'd just
put some comments.
19
00:00:51 --> 00:00:57
Often I make comments that go
beyond what specific things
20
00:00:57 --> 00:00:59
we will do in detail.
21
00:00:59 --> 00:01:05
Certainly the workhorse of, if
I'm in MATLAB notation, the
22
00:01:05 --> 00:01:08
workhorse would be backslash.
23
00:01:08 --> 00:01:11
That command is the
quick, natural way
24
00:01:11 --> 00:01:14
to get the answer u.
25
00:01:14 --> 00:01:18
And I want to say more
about backslash.
26
00:01:18 --> 00:01:23
I won't be able to say
all the things it does.
27
00:01:23 --> 00:01:28
There's a lot built
into that command.
28
00:01:28 --> 00:01:32
And the MATLAB helpdesk would
give the details of all
29
00:01:32 --> 00:01:35
the different things that
backslash actually does.
30
00:01:35 --> 00:01:40
It looks first, can it divide
the problem into blocks,
31
00:01:40 --> 00:01:44
into smaller problems.
32
00:01:44 --> 00:01:47
It looks to see if
A is symmetric.
33
00:01:47 --> 00:01:50
If it's symmetric, because of
course, that would cut the work
34
00:01:50 --> 00:01:53
in half because you know half
the matrix from the other
35
00:01:53 --> 00:01:57
half in the symmetric case.
36
00:01:57 --> 00:02:02
It goes hopefully along
and eventually gives
37
00:02:02 --> 00:02:03
you the answer.
38
00:02:03 --> 00:02:05
Ok, so I'm going to
say more about that.
39
00:02:05 --> 00:02:08
Because that's by elimination.
40
00:02:08 --> 00:02:11
The method there
is elimination.
41
00:02:11 --> 00:02:14
I just thought I would mention
that if I had a large sparse
42
00:02:14 --> 00:02:22
matrix, so I was in sparse math
MATLAB and used backslash, it
43
00:02:22 --> 00:02:26
would try to reorder the rows.
44
00:02:26 --> 00:02:30
Or it will try to reorder
the rows in an optimal way.
45
00:02:30 --> 00:02:32
Now, what would be
an optimal way?
46
00:02:32 --> 00:02:34
Just so you have
an idea of what.
47
00:02:34 --> 00:02:38
So elimination takes
this matrix, right?
48
00:02:38 --> 00:02:41
It's got all it's rows.
49
00:02:41 --> 00:02:44
And we'll do examples.
50
00:02:44 --> 00:02:51
So the idea of elimination
is that it produces a zero.
51
00:02:51 --> 00:02:57
So let's suppose it accepts the
first row as the pivot row.
52
00:02:57 --> 00:03:00
And this 1, 1 entry
would be the pivot.
53
00:03:00 --> 00:03:03
And then how does
it use the pivot?
54
00:03:03 --> 00:03:08
It subtracts multiples of that
pivot row from the other rows
55
00:03:08 --> 00:03:12
in order to get zeroes
in the first column.
56
00:03:12 --> 00:03:17
So then it has a new
matrix that's like
57
00:03:17 --> 00:03:19
it's one size smaller.
58
00:03:19 --> 00:03:26
And then it just continues all
the way until it gets to a
59
00:03:26 --> 00:03:29
triangular matrix with all the
pivots sitting there
60
00:03:29 --> 00:03:33
on the diagonal.
61
00:03:33 --> 00:03:35
So that's the idea
of elimination.
62
00:03:35 --> 00:03:38
And we'll do small examples.
63
00:03:38 --> 00:03:44
So why would you want
to reorder the rows?
64
00:03:44 --> 00:03:47
Two reason, two good reasons
for reordering the rows.
65
00:03:47 --> 00:03:53
One reason which MATLAB will
always unless it has some good
66
00:03:53 --> 00:04:00
reason not to, is if that pivot
is too small compared to
67
00:04:00 --> 00:04:08
numbers below it it will pick
the largest and reorder to put
68
00:04:08 --> 00:04:10
the largest pivot up there.
69
00:04:10 --> 00:04:11
So you like large pivots.
70
00:04:11 --> 00:04:13
Why do you like large pivots?
71
00:04:13 --> 00:04:17
Because for numerical
stability that gives
72
00:04:17 --> 00:04:19
you small multipliers.
73
00:04:19 --> 00:04:23
If this is the biggest number
then all the multiples that you
74
00:04:23 --> 00:04:28
need, the multiple of that row
that you want to subtract from
75
00:04:28 --> 00:04:35
the other rows will
be less than one.
76
00:04:35 --> 00:04:41
The multiplier is, so just
let's remember, the multiplier
77
00:04:41 --> 00:04:51
is the entry to eliminate
divided by the pivot.
78
00:04:51 --> 00:04:55
So if that entry to be
eliminated is smaller then the
79
00:04:55 --> 00:04:57
pivot, then the multiplier's
less than one and
80
00:04:57 --> 00:04:59
things stay stable.
81
00:04:59 --> 00:05:04
So that's a picture of
elimination, a sort
82
00:05:04 --> 00:05:05
of first picture.
83
00:05:05 --> 00:05:12
And if it's sparse that means
there's a lot of zeroes.
84
00:05:12 --> 00:05:15
And where would we like
those zeroes to be?
85
00:05:15 --> 00:05:21
Where would we like to find
zeroes to save time and use to
86
00:05:21 --> 00:05:26
avoid all the steps, to avoid
going through every step.
87
00:05:26 --> 00:05:33
If this was already a zero
then it's there and we
88
00:05:33 --> 00:05:36
don't need to get it there.
89
00:05:36 --> 00:05:37
The job is already done.
90
00:05:37 --> 00:05:39
We can go forward.
91
00:05:39 --> 00:05:44
And if we have a zero here then
it's already zero in the next
92
00:05:44 --> 00:05:48
one and we don't have to do,
that would be the second pivot
93
00:05:48 --> 00:05:52
row, we don't have to subtract
it from this row because
94
00:05:52 --> 00:05:53
a zero is already there.
95
00:05:53 --> 00:05:55
So we like zeroes.
96
00:05:55 --> 00:05:57
And sparse matrices
have a lot of them.
97
00:05:57 --> 00:06:00
But we only like them if
they're in the right place.
98
00:06:00 --> 00:06:02
And what's the right place?
99
00:06:02 --> 00:06:06
It's, roughly speaking, it's
the beginning of a row.
100
00:06:06 --> 00:06:11
So we're very happy with, we
would like to get an ordering
101
00:06:11 --> 00:06:13
that looks somehow like this.
102
00:06:13 --> 00:06:18
We would like to get a
bunch of zeroes in there.
103
00:06:18 --> 00:06:21
Well that's, I
really overdid it.
104
00:06:21 --> 00:06:23
Maybe we can't get that many.
105
00:06:23 --> 00:06:26
But my point is, you see the
point that those are at
106
00:06:26 --> 00:06:27
the beginning of the rows.
107
00:06:27 --> 00:06:34
So let's say we can't quite
get to that, but maybe
108
00:06:34 --> 00:06:35
something more like that.
109
00:06:35 --> 00:06:36
Zero.
110
00:06:36 --> 00:06:41
So we get some rows starting
with zeroes and then we
111
00:06:41 --> 00:06:47
get rows that don't.
112
00:06:47 --> 00:06:49
Which of course, I mean, they
were there in the first place.
113
00:06:49 --> 00:06:52
What we've done is move
them to the bottom.
114
00:06:52 --> 00:06:56
And the reason for moving them
to the bottom is that they
115
00:06:56 --> 00:07:00
don't destroy rows below them.
116
00:07:00 --> 00:07:03
Yeah.
117
00:07:03 --> 00:07:06
I didn't show how
that would happen.
118
00:07:06 --> 00:07:09
Suppose I had a number there.
119
00:07:09 --> 00:07:10
Suppose that wasn't a zero.
120
00:07:10 --> 00:07:12
And all these were.
121
00:07:12 --> 00:07:16
Then when I do that elimination
I subtract a multiple of
122
00:07:16 --> 00:07:18
that row from this row.
123
00:07:18 --> 00:07:21
Do you see that those
zeroes will fill in?
124
00:07:21 --> 00:07:24
So fill in is the bad thing
that you're trying to avoid.
125
00:07:24 --> 00:07:30
So in this quick discussion of
reordering you're trying to
126
00:07:30 --> 00:07:34
order the rows so that they
start with zeroes for as long
127
00:07:34 --> 00:07:38
as you can because those
zeroes will never fill in.
128
00:07:38 --> 00:07:42
Zeroes inside a band, yes, just
to make the point again,
129
00:07:42 --> 00:07:47
suppose I have a long row.
130
00:07:47 --> 00:07:51
Then if I have some zeroes
here, here, here, useless.
131
00:07:51 --> 00:07:58
Because when I go to do
elimination on these rows, when
132
00:07:58 --> 00:08:01
I subtract a multiple of that
row from that row, that zero
133
00:08:01 --> 00:08:03
will fill in, it didn't
help, it's gone.
134
00:08:03 --> 00:08:05
And this and this.
135
00:08:05 --> 00:08:10
So zeroes sort of near
the diagonal are often
136
00:08:10 --> 00:08:12
going to fill in.
137
00:08:12 --> 00:08:16
It's zeroes at the far
left that are good.
138
00:08:16 --> 00:08:20
So that's a discussion of
a topic that actually
139
00:08:20 --> 00:08:23
comes into 18.086.
140
00:08:23 --> 00:08:26
And so does the third words
that I thought I would
141
00:08:26 --> 00:08:27
just write on the board.
142
00:08:27 --> 00:08:32
So this would be topic
specializing in how do
143
00:08:32 --> 00:08:34
you solve large systems.
144
00:08:34 --> 00:08:38
And reordering with
elimination is one way.
145
00:08:38 --> 00:08:42
And a second approach is, I
just put these words up here
146
00:08:42 --> 00:08:44
so that you've seen them.
147
00:08:44 --> 00:08:47
Conjugate gradient method,
that's a giant success in
148
00:08:47 --> 00:08:51
solving symmetric problems.
149
00:08:51 --> 00:08:52
Large symmetric problems.
150
00:08:52 --> 00:08:59
Something called multigrid is
a terrific, incomplete LU.
151
00:08:59 --> 00:09:05
People have worked hard on
ideas for solving problems
152
00:09:05 --> 00:09:09
so large that this
becomes too expensive.
153
00:09:09 --> 00:09:14
But backslash is the
natural choice.
154
00:09:14 --> 00:09:16
If you can do it, it's
like it's so simple.
155
00:09:16 --> 00:09:19
So let me focus on backslash.
156
00:09:19 --> 00:09:22
So backslash is the
key to talk about.
157
00:09:22 --> 00:09:28
Let me see, let's just think
what does backslash do?
158
00:09:28 --> 00:09:33
Suppose I had two equations,
two different right-hand sides,
159
00:09:33 --> 00:09:38
b and c with the same matrix A.
160
00:09:38 --> 00:09:44
Would I solve them, would I do
u=A\b and then separately A\c?
161
00:09:45 --> 00:09:46
No.
162
00:09:46 --> 00:09:52
If I had two equations with the
same matrix, there's a big
163
00:09:52 --> 00:09:58
saving in, suppose I have
two equations, two,
164
00:09:58 --> 00:09:59
well, can I put it?
165
00:09:59 --> 00:10:01
I'll make a matrix.
166
00:10:01 --> 00:10:03
I mean this is what
MATLAB would always do.
167
00:10:03 --> 00:10:11
Put those two right-hand
sides into a matrix.
168
00:10:11 --> 00:10:15
Then backslash is
happy with that.
169
00:10:15 --> 00:10:20
That backslash command would
solve both equations.
170
00:10:20 --> 00:10:23
It would give you the answer
then, would be, it would
171
00:10:23 --> 00:10:25
be two parts to it.
172
00:10:25 --> 00:10:29
What would be the first
column of u then?
173
00:10:29 --> 00:10:31
So, do you see what I have now?
174
00:10:31 --> 00:10:35
I have a square matrix
A times an unknown u.
175
00:10:35 --> 00:10:37
And I've got two
right-hand sides.
176
00:10:37 --> 00:10:39
The point is this
often happens.
177
00:10:39 --> 00:10:45
If you're doing, say,
a design problem.
178
00:10:45 --> 00:10:46
You might try
different designs.
179
00:10:46 --> 00:10:50
So those different designs give
you different right-hand sides.
180
00:10:50 --> 00:10:54
And then you have a
problem to solve.
181
00:10:54 --> 00:10:56
What's the response
to those designs?
182
00:10:56 --> 00:11:00
What are the displacements,
what happens?
183
00:11:00 --> 00:11:05
So you want to solve with two
right-hand sides and the point
184
00:11:05 --> 00:11:10
is you don't want to go
through the elimination
185
00:11:10 --> 00:11:13
process on A twice.
186
00:11:13 --> 00:11:15
That's crazy.
187
00:11:15 --> 00:11:20
You see how that elimination
process on A has this, what
188
00:11:20 --> 00:11:23
I described here,
didn't look at b.
189
00:11:23 --> 00:11:26
I didn't even get to the
right-hand side part.
190
00:11:26 --> 00:11:28
I was dealing with the
expensive part, which
191
00:11:28 --> 00:11:30
is the matrix A.
192
00:11:30 --> 00:11:32
So we don't want to
pay that price twice.
193
00:11:32 --> 00:11:37
And therefore backslash
is all set up.
194
00:11:37 --> 00:11:42
So what's the first
column of u then?
195
00:11:42 --> 00:11:42
It's A\b.
196
00:11:43 --> 00:11:47
And now what's another way to
write, a more mathematical
197
00:11:47 --> 00:11:48
way to write A?
198
00:11:48 --> 00:11:50
What's the answer to Au=b.
199
00:11:50 --> 00:11:53
200
00:11:53 --> 00:11:57
It's u equal A
inverse b, right?
201
00:11:57 --> 00:12:00
That's the answer in the first
column and A inverse c would be
202
00:12:00 --> 00:12:03
the answer in the
second column.
203
00:12:03 --> 00:12:06
So it would produce
those answers.
204
00:12:06 --> 00:12:09
Backslash will produce the
answer to the first equation
205
00:12:09 --> 00:12:11
and the answer the
second equation.
206
00:12:11 --> 00:12:16
Will it do that by
finding A inverse?
207
00:12:16 --> 00:12:18
No.
208
00:12:18 --> 00:12:22
A inverse, for multiple
reasons, we don't often
209
00:12:22 --> 00:12:28
compute, if it's 3 by 3, 4 by
4, then it's not a bad idea to
210
00:12:28 --> 00:12:30
see what A inverse looks like.
211
00:12:30 --> 00:12:35
We'll do that actually, because
it's very enlightening if the
212
00:12:35 --> 00:12:38
matrix is small, you can
see what's going on.
213
00:12:38 --> 00:12:42
But for a large problem
we don't want A inverse.
214
00:12:42 --> 00:12:43
We want the answer.
215
00:12:43 --> 00:12:46
And backslash goes
to the answer.
216
00:12:46 --> 00:12:53
It doesn't get there by
computing A inverse.
217
00:12:53 --> 00:12:56
And our examples will show how.
218
00:12:56 --> 00:13:05
So I'm giving A inverse a
little bad comments right now.
219
00:13:05 --> 00:13:08
So I maybe should finish that
sentence and then you'll
220
00:13:08 --> 00:13:11
see me turn around and
compute the darn thing.
221
00:13:11 --> 00:13:16
But why do we not
use A inverse?
222
00:13:16 --> 00:13:18
Two reasons.
223
00:13:18 --> 00:13:20
One is it's more expensive.
224
00:13:20 --> 00:13:23
If I have to compute A
inverse and multiply by b,
225
00:13:23 --> 00:13:26
that's taking too long.
226
00:13:26 --> 00:13:30
Second reason is A inverse
could easily be a
227
00:13:30 --> 00:13:32
full, dense matrix.
228
00:13:32 --> 00:13:34
All non-zero.
229
00:13:34 --> 00:13:37
Where A itself was
like, tridiagonal.
230
00:13:37 --> 00:13:42
So if A is tridiagonal, all the
numbers we need are there in
231
00:13:42 --> 00:13:47
three diagonals, we don't want
A inverse, you'll see,
232
00:13:47 --> 00:13:48
A inverse is full.
233
00:13:48 --> 00:13:51
So two reasons for
not using A inverse.
234
00:13:51 --> 00:13:58
Takes too long in the first
place even in the good case and
235
00:13:58 --> 00:14:03
often our matrix A has got lots
is zeroes that are not there in
236
00:14:03 --> 00:14:07
A inverse so we've wasted time.
237
00:14:07 --> 00:14:13
Nevertheless let me say
something about A inverse
238
00:14:13 --> 00:14:16
on the next board.
239
00:14:16 --> 00:14:17
I don't know if you
ever thought about
240
00:14:17 --> 00:14:20
the inverse matrix.
241
00:14:20 --> 00:14:22
Let me ask you this question.
242
00:14:22 --> 00:14:27
Suppose I use the command A\I.
243
00:14:27 --> 00:14:32
244
00:14:32 --> 00:14:33
What's that doing?
245
00:14:33 --> 00:14:37
That's putting the identity on
the right-hand side instead of
246
00:14:37 --> 00:14:41
a single vector b or instead of
two vectors b and c
247
00:14:41 --> 00:14:44
I'm now putting n.
248
00:14:44 --> 00:14:46
That's ok.
249
00:14:46 --> 00:14:48
Backslash will work with that.
250
00:14:48 --> 00:14:51
That's a shorthand for solving.
251
00:14:51 --> 00:14:56
This solves all
these equations.
252
00:14:56 --> 00:14:59
A, it'll get my different
answers, u_1, u_2.
253
00:15:00 --> 00:15:04
It'll get n different answers
from the n right-hand
254
00:15:04 --> 00:15:07
sides, the columns of I.
255
00:15:07 --> 00:15:09
Let me take to be three.
256
00:15:09 --> 00:15:12
0, 1, 0; 0, 0, 1.
257
00:15:12 --> 00:15:19
But I'll take n to be three.
258
00:15:19 --> 00:15:21
That's the identity.
259
00:15:21 --> 00:15:23
So this solves that equation.
260
00:15:23 --> 00:15:29
A\I will output u,
will output u.
261
00:15:29 --> 00:15:31
And here's the question.
262
00:15:31 --> 00:15:32
What have I got?
263
00:15:32 --> 00:15:38
If you have a matrix A, square
matrix and of course you have
264
00:15:38 --> 00:15:45
to create I as eye(3), that I
would be, this would be eye(3).
265
00:15:47 --> 00:15:52
Cleve Moler's lousy pun.
266
00:15:52 --> 00:15:56
So what would I get?
267
00:15:56 --> 00:15:57
What would I get?
268
00:15:57 --> 00:15:59
The inverse, yes.
269
00:15:59 --> 00:16:00
I'd get the inverse.
270
00:16:00 --> 00:16:04
I'd get the inverse matrix.
271
00:16:04 --> 00:16:06
A backslash-- why's that?
272
00:16:06 --> 00:16:11
Because what matrix solves,
what's the solution u to A
273
00:16:11 --> 00:16:13
times something equal I?
274
00:16:13 --> 00:16:17
The solution to this
equation is Au=I.
275
00:16:17 --> 00:16:20
276
00:16:20 --> 00:16:24
The solution to that equation
is the inverse matrix.
277
00:16:24 --> 00:16:28
u will be A inverse.
278
00:16:28 --> 00:16:33
So that is a pretty, I mean,
that's pretty, if I wanted
279
00:16:33 --> 00:16:35
A inverse that's a
good way to do it.
280
00:16:35 --> 00:16:40
Other ways to do it would be
inv(A) in MATLAB and other
281
00:16:40 --> 00:16:44
ways, but this is about
as good as you get.
282
00:16:44 --> 00:16:51
Do you see that you get the
inverse matrix that way?
283
00:16:51 --> 00:17:03
And it's worth giving
some words to that fact.
284
00:17:03 --> 00:17:05
How would I describe this?
285
00:17:05 --> 00:17:10
This is a set of three
different problems.
286
00:17:10 --> 00:17:14
I would describe <1, 0,
0>, that right-hand
287
00:17:14 --> 00:17:18
side as an impulse.
288
00:17:18 --> 00:17:19
That's an impulse.
289
00:17:19 --> 00:17:26
A delta vector with an impulse
in the first component.
290
00:17:26 --> 00:17:29
I'd call that an impulse
in the second component.
291
00:17:29 --> 00:17:31
I'd call that an impulse
in the third component.
292
00:17:31 --> 00:17:37
So my inputs are three
impulses and my outputs
293
00:17:37 --> 00:17:39
are u_1, u_2, u_3.
294
00:17:39 --> 00:17:42
What words might I use?
295
00:17:42 --> 00:17:46
I could call those
impulse response.
296
00:17:46 --> 00:17:48
If I were in Course 6,
I certainly would.
297
00:17:48 --> 00:17:52
These would be impulses, these
would be the responses to that
298
00:17:52 --> 00:17:55
impulse from our system.
299
00:17:55 --> 00:18:01
So those are impulse responses
but in linear algebra the words
300
00:18:01 --> 00:18:08
I would use, u_1, u_2, u_3 are
the columns of A
301
00:18:08 --> 00:18:08
inverse, right?
302
00:18:08 --> 00:18:12
That's what we just
said. u_1, u_2, u_3.
303
00:18:12 --> 00:18:15
So the columns of-- Let
me write that down.
304
00:18:15 --> 00:18:25
The columns of A inverse are
the responses, the u's,
305
00:18:25 --> 00:18:33
the solutions to the
impulses, to n impulses.
306
00:18:33 --> 00:18:40
And these are the columns of I.
307
00:18:40 --> 00:18:41
Do you see?
308
00:18:41 --> 00:18:45
Nothing I've said so far
is deep or anything.
309
00:18:45 --> 00:18:51
But it's just, this comes up so
much that it's nice to have
310
00:18:51 --> 00:18:53
different ways to
think about this.
311
00:18:53 --> 00:18:59
And actually, yeah, if I had to
solve by hand, if I had to find
312
00:18:59 --> 00:19:06
the inverse by hand I would
use elimination on this
313
00:19:06 --> 00:19:08
system of equations.
314
00:19:08 --> 00:19:10
I would take A and put
the identity next to it.
315
00:19:10 --> 00:19:15
I would do elimination a lot.
316
00:19:15 --> 00:19:17
Actually I'll put that in one.
317
00:19:17 --> 00:19:24
If I had to find the inverse I
would take this block matrix.
318
00:19:24 --> 00:19:28
Is that the first block
matrix we've seen?
319
00:19:28 --> 00:19:34
Block matrices are really, you
should just get familiar, when
320
00:19:34 --> 00:19:37
you're getting familiar with
matrices, they often
321
00:19:37 --> 00:19:38
come in blocks.
322
00:19:38 --> 00:19:41
So here's a three by three
block, here's a three by three
323
00:19:41 --> 00:19:44
block, the whole matrix
is three by six.
324
00:19:44 --> 00:19:48
I can go through the
elimination steps on it.
325
00:19:48 --> 00:19:52
If I really go haywire on
elimination and keep going and
326
00:19:52 --> 00:20:00
going and going all the way
until A gets to the identity,
327
00:20:00 --> 00:20:04
so I do elimination and I get
it triangular then I even clean
328
00:20:04 --> 00:20:07
out up above the pivots and
then I change all the pivots to
329
00:20:07 --> 00:20:10
one, I can get all the way
to the identity if the
330
00:20:10 --> 00:20:12
matrix is invertible.
331
00:20:12 --> 00:20:17
And what do you think will
show up in the right half?
332
00:20:17 --> 00:20:19
A inverse, yeah.
333
00:20:19 --> 00:20:23
So A inverse will
show up there.
334
00:20:23 --> 00:20:28
So in 18.06 I would explain,
go through examples of
335
00:20:28 --> 00:20:35
this computation just to
see A inverse appear.
336
00:20:35 --> 00:20:38
There's no reason for
us to go through long
337
00:20:38 --> 00:20:40
examples like that.
338
00:20:40 --> 00:20:45
M one three by three would be
worth doing, but in the big
339
00:20:45 --> 00:20:49
picture we're going
to use backslash.
340
00:20:49 --> 00:20:51
Questions or discussion.
341
00:20:51 --> 00:20:55
So this is the sort of overall
picture about the inverse.
342
00:20:55 --> 00:20:58
Well this is about the inverse.
343
00:20:58 --> 00:21:02
This was about elimination.
344
00:21:02 --> 00:21:04
I've gotta take a little
time on elimination.
345
00:21:04 --> 00:21:07
No it's just too important.
346
00:21:07 --> 00:21:11
It's sort of straightforward
and mechanical but it's like,
347
00:21:11 --> 00:21:13
too important to blow away.
348
00:21:13 --> 00:21:23
So let me remove that and
put something better there.
349
00:21:23 --> 00:21:25
So now I'm talking
about elimination.
350
00:21:25 --> 00:21:28
I'm talking about one equation.
351
00:21:28 --> 00:21:30
I'm back to Au=b.
352
00:21:30 --> 00:21:34
Au=b.
353
00:21:34 --> 00:21:41
And notice I'm using the letter
A rather than K or one of
354
00:21:41 --> 00:21:44
our special letters because
right now I don't know that
355
00:21:44 --> 00:21:46
that's a special matrix.
356
00:21:46 --> 00:21:50
In a minute the example I do
will be one of our special
357
00:21:50 --> 00:21:54
matrices of course, and
then I'll use it's letter.
358
00:21:54 --> 00:21:58
Just a word, though.
359
00:21:58 --> 00:22:02
I thought I would take-- you're
getting a lot of big picture
360
00:22:02 --> 00:22:05
here for a minute and then
we'll get into the details.
361
00:22:05 --> 00:22:08
I thought I would just,
like, this is an
362
00:22:08 --> 00:22:11
occasion to look ahead.
363
00:22:11 --> 00:22:17
To say a word about the big
picture of linear algebra.
364
00:22:17 --> 00:22:22
It's got four major
problems, linear algebra.
365
00:22:22 --> 00:22:25
And there are four commands
to solve those problems.
366
00:22:25 --> 00:22:27
And those commands,
why not know?
367
00:22:27 --> 00:22:30
So those commands are LU.
368
00:22:30 --> 00:22:36
I'm speaking about MABLAB
notation but Octave, Scilab,
369
00:22:36 --> 00:22:42
Python, R, all other
would have those.
370
00:22:42 --> 00:22:45
Would do these same things.
371
00:22:45 --> 00:22:53
So LU is the command that
produces this, but I didn't
372
00:22:53 --> 00:22:56
say what that is yet.
373
00:22:56 --> 00:22:57
Ooh.
374
00:22:57 --> 00:22:58
So that's my job, right.
375
00:22:58 --> 00:23:03
What does this mean?
376
00:23:03 --> 00:23:05
What's up there?
377
00:23:05 --> 00:23:10
So, okay, to nobody's surprise
MATLAB thought, ok, LU was
378
00:23:10 --> 00:23:12
a good letter for that.
379
00:23:12 --> 00:23:15
And what did MATLAB think of
as a good letter for the
380
00:23:15 --> 00:23:17
command that does this?
381
00:23:17 --> 00:23:18
QR.
382
00:23:18 --> 00:23:30
So if I did lu(A) or qr(A) I
would get-- I mean, this is
383
00:23:30 --> 00:23:34
sometimes associated with
the names of Gram-Schmidt.
384
00:23:34 --> 00:23:36
It makes vectors orthogonal.
385
00:23:36 --> 00:23:39
Not to worry about this stuff.
386
00:23:39 --> 00:23:44
You can like, close your eyes
for a moment here. lu is the
387
00:23:44 --> 00:23:48
first command and that's what
today's about. qr is the key
388
00:23:48 --> 00:23:51
command for least
squares problems.
389
00:23:51 --> 00:23:55
Maybe the biggest application
of rectangular matrices,
390
00:23:55 --> 00:23:57
I'm sure that's the big.
391
00:23:57 --> 00:24:00
Eigenvalues, do you know
about eigenvalues?
392
00:24:00 --> 00:24:04
Well we'll just name
the command eig(A).
393
00:24:04 --> 00:24:09
And the singular value D
composition, which you may
394
00:24:09 --> 00:24:18
never have heard of, but
you will, is svd(A).
395
00:24:18 --> 00:24:21
Can I leave that?
396
00:24:21 --> 00:24:23
It's in the videotape now.
397
00:24:23 --> 00:24:32
And my point is that when we've
spoken about those four, we
398
00:24:32 --> 00:24:36
really have got numerical
linear algebra and a lot of
399
00:24:36 --> 00:24:42
pure linear algebra explained.
400
00:24:42 --> 00:24:44
These are the four big ones.
401
00:24:44 --> 00:24:49
And I guess what I'm saying is,
the four big problems of linear
402
00:24:49 --> 00:24:56
algebra turn out to, a good way
to describe the answer is as a
403
00:24:56 --> 00:24:59
factorization of the matrix.
404
00:24:59 --> 00:25:00
This is a factor.
405
00:25:00 --> 00:25:02
So now let me say
what this one is.
406
00:25:02 --> 00:25:08
I start with a matrix and what
elimination is really doing, if
407
00:25:08 --> 00:25:11
you look to see what is it
doing, it's producing a lower
408
00:25:11 --> 00:25:15
triangular times an
upper triangular.
409
00:25:15 --> 00:25:17
Let's go directly to that.
410
00:25:17 --> 00:25:18
Let me go directly to that.
411
00:25:18 --> 00:25:21
Let me take an example.
412
00:25:21 --> 00:25:22
So here's my matrix.
413
00:25:22 --> 00:25:34
Well, I don't have to call it
A because you recognize it.
414
00:25:34 --> 00:25:37
So what's our name
for that matrix?
415
00:25:37 --> 00:25:37
T.
416
00:25:37 --> 00:25:41
T because the top boundary
condition is free.
417
00:25:41 --> 00:25:44
Oh, that reminds me.
418
00:25:44 --> 00:25:49
Some good comments after class
Friday brought out something
419
00:25:49 --> 00:25:51
that I sloughed over.
420
00:25:51 --> 00:25:58
That the free-fixed matrix,
the free-fixed problem is
421
00:25:58 --> 00:26:02
usually one unknown larger
than the fixed-fixed.
422
00:26:02 --> 00:26:04
Because remember the
fixed-fixed problem
423
00:26:04 --> 00:26:05
had both ends fixed.
424
00:26:05 --> 00:26:07
They were not unknowns.
425
00:26:07 --> 00:26:12
The only unknowns were one,
two, three to n in the middle.
426
00:26:12 --> 00:26:16
But people noticed when I was
talking about the free boundary
427
00:26:16 --> 00:26:21
condition that u_0 came into
it and u_0 is not known.
428
00:26:21 --> 00:26:26
So really, the free boundary
condition like, has an extra
429
00:26:26 --> 00:26:30
unknown, an extra row and
column in the matrix
430
00:26:30 --> 00:26:33
and that's correct.
431
00:26:33 --> 00:26:38
We'll see later in Fourier
transforms cosine matrices
432
00:26:38 --> 00:26:42
are one size bigger
than sine matrices.
433
00:26:42 --> 00:26:47
The cosine matrices are
free-free and the sine
434
00:26:47 --> 00:26:48
matrices are fixed-fixed.
435
00:26:48 --> 00:26:54
And now here we're
at free-fixed.
436
00:26:54 --> 00:26:59
I want to do elimination
on that matrix.
437
00:26:59 --> 00:27:03
And while I'm at it,
we'll find the inverse.
438
00:27:03 --> 00:27:08
But let's do elimination.
439
00:27:08 --> 00:27:10
Just on that matrix.
440
00:27:10 --> 00:27:15
Just to see what this
L and U stuff is.
441
00:27:15 --> 00:27:19
What do we do?
442
00:27:19 --> 00:27:21
The first pivot is?
443
00:27:21 --> 00:27:24
One, it's fine.
444
00:27:24 --> 00:27:27
Not going to worry about that.
445
00:27:27 --> 00:27:28
We'll use it now.
446
00:27:28 --> 00:27:29
So how do I use it?
447
00:27:29 --> 00:27:33
I use a pivot, now listen
because here is a convention
448
00:27:33 --> 00:27:36
here, I'm going to use
the word subtract.
449
00:27:36 --> 00:27:41
You would say add that
row to that row, right?
450
00:27:41 --> 00:27:44
Because you want to
get a zero here.
451
00:27:44 --> 00:27:46
Forgive me for making
it sound harder.
452
00:27:46 --> 00:27:50
I'm going to say subtract
because I like subtraction.
453
00:27:50 --> 00:27:54
Subtract minus one of
that row, my multiplier
454
00:27:54 --> 00:27:56
is minus one here.
455
00:27:56 --> 00:28:00
I'm going to say subtract minus
one of that row from that.
456
00:28:00 --> 00:28:02
Same thing.
457
00:28:02 --> 00:28:04
You'll say okay.
458
00:28:04 --> 00:28:05
No problem.
459
00:28:05 --> 00:28:07
Let's just do it.
460
00:28:07 --> 00:28:14
So there's the pivot row and
now when I-- shall I just add?
461
00:28:14 --> 00:28:25
When I add that to or does my
superego thing subtract minus
462
00:28:25 --> 00:28:27
one of that from that.
463
00:28:27 --> 00:28:28
What do I get?
464
00:28:28 --> 00:28:33
I get the zero, the one
and the minus one.
465
00:28:33 --> 00:28:34
And then what do I get?
466
00:28:34 --> 00:28:36
What's the multiplier?
467
00:28:36 --> 00:28:42
So let's just put these L,
these multipliers, the l_21.
468
00:28:42 --> 00:28:44
That's the multiplier.
469
00:28:44 --> 00:28:48
2, 1 refers to row
two, column one.
470
00:28:48 --> 00:28:51
And this step got the zero
in row two, column one.
471
00:28:51 --> 00:28:54
And what was the
multiplier that did it?
472
00:28:54 --> 00:28:58
It's the number that I
multiplied row one by and
473
00:28:58 --> 00:29:02
subtracted from row two,
so it was minus one.
474
00:29:02 --> 00:29:05
What's l_31?
475
00:29:05 --> 00:29:10
What's the multiplier that
produces a zero in the three,
476
00:29:10 --> 00:29:13
row three, column one position?
477
00:29:13 --> 00:29:15
It's zero.
478
00:29:15 --> 00:29:21
I take zero of this row
away from this row because
479
00:29:21 --> 00:29:23
it's zero already.
480
00:29:23 --> 00:29:26
So I'm not going to change,
that row won't change
481
00:29:26 --> 00:29:30
and l_31 was zero.
482
00:29:30 --> 00:29:35
Now I know the next pivot.
483
00:29:35 --> 00:29:36
I'm ready to use it.
484
00:29:36 --> 00:29:39
I want to get a zero below
it because I'm aiming at
485
00:29:39 --> 00:29:42
this upper triangular u.
486
00:29:42 --> 00:29:45
And what's the multiplier now?
487
00:29:45 --> 00:29:47
And what's its number?
488
00:29:47 --> 00:29:49
What's the multiplier number?
489
00:29:49 --> 00:29:55
3, 2 because I'm trying to
fix row three, column two.
490
00:29:55 --> 00:29:58
And what do I multiply
this by and subtract from
491
00:29:58 --> 00:30:02
this to make it zero?
492
00:30:02 --> 00:30:04
It's negative one again.
493
00:30:04 --> 00:30:05
Negative one, right.
494
00:30:05 --> 00:30:06
It's negative one.
495
00:30:06 --> 00:30:08
Sorry.
496
00:30:08 --> 00:30:09
Right.
497
00:30:09 --> 00:30:11
And now what happens
when I do that?
498
00:30:11 --> 00:30:16
Can I just do it in place here?
499
00:30:16 --> 00:30:19
Forgive me if I just
add that to that.
500
00:30:19 --> 00:30:25
And I'll get zero and one.
501
00:30:25 --> 00:30:31
And now what do I know at this
point, what have I learned?
502
00:30:31 --> 00:30:32
The most important thing
I've learned is the
503
00:30:32 --> 00:30:34
matrix is invertible.
504
00:30:34 --> 00:30:39
Because the pivots one, one,
and one, well they're all
505
00:30:39 --> 00:30:40
here on the diagonal.
506
00:30:40 --> 00:30:42
This is my matrix U.
507
00:30:42 --> 00:30:45
That's my upper
triangular matrix.
508
00:30:45 --> 00:30:48
And-- yeah, of course?
509
00:30:48 --> 00:31:05
I'm subtracting from this, so
I've got the two is there,
510
00:31:05 --> 00:31:06
yeah, yeah, that's right.
511
00:31:06 --> 00:31:11
So the two is sitting there and
I'm subtracting minus one of
512
00:31:11 --> 00:31:16
that row from it and that would
mean taking, yeah, yeah.
513
00:31:16 --> 00:31:18
Right.
514
00:31:18 --> 00:31:21
That would mean I'm subtracting
one from the two and
515
00:31:21 --> 00:31:21
getting the one.
516
00:31:21 --> 00:31:22
Yeah, yeah.
517
00:31:22 --> 00:31:27
So the row, the typical entry
is, the typical result is the
518
00:31:27 --> 00:31:31
row you have minus L
times the pivot row.
519
00:31:31 --> 00:31:34
The row you have minus the
multiplier times the pivot row.
520
00:31:34 --> 00:31:39
That's the operation that
elimination lives on.
521
00:31:39 --> 00:31:42
Elimination does
that all the time.
522
00:31:42 --> 00:31:45
It's one of the basic linear
algebra subroutines.
523
00:31:45 --> 00:31:48
B L A S.
524
00:31:48 --> 00:31:52
Now, this is my U.
525
00:31:52 --> 00:31:55
So that's the goal
of elimination, get
526
00:31:55 --> 00:31:57
upper triangular.
527
00:31:57 --> 00:31:59
And the reason is, you can
solve upper triangular
528
00:31:59 --> 00:32:02
systems really fast.
529
00:32:02 --> 00:32:08
These multipliers l, l_21,
2, and so on, they and
530
00:32:08 --> 00:32:12
go into the L matrix.
531
00:32:12 --> 00:32:17
And now, let me just say it
here that in a way, that
532
00:32:17 --> 00:32:20
example is too beautiful.
533
00:32:20 --> 00:32:25
Seldom am I sorry to see an
example come out beautifully,
534
00:32:25 --> 00:32:28
but why do I say this
is too beautiful?
535
00:32:28 --> 00:32:30
It's not typical.
536
00:32:30 --> 00:32:34
If I had other numbers here, I
would get to other numbers here
537
00:32:34 --> 00:32:37
and what would be the
difference, typically?
538
00:32:37 --> 00:32:44
The pivots wouldn't
be all ones.
539
00:32:44 --> 00:32:48
That's what's too beautiful
here, but let's go with it.
540
00:32:48 --> 00:32:52
I mean, it was worth it because
everything came out simple.
541
00:32:52 --> 00:32:57
But the pivots for another
problem, ooh, let me just
542
00:32:57 --> 00:33:03
do a second problem here.
543
00:33:03 --> 00:33:06
I'll do the fixed-fixed guy.
544
00:33:06 --> 00:33:09
Ok, so let's just do
elimination on that.
545
00:33:09 --> 00:33:11
That's the first pivot.
546
00:33:11 --> 00:33:12
Subtract.
547
00:33:12 --> 00:33:17
Now what's the multiplier now?
548
00:33:17 --> 00:33:18
You're not as quick as MATLAB.
549
00:33:18 --> 00:33:20
MATLAB is ahead of you.
550
00:33:20 --> 00:33:26
So the multiplier
is negative 1/2.
551
00:33:26 --> 00:33:32
So the multiplier is, l_21 is
negative 1/2. l_31 will again
552
00:33:32 --> 00:33:36
be zero and let's use it, so
it knocks that guys out.
553
00:33:36 --> 00:33:39
And what did that
number come out to be?
554
00:33:39 --> 00:33:40
Do you remember?
555
00:33:40 --> 00:33:42
That was 3/2.
556
00:33:42 --> 00:33:44
I think we looked at that once.
557
00:33:44 --> 00:33:46
And that would be all the same.
558
00:33:46 --> 00:33:52
And then the next multiplier,
l_32 will be negative 2/3
559
00:33:52 --> 00:33:56
because when I multiply that by
2/3 it gives me the negative
560
00:33:56 --> 00:34:02
one and then I subtract and it
kills this and I get 4/3.
561
00:34:02 --> 00:34:04
I just did that quickly.
562
00:34:04 --> 00:34:11
And my main point was the
pivots are on the diagonal.
563
00:34:11 --> 00:34:13
They're not all ones now.
564
00:34:13 --> 00:34:15
So this is a more typical one.
565
00:34:15 --> 00:34:18
This is, again our u.
566
00:34:18 --> 00:34:27
And our L matrix will be,
oh, oh, that's the point.
567
00:34:27 --> 00:34:30
That these l's, these
multipliers fit right into
568
00:34:30 --> 00:34:32
a lower triangular matrix.
569
00:34:32 --> 00:34:37
All these multipliers, and
we'll put ones on the diagonal
570
00:34:37 --> 00:34:42
of that guy and these lower
triangular ones will fit in
571
00:34:42 --> 00:34:45
just right perfectly in there.
572
00:34:45 --> 00:34:51
Over here the L would be,
let me construct the L.
573
00:34:51 --> 00:34:55
Ones on the diagonal
representing the pivot rows
574
00:34:55 --> 00:35:02
that stayed put and minus one,
zero, and minus one as the
575
00:35:02 --> 00:35:10
multipliers that, so this was
L, the multipliers
576
00:35:10 --> 00:35:16
that we used.
577
00:35:16 --> 00:35:18
One more.
578
00:35:18 --> 00:35:23
We're doing lots of good stuff
here and it's not deep, but
579
00:35:23 --> 00:35:29
it's-- Suppose the matrix
had been singular.
580
00:35:29 --> 00:35:34
We have to realize, okay, this
elimination method is great.
581
00:35:34 --> 00:35:38
But it can break down and it's
going to break down, it has
582
00:35:38 --> 00:35:41
to break down somehow if
the matrix is singular.
583
00:35:41 --> 00:35:46
Now what's our example of
a singular matrix here?
584
00:35:46 --> 00:35:51
The matrix, this is free-fixed
and that by fixing one support
585
00:35:51 --> 00:35:53
it wasn't singular, but if I
want to make it singular,
586
00:35:53 --> 00:35:55
what'll I take?
587
00:35:55 --> 00:35:57
Free-free.
588
00:35:57 --> 00:35:59
Free-free matrix.
589
00:35:59 --> 00:36:04
So can I, if I had thought to
bring colored chalk, I'll just
590
00:36:04 --> 00:36:08
erase for a moment
for the bad case.
591
00:36:08 --> 00:36:12
The bad case would
be free-free.
592
00:36:12 --> 00:36:16
And how would it show up
as bad in elimination.
593
00:36:16 --> 00:36:22
How does a singular
matrix reveal itself as
594
00:36:22 --> 00:36:24
elimination goes forward?
595
00:36:24 --> 00:36:30
Because you can't tell
at the beginning.
596
00:36:30 --> 00:36:30
What would have gone wrong?
597
00:36:30 --> 00:36:35
We would have had a zero there.
598
00:36:35 --> 00:36:38
We had a two that
dropped to one.
599
00:36:38 --> 00:36:41
But if we start with a
one, it'll drop to zero.
600
00:36:41 --> 00:36:43
That would have
been a zero there.
601
00:36:43 --> 00:36:49
The matrix would not
have had three pivots.
602
00:36:49 --> 00:36:53
This upper triangular matrix
is singular, no good.
603
00:36:53 --> 00:36:55
And that tells us back there
that the original matrix
604
00:36:55 --> 00:36:58
is singular, no good.
605
00:36:58 --> 00:37:04
So if I can't get to three
pivots somehow, the matrix'll
606
00:37:04 --> 00:37:09
be singular and that's
an example that is.
607
00:37:09 --> 00:37:12
And MATLAB would immediately
tell us, of course.
608
00:37:12 --> 00:37:17
So let's go back to the good
case for the main point.
609
00:37:17 --> 00:37:19
The good case for
the main point.
610
00:37:19 --> 00:37:26
So the good case
was three pivots.
611
00:37:26 --> 00:37:28
In fact it was extra
good because they all
612
00:37:28 --> 00:37:30
turned out to be ones.
613
00:37:30 --> 00:37:36
Now, oh, now we're
ready for LU.
614
00:37:36 --> 00:37:38
Here's the magic.
615
00:37:38 --> 00:37:40
And I'm not giving a proof.
616
00:37:40 --> 00:37:48
The magic is that the result U,
if I multiply the multiplier
617
00:37:48 --> 00:37:55
matrix L times the result
U, I'll bring back A.
618
00:37:55 --> 00:37:57
I'll bring back A.
619
00:37:57 --> 00:37:59
So let me just see.
620
00:37:59 --> 00:38:05
If I multiply L by U, so this
is now L times U, maybe
621
00:38:05 --> 00:38:07
you can see that I get A.
622
00:38:07 --> 00:38:09
So what is U?
623
00:38:09 --> 00:38:10
I just have to copy it.
624
00:38:10 --> 00:38:15
[1, 1, 1; -1, -1, 0].
625
00:38:15 --> 00:38:22
I could fill in the zeroes
but I know they're there.
626
00:38:22 --> 00:38:25
That's L times U.
627
00:38:25 --> 00:38:28
And sure enough, if I do the
multiplication, this-- How
628
00:38:28 --> 00:38:29
would you to that
multiplication?
629
00:38:29 --> 00:38:32
I would say this is one of the
first row when I see that.
630
00:38:32 --> 00:38:34
1, 0, 0 multiplying these.
631
00:38:34 --> 00:38:37
I'd say get one of
the first row.
632
00:38:37 --> 00:38:39
That's correct in A.
633
00:38:39 --> 00:38:43
Here I would say this is minus
one of the first row, plus one
634
00:38:43 --> 00:38:46
of the second row, and
sure enough it's the
635
00:38:46 --> 00:38:48
right part of A.
636
00:38:48 --> 00:38:51
And this is minus one of the
second row, plus one of the
637
00:38:51 --> 00:38:54
third row, and sure enough it's
the right third row of A.
638
00:38:54 --> 00:38:57
I get A.
639
00:38:57 --> 00:39:01
And that's when elimination
goes through with no zero
640
00:39:01 --> 00:39:07
pivots, no problems, just a
bunch of multipliers, then
641
00:39:07 --> 00:39:15
that wonderful description
of it, A=LU is correct.
642
00:39:15 --> 00:39:17
I don't know how
many that's new to.
643
00:39:17 --> 00:39:19
I should maybe have
thought ahead.
644
00:39:19 --> 00:39:22
How many have seen like,
L times U before?
645
00:39:22 --> 00:39:24
Just to give me an idea?
646
00:39:24 --> 00:39:25
Quite a few.
647
00:39:25 --> 00:39:25
Ok.
648
00:39:25 --> 00:39:31
So it's terrific.
649
00:39:31 --> 00:39:34
Oh, here I would get L times U.
650
00:39:34 --> 00:39:36
Now this is like a little
more interesting because
651
00:39:36 --> 00:39:43
the pivots were not ones.
652
00:39:43 --> 00:39:46
So that's my matrix U.
653
00:39:46 --> 00:39:49
And here's my matrix L, right?
654
00:39:49 --> 00:39:51
Okay, big point.
655
00:39:51 --> 00:39:56
Because we're so interested in
symmetric matrices and this one
656
00:39:56 --> 00:40:00
in particular, or that one,
symmetric matrices are good.
657
00:40:00 --> 00:40:05
Now, I'm unhappy
about one aspect.
658
00:40:05 --> 00:40:10
So now there's just
one part of this.
659
00:40:10 --> 00:40:14
This was great, we got three
non-zero pivots, we got to U,
660
00:40:14 --> 00:40:20
we got the multiplier matrix
all fine and we would be ready
661
00:40:20 --> 00:40:23
for the right-hand side and we
would be ready for two
662
00:40:23 --> 00:40:25
right-hand sides, we would even
be ready for all three
663
00:40:25 --> 00:40:28
right-hand sides, whatever.
664
00:40:28 --> 00:40:32
But I have one criticism.
665
00:40:32 --> 00:40:37
The matrix A which was
our K, this was really
666
00:40:37 --> 00:40:40
K, was symmetric.
667
00:40:40 --> 00:40:43
That was the very first
thing you did, told me
668
00:40:43 --> 00:40:45
on the very first day.
669
00:40:45 --> 00:40:48
And now it's equal to L times
U, but what's happened?
670
00:40:48 --> 00:40:51
The symmetry is lost.
671
00:40:51 --> 00:40:57
Somehow the L has ones on the
diagonal, the U as we have it
672
00:40:57 --> 00:41:01
has pivots on the diagonal, now
the pivots are not all ones.
673
00:41:01 --> 00:41:06
So you see the symmetry of
the problem got lost, and
674
00:41:06 --> 00:41:07
that shouldn't happen.
675
00:41:07 --> 00:41:10
And there ought to be
a way to get back.
676
00:41:10 --> 00:41:11
Ok.
677
00:41:11 --> 00:41:15
And now I want to describe the
way to get back to symmetry.
678
00:41:15 --> 00:41:19
So LU doesn't keep
the symmetry.
679
00:41:19 --> 00:41:22
L has ones, U has pivots.
680
00:41:22 --> 00:41:24
Different.
681
00:41:24 --> 00:41:29
But a very simple idea will
bring back the symmetry.
682
00:41:29 --> 00:41:37
That is peel off the pivots
into a diagonal matrix.
683
00:41:37 --> 00:41:40
In other words, there's
a matrix, I'll call
684
00:41:40 --> 00:41:45
it D, D for diagonal.
685
00:41:45 --> 00:41:49
I'll divide those numbers
out of each row.
686
00:41:49 --> 00:41:53
And can I just do that?
687
00:41:53 --> 00:41:57
So I'm just going to write this
U as a product of this diagonal
688
00:41:57 --> 00:42:00
D where I'm going to be
dividing the two out.
689
00:42:00 --> 00:42:03
So when I divide the two
out from that row I'm left
690
00:42:03 --> 00:42:06
with one, minus 1/2, zero.
691
00:42:06 --> 00:42:11
And when I divide 3/2, the
pivot then, it makes that pivot
692
00:42:11 --> 00:42:15
into a one and what does
it produce for that guy?
693
00:42:15 --> 00:42:19
When I divide 3/2, when
I divide that minus one
694
00:42:19 --> 00:42:22
by 3/2, what do I get?
695
00:42:22 --> 00:42:25
I get negative, division
will be 2/3, I'll
696
00:42:25 --> 00:42:28
get a negative 2/3.
697
00:42:28 --> 00:42:32
And now, on the last row I'm
dividing that row by 4/3.
698
00:42:32 --> 00:42:35
When I divide that row
by 4/3, what row do I
699
00:42:35 --> 00:42:38
get here? .
700
00:42:38 --> 00:42:42
Because I've made the pivots
one, well they're not pivots.
701
00:42:42 --> 00:42:45
What I've done is
separate out the pivots.
702
00:42:45 --> 00:42:51
So I've made the diagonal ones
just by separating it out.
703
00:42:51 --> 00:42:58
And what's happened?
704
00:42:58 --> 00:43:02
My goal was to get back
some symmetry that was
705
00:43:02 --> 00:43:03
there at the start.
706
00:43:03 --> 00:43:10
Now so I have a pivot matrix
D, and what's that matrix?
707
00:43:10 --> 00:43:12
You could say, well,
it's the rest.
708
00:43:12 --> 00:43:15
But that's not what
I'm looking for.
709
00:43:15 --> 00:43:17
What is it?
710
00:43:17 --> 00:43:19
Can everybody have
a look at it?
711
00:43:19 --> 00:43:21
I can't raise it.
712
00:43:21 --> 00:43:26
If you look at what we
got there, what is it?
713
00:43:26 --> 00:43:29
What's the right
name to give it?
714
00:43:29 --> 00:43:30
L transpose, exactly!
715
00:43:30 --> 00:43:32
That's the right name.
716
00:43:32 --> 00:43:34
L transpose.
717
00:43:34 --> 00:43:37
So what am I concluding then?
718
00:43:37 --> 00:43:42
I'm concluding that, let's
see, where shall I put this?
719
00:43:42 --> 00:43:43
And it'll come back to it.
720
00:43:43 --> 00:43:49
Well, here we had just to show
it wasn't an accident, here we
721
00:43:49 --> 00:43:54
had L, L transpose and what
was the pivot matrix in this
722
00:43:54 --> 00:43:56
too beautiful problem case?
723
00:43:56 --> 00:43:58
It was the identity.
724
00:43:58 --> 00:44:00
So we didn't notice it.
725
00:44:00 --> 00:44:02
So can I squeeze
in the identity?
726
00:44:02 --> 00:44:04
That's the pivot matrix there.
727
00:44:04 --> 00:44:08
But and again, we had
L times L transpose.
728
00:44:08 --> 00:44:11
The beauty was there,
the symmetry was there.
729
00:44:11 --> 00:44:13
And now what's the usual thing?
730
00:44:13 --> 00:44:21
So really I'm completing
this to one more thought.
731
00:44:21 --> 00:44:24
In that case when
A is symmetric.
732
00:44:24 --> 00:44:26
I'm completing, I have the L.
733
00:44:26 --> 00:44:28
I'm factoring out the D.
734
00:44:28 --> 00:44:33
And what's left is L transpose.
735
00:44:33 --> 00:44:39
I hope you like LD*L transpose.
736
00:44:39 --> 00:44:45
Seeing a matrix on one side and
the transpose on the other
737
00:44:45 --> 00:44:49
side, the matrix L at the left
and L transpose at the
738
00:44:49 --> 00:44:52
right is just right.
739
00:44:52 --> 00:45:10
So the point of symmetric case
we have, and I'll use the
740
00:45:10 --> 00:45:13
letter K rather than A because
now we're getting the matrix
741
00:45:13 --> 00:45:14
that's more special.
742
00:45:14 --> 00:45:19
It's that K or it's this
T or it's any other
743
00:45:19 --> 00:45:20
symmetric matrix.
744
00:45:20 --> 00:45:27
The elimination leads to, uses
multipliers L and if I factor
745
00:45:27 --> 00:45:33
out the pivot matrix then the
other part is L transpose.
746
00:45:33 --> 00:45:36
We've seen that
just by example.
747
00:45:36 --> 00:45:39
By two examples.
748
00:45:39 --> 00:45:42
Now I want to just
look at that.
749
00:45:42 --> 00:45:50
Because this that describes not
only the result of elimination
750
00:45:50 --> 00:45:56
which is the key operation, but
it also keeps the symmetry.
751
00:45:56 --> 00:46:03
In fact every matrix of
that sort is symmetric.
752
00:46:03 --> 00:46:07
No, yeah, that's important.
753
00:46:07 --> 00:46:13
This is sure to be symmetric.
754
00:46:13 --> 00:46:21
We will often see matrices
multiplied by their transpose.
755
00:46:21 --> 00:46:27
So what I'm saying is that if
you gave me any matrix L, any
756
00:46:27 --> 00:46:32
diagonal matrix D, and then the
transpose of L, if I multiplied
757
00:46:32 --> 00:46:34
those out, I would get
a symmetric matrix.
758
00:46:34 --> 00:46:38
And going the other way, if I
started with a symmetric matrix
759
00:46:38 --> 00:46:43
and I did elimination and got
an L, then the D factoring out
760
00:46:43 --> 00:46:44
would leave me L transpose.
761
00:46:44 --> 00:46:49
So what you've seen by
example is what will
762
00:46:49 --> 00:46:51
happen all the time.
763
00:46:51 --> 00:46:57
Now why is that
matrix symmetric?
764
00:46:57 --> 00:47:01
Here we get a chance to
show the power of matrix
765
00:47:01 --> 00:47:03
notation, really.
766
00:47:03 --> 00:47:08
I just think that if I have any
matrix L, in this case it
767
00:47:08 --> 00:47:10
happened to be lower
triangular, but if I have any
768
00:47:10 --> 00:47:14
matrix L and I have a nice,
symmetric diagonal guy in the
769
00:47:14 --> 00:47:18
middle and I have the transpose
of this matrix on the other
770
00:47:18 --> 00:47:24
side I think the result is a
symmetric matrix
771
00:47:24 --> 00:47:25
when I multiply.
772
00:47:25 --> 00:47:33
So it's these symmetric
factorizations that we're
773
00:47:33 --> 00:47:36
getting to and are important
problems because our important
774
00:47:36 --> 00:47:39
problems are symmetric.
775
00:47:39 --> 00:47:42
Ok, why is that sure
to be symmetric?
776
00:47:42 --> 00:47:49
Suppose I asked you as a
exercise, prove that L times a
777
00:47:49 --> 00:47:55
diagonal times L transpose is
always a symmetric matrix.
778
00:47:55 --> 00:47:57
How could you do that?
779
00:47:57 --> 00:47:59
How could you do that?
780
00:47:59 --> 00:48:02
You could certainly create an
example that did it and check
781
00:48:02 --> 00:48:05
it out, multiply,
it would work.
782
00:48:05 --> 00:48:11
But we want to see that this
is going to be true always.
783
00:48:11 --> 00:48:16
So how would you do that?
784
00:48:16 --> 00:48:19
I guess, let me get started.
785
00:48:19 --> 00:48:22
I would take its transpose.
786
00:48:22 --> 00:48:26
If I want to show something's
symmetric, I transpose it
787
00:48:26 --> 00:48:28
and see if I get the
same matrix again.
788
00:48:28 --> 00:48:32
So let me take the
transpose of this.
789
00:48:32 --> 00:48:35
So I'm answering, Why is
it sure to be symmetric?
790
00:48:35 --> 00:48:37
So let me take K transpose.
791
00:48:37 --> 00:48:42
So this is the transpose of, I
have K equals something times
792
00:48:42 --> 00:48:48
something times something, A
times B times C, you could say.
793
00:48:48 --> 00:48:56
If I transpose a matrix, how
can I create transposes
794
00:48:56 --> 00:48:59
out of a, B and C.
795
00:48:59 --> 00:49:02
Do you remember what happens?
796
00:49:02 --> 00:49:04
They reverse the order.
797
00:49:04 --> 00:49:05
It's like inverses.
798
00:49:05 --> 00:49:14
Transposes and inverses
both have that key rule.
799
00:49:14 --> 00:49:18
When you have a product and
you invert it, they come
800
00:49:18 --> 00:49:19
in the opposite order.
801
00:49:19 --> 00:49:22
When you transpose it, they
come in the opposite order.
802
00:49:22 --> 00:49:26
So let me try put these
separate transposes in
803
00:49:26 --> 00:49:31
the opposite order.
804
00:49:31 --> 00:49:34
So I've used the most
important fact there.
805
00:49:34 --> 00:49:43
Which is just a fact about
transposing a product.
806
00:49:43 --> 00:49:47
Ok, what have I got now?
807
00:49:47 --> 00:49:50
What's L transpose transposed?
808
00:49:50 --> 00:49:52
It's L, great.
809
00:49:52 --> 00:49:54
What's L transpose
L transposed?
810
00:49:54 --> 00:49:55
Nothing but L.
811
00:49:55 --> 00:49:57
Transpose twice and
I'm back to L.
812
00:49:57 --> 00:50:00
What about D transpose?
813
00:50:00 --> 00:50:01
Same as D.
814
00:50:01 --> 00:50:04
Because D was symmetric,
in fact diagonal.
815
00:50:04 --> 00:50:08
So what have I learned?
816
00:50:08 --> 00:50:11
The proof is done.
817
00:50:11 --> 00:50:13
I've got K back again.
818
00:50:13 --> 00:50:15
This was the original K.
819
00:50:15 --> 00:50:17
So I've learned that
K transpose is K.
820
00:50:17 --> 00:50:22
So you're going to see time
after time, let me just
821
00:50:22 --> 00:50:25
put these things there,
you're going to see an
822
00:50:25 --> 00:50:28
A transpose times an A.
823
00:50:28 --> 00:50:32
That's the most important,
most highly important
824
00:50:32 --> 00:50:33
multiplication.
825
00:50:33 --> 00:50:36
Take a matrix, maybe
rectangular, multiply
826
00:50:36 --> 00:50:38
by A transpose.
827
00:50:38 --> 00:50:44
So this matrix is certainly
square because A
828
00:50:44 --> 00:50:47
could be m by n.
829
00:50:47 --> 00:50:51
And then A transpose
would be n by m and the
830
00:50:51 --> 00:50:55
result would be n by n.
831
00:50:55 --> 00:50:57
So it's certainly square.
832
00:50:57 --> 00:51:00
But now what's the new
property we now know?
833
00:51:00 --> 00:51:02
It's symmetric.
834
00:51:02 --> 00:51:06
It's symmetric because if I
transpose it, the transpose of
835
00:51:06 --> 00:51:09
A will go on this side, the
double transpose will go on
836
00:51:09 --> 00:51:12
this side, but the double
transpose is A again,
837
00:51:12 --> 00:51:16
so symmetric.
838
00:51:16 --> 00:51:24
So I'm plugging away here on
symmetric matrices because
839
00:51:24 --> 00:51:31
they're just-- yeah, what does
symmetry mean in, yeah, can I
840
00:51:31 --> 00:51:42
just come back to this
idea of responses?
841
00:51:42 --> 00:51:50
And by the way, if this
was symmetric, would it's
842
00:51:50 --> 00:51:52
inverse be symmetric?
843
00:51:52 --> 00:51:54
The answer is yes.
844
00:51:54 --> 00:51:59
If a matrix is symmetric,
it's inverse is symmetric.
845
00:51:59 --> 00:52:04
These symmetric matrices
are a fantastic family.
846
00:52:04 --> 00:52:07
So I could add that to this.
847
00:52:07 --> 00:52:17
K inverse will also symmetric
without having yet said why.
848
00:52:17 --> 00:52:19
But maybe in words, I'll
just say a few words
849
00:52:19 --> 00:52:27
here at the end.
850
00:52:27 --> 00:52:31
So what's a typical entry?
851
00:52:31 --> 00:52:36
Say the 2, 1 entry, just
to carry on with this
852
00:52:36 --> 00:52:39
language one more moment.
853
00:52:39 --> 00:52:43
This is A inverse here.
854
00:52:43 --> 00:52:49
Now the 2, 1 entry in the
inverse is an impulse is in
855
00:52:49 --> 00:52:56
the first, the first mass,
whatever gets an impulse.
856
00:52:56 --> 00:53:01
And that is the response
of the second mass.
857
00:53:01 --> 00:53:06
The response in position two to
the impulse in position one.
858
00:53:06 --> 00:53:09
Now my matrix is symmetric,
thinking about symmetric
859
00:53:09 --> 00:53:10
matrices here.
860
00:53:10 --> 00:53:12
So what about here?
861
00:53:12 --> 00:53:17
Here, if I take the impulse in
position two and look at the
862
00:53:17 --> 00:53:22
response in position one, so
do you see the difference?
863
00:53:22 --> 00:53:24
In general, those
could be different.
864
00:53:24 --> 00:53:28
This is the response
at position two to
865
00:53:28 --> 00:53:30
an impulse at one.
866
00:53:30 --> 00:53:34
This is the response at
one to an impulse at two.
867
00:53:34 --> 00:53:37
You see that I'm multiplying
those columns and
868
00:53:37 --> 00:53:38
getting these columns.
869
00:53:38 --> 00:53:42
And what's the point
about symmetry?
870
00:53:42 --> 00:53:43
Those are the same.
871
00:53:43 --> 00:53:49
Symmetry is expressing this
physical meaning that the
872
00:53:49 --> 00:53:54
response that i, to an impulse
at j is the same as the
873
00:53:54 --> 00:53:58
response at j to
an impulse at i.
874
00:53:58 --> 00:54:03
And that's sort of, that's
such an important property,
875
00:54:03 --> 00:54:06
you want to notice it.
876
00:54:06 --> 00:54:09
And it goes into symmetry.
877
00:54:09 --> 00:54:14
So many, many problems will be
symmetric and then some won't.
878
00:54:14 --> 00:54:19
We'll have to admit this won't
cover everything, but it
879
00:54:19 --> 00:54:22
covers such an important
and beautiful part.
880
00:54:22 --> 00:54:30
So that's today's lecture on LU
elimination, solving linear
881
00:54:30 --> 00:54:38
systems, and then let's move
forward to understanding
882
00:54:38 --> 00:54:40
the actual inverses.