1
00:00:00 --> 00:00:01
 
 

2
00:00:01 --> 00:00:02
The following content is
provided under a creative

3
00:00:03 --> 00:00:03
commons license.
 

4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to

5
00:00:06 --> 00:00:09
offer high-quality educational
resources for free.

6
00:00:09 --> 00:00:12
To make a donation or to view
additional materials from

7
00:00:12 --> 00:00:16
hundreds of MIT courses visit
MIT OpenCourseWare

8
00:00:16 --> 00:00:19
at ocw.mit.edu.
 

9
00:00:19 --> 00:00:24
PROFESSOR STRANG: Starting
today on section 1.3, actually

10
00:00:24 --> 00:00:26
we'll finish that today.
 

11
00:00:26 --> 00:00:31
And that's about the big
problem in, or the most common

12
00:00:31 --> 00:00:33
problem in scientific
computing, solving

13
00:00:33 --> 00:00:35
a linear system.
 

14
00:00:35 --> 00:00:35
Au=b.
 

15
00:00:36 --> 00:00:37
How would you do that?
 

16
00:00:37 --> 00:00:43
So we will certainly do
it for some examples.

17
00:00:43 --> 00:00:49
Like our free-fixed or our
fixed-fixed matrices.

18
00:00:49 --> 00:00:51
But I thought I'd just
put some comments.

19
00:00:51 --> 00:00:57
Often I make comments that go
beyond what specific things

20
00:00:57 --> 00:00:59
we will do in detail.
 

21
00:00:59 --> 00:01:05
Certainly the workhorse of, if
I'm in MATLAB notation, the

22
00:01:05 --> 00:01:08
workhorse would be backslash.
 

23
00:01:08 --> 00:01:11
That command is the
quick, natural way

24
00:01:11 --> 00:01:14
to get the answer u.
 

25
00:01:14 --> 00:01:18
And I want to say more
about backslash.

26
00:01:18 --> 00:01:23
I won't be able to say
all the things it does.

27
00:01:23 --> 00:01:28
There's a lot built
into that command.

28
00:01:28 --> 00:01:32
And the MATLAB helpdesk would
give the details of all

29
00:01:32 --> 00:01:35
the different things that
backslash actually does.

30
00:01:35 --> 00:01:40
It looks first, can it divide
the problem into blocks,

31
00:01:40 --> 00:01:44
into smaller problems.
 

32
00:01:44 --> 00:01:47
It looks to see if
A is symmetric.

33
00:01:47 --> 00:01:50
If it's symmetric, because of
course, that would cut the work

34
00:01:50 --> 00:01:53
in half because you know half
the matrix from the other

35
00:01:53 --> 00:01:57
half in the symmetric case.
 

36
00:01:57 --> 00:02:02
It goes hopefully along
and eventually gives

37
00:02:02 --> 00:02:03
you the answer.
 

38
00:02:03 --> 00:02:05
Ok, so I'm going to
say more about that.

39
00:02:05 --> 00:02:08
Because that's by elimination.
 

40
00:02:08 --> 00:02:11
The method there
is elimination.

41
00:02:11 --> 00:02:14
I just thought I would mention
that if I had a large sparse

42
00:02:14 --> 00:02:22
matrix, so I was in sparse math
MATLAB and used backslash, it

43
00:02:22 --> 00:02:26
would try to reorder the rows.
 

44
00:02:26 --> 00:02:30
Or it will try to reorder
the rows in an optimal way.

45
00:02:30 --> 00:02:32
Now, what would be
an optimal way?

46
00:02:32 --> 00:02:34
Just so you have
an idea of what.

47
00:02:34 --> 00:02:38
So elimination takes
this matrix, right?

48
00:02:38 --> 00:02:41
It's got all it's rows.
 

49
00:02:41 --> 00:02:44
And we'll do examples.
 

50
00:02:44 --> 00:02:51
So the idea of elimination
is that it produces a zero.

51
00:02:51 --> 00:02:57
So let's suppose it accepts the
first row as the pivot row.

52
00:02:57 --> 00:03:00
And this 1, 1 entry
would be the pivot.

53
00:03:00 --> 00:03:03
And then how does
it use the pivot?

54
00:03:03 --> 00:03:08
It subtracts multiples of that
pivot row from the other rows

55
00:03:08 --> 00:03:12
in order to get zeroes
in the first column.

56
00:03:12 --> 00:03:17
So then it has a new
matrix that's like

57
00:03:17 --> 00:03:19
it's one size smaller.
 

58
00:03:19 --> 00:03:26
And then it just continues all
the way until it gets to a

59
00:03:26 --> 00:03:29
triangular matrix with all the
pivots sitting there

60
00:03:29 --> 00:03:33
on the diagonal.
 

61
00:03:33 --> 00:03:35
So that's the idea
of elimination.

62
00:03:35 --> 00:03:38
And we'll do small examples.
 

63
00:03:38 --> 00:03:44
So why would you want
to reorder the rows?

64
00:03:44 --> 00:03:47
Two reason, two good reasons
for reordering the rows.

65
00:03:47 --> 00:03:53
One reason which MATLAB will
always unless it has some good

66
00:03:53 --> 00:04:00
reason not to, is if that pivot
is too small compared to

67
00:04:00 --> 00:04:08
numbers below it it will pick
the largest and reorder to put

68
00:04:08 --> 00:04:10
the largest pivot up there.
 

69
00:04:10 --> 00:04:11
So you like large pivots.
 

70
00:04:11 --> 00:04:13
Why do you like large pivots?
 

71
00:04:13 --> 00:04:17
Because for numerical
stability that gives

72
00:04:17 --> 00:04:19
you small multipliers.
 

73
00:04:19 --> 00:04:23
If this is the biggest number
then all the multiples that you

74
00:04:23 --> 00:04:28
need, the multiple of that row
that you want to subtract from

75
00:04:28 --> 00:04:35
the other rows will
be less than one.

76
00:04:35 --> 00:04:41
The multiplier is, so just
let's remember, the multiplier

77
00:04:41 --> 00:04:51
is the entry to eliminate
divided by the pivot.

78
00:04:51 --> 00:04:55
So if that entry to be
eliminated is smaller then the

79
00:04:55 --> 00:04:57
pivot, then the multiplier's
less than one and

80
00:04:57 --> 00:04:59
things stay stable.
 

81
00:04:59 --> 00:05:04
So that's a picture of
elimination, a sort

82
00:05:04 --> 00:05:05
of first picture.
 

83
00:05:05 --> 00:05:12
And if it's sparse that means
there's a lot of zeroes.

84
00:05:12 --> 00:05:15
And where would we like
those zeroes to be?

85
00:05:15 --> 00:05:21
Where would we like to find
zeroes to save time and use to

86
00:05:21 --> 00:05:26
avoid all the steps, to avoid
going through every step.

87
00:05:26 --> 00:05:33
If this was already a zero
then it's there and we

88
00:05:33 --> 00:05:36
don't need to get it there.
 

89
00:05:36 --> 00:05:37
The job is already done.
 

90
00:05:37 --> 00:05:39
We can go forward.
 

91
00:05:39 --> 00:05:44
And if we have a zero here then
it's already zero in the next

92
00:05:44 --> 00:05:48
one and we don't have to do,
that would be the second pivot

93
00:05:48 --> 00:05:52
row, we don't have to subtract
it from this row because

94
00:05:52 --> 00:05:53
a zero is already there.
 

95
00:05:53 --> 00:05:55
So we like zeroes.
 

96
00:05:55 --> 00:05:57
And sparse matrices
have a lot of them.

97
00:05:57 --> 00:06:00
But we only like them if
they're in the right place.

98
00:06:00 --> 00:06:02
And what's the right place?
 

99
00:06:02 --> 00:06:06
It's, roughly speaking, it's
the beginning of a row.

100
00:06:06 --> 00:06:11
So we're very happy with, we
would like to get an ordering

101
00:06:11 --> 00:06:13
that looks somehow like this.
 

102
00:06:13 --> 00:06:18
We would like to get a
bunch of zeroes in there.

103
00:06:18 --> 00:06:21
Well that's, I
really overdid it.

104
00:06:21 --> 00:06:23
Maybe we can't get that many.
 

105
00:06:23 --> 00:06:26
But my point is, you see the
point that those are at

106
00:06:26 --> 00:06:27
the beginning of the rows.
 

107
00:06:27 --> 00:06:34
So let's say we can't quite
get to that, but maybe

108
00:06:34 --> 00:06:35
something more like that.
 

109
00:06:35 --> 00:06:36
Zero.
 

110
00:06:36 --> 00:06:41
So we get some rows starting
with zeroes and then we

111
00:06:41 --> 00:06:47
get rows that don't.
 

112
00:06:47 --> 00:06:49
Which of course, I mean, they
were there in the first place.

113
00:06:49 --> 00:06:52
What we've done is move
them to the bottom.

114
00:06:52 --> 00:06:56
And the reason for moving them
to the bottom is that they

115
00:06:56 --> 00:07:00
don't destroy rows below them.
 

116
00:07:00 --> 00:07:03
Yeah.
 

117
00:07:03 --> 00:07:06
I didn't show how
that would happen.

118
00:07:06 --> 00:07:09
Suppose I had a number there.
 

119
00:07:09 --> 00:07:10
Suppose that wasn't a zero.
 

120
00:07:10 --> 00:07:12
And all these were.
 

121
00:07:12 --> 00:07:16
Then when I do that elimination
I subtract a multiple of

122
00:07:16 --> 00:07:18
that row from this row.
 

123
00:07:18 --> 00:07:21
Do you see that those
zeroes will fill in?

124
00:07:21 --> 00:07:24
So fill in is the bad thing
that you're trying to avoid.

125
00:07:24 --> 00:07:30
So in this quick discussion of
reordering you're trying to

126
00:07:30 --> 00:07:34
order the rows so that they
start with zeroes for as long

127
00:07:34 --> 00:07:38
as you can because those
zeroes will never fill in.

128
00:07:38 --> 00:07:42
Zeroes inside a band, yes, just
to make the point again,

129
00:07:42 --> 00:07:47
suppose I have a long row.
 

130
00:07:47 --> 00:07:51
Then if I have some zeroes
here, here, here, useless.

131
00:07:51 --> 00:07:58
Because when I go to do
elimination on these rows, when

132
00:07:58 --> 00:08:01
I subtract a multiple of that
row from that row, that zero

133
00:08:01 --> 00:08:03
will fill in, it didn't
help, it's gone.

134
00:08:03 --> 00:08:05
And this and this.
 

135
00:08:05 --> 00:08:10
So zeroes sort of near
the diagonal are often

136
00:08:10 --> 00:08:12
going to fill in.
 

137
00:08:12 --> 00:08:16
It's zeroes at the far
left that are good.

138
00:08:16 --> 00:08:20
So that's a discussion of
a topic that actually

139
00:08:20 --> 00:08:23
comes into 18.086.
 

140
00:08:23 --> 00:08:26
And so does the third words
that I thought I would

141
00:08:26 --> 00:08:27
just write on the board.
 

142
00:08:27 --> 00:08:32
So this would be topic
specializing in how do

143
00:08:32 --> 00:08:34
you solve large systems.
 

144
00:08:34 --> 00:08:38
And reordering with
elimination is one way.

145
00:08:38 --> 00:08:42
And a second approach is, I
just put these words up here

146
00:08:42 --> 00:08:44
so that you've seen them.
 

147
00:08:44 --> 00:08:47
Conjugate gradient method,
that's a giant success in

148
00:08:47 --> 00:08:51
solving symmetric problems.
 

149
00:08:51 --> 00:08:52
Large symmetric problems.
 

150
00:08:52 --> 00:08:59
Something called multigrid is
a terrific, incomplete LU.

151
00:08:59 --> 00:09:05
People have worked hard on
ideas for solving problems

152
00:09:05 --> 00:09:09
so large that this
becomes too expensive.

153
00:09:09 --> 00:09:14
But backslash is the
natural choice.

154
00:09:14 --> 00:09:16
If you can do it, it's
like it's so simple.

155
00:09:16 --> 00:09:19
So let me focus on backslash.
 

156
00:09:19 --> 00:09:22
So backslash is the
key to talk about.

157
00:09:22 --> 00:09:28
Let me see, let's just think
what does backslash do?

158
00:09:28 --> 00:09:33
Suppose I had two equations,
two different right-hand sides,

159
00:09:33 --> 00:09:38
b and c with the same matrix A.
 

160
00:09:38 --> 00:09:44
Would I solve them, would I do
u=A\b and then separately A\c?

161
00:09:45 --> 00:09:46
No.
 

162
00:09:46 --> 00:09:52
If I had two equations with the
same matrix, there's a big

163
00:09:52 --> 00:09:58
saving in, suppose I have
two equations, two,

164
00:09:58 --> 00:09:59
well, can I put it?
 

165
00:09:59 --> 00:10:01
I'll make a matrix.
 

166
00:10:01 --> 00:10:03
I mean this is what
MATLAB would always do.

167
00:10:03 --> 00:10:11
Put those two right-hand
sides into a matrix.

168
00:10:11 --> 00:10:15
Then backslash is
happy with that.

169
00:10:15 --> 00:10:20
That backslash command would
solve both equations.

170
00:10:20 --> 00:10:23
It would give you the answer
then, would be, it would

171
00:10:23 --> 00:10:25
be two parts to it.
 

172
00:10:25 --> 00:10:29
What would be the first
column of u then?

173
00:10:29 --> 00:10:31
So, do you see what I have now?
 

174
00:10:31 --> 00:10:35
I have a square matrix
A times an unknown u.

175
00:10:35 --> 00:10:37
And I've got two
right-hand sides.

176
00:10:37 --> 00:10:39
The point is this
often happens.

177
00:10:39 --> 00:10:45
If you're doing, say,
a design problem.

178
00:10:45 --> 00:10:46
You might try
different designs.

179
00:10:46 --> 00:10:50
So those different designs give
you different right-hand sides.

180
00:10:50 --> 00:10:54
And then you have a
problem to solve.

181
00:10:54 --> 00:10:56
What's the response
to those designs?

182
00:10:56 --> 00:11:00
What are the displacements,
what happens?

183
00:11:00 --> 00:11:05
So you want to solve with two
right-hand sides and the point

184
00:11:05 --> 00:11:10
is you don't want to go
through the elimination

185
00:11:10 --> 00:11:13
process on A twice.
 

186
00:11:13 --> 00:11:15
That's crazy.
 

187
00:11:15 --> 00:11:20
You see how that elimination
process on A has this, what

188
00:11:20 --> 00:11:23
I described here,
didn't look at b.

189
00:11:23 --> 00:11:26
I didn't even get to the
right-hand side part.

190
00:11:26 --> 00:11:28
I was dealing with the
expensive part, which

191
00:11:28 --> 00:11:30
is the matrix A.
 

192
00:11:30 --> 00:11:32
So we don't want to
pay that price twice.

193
00:11:32 --> 00:11:37
And therefore backslash
is all set up.

194
00:11:37 --> 00:11:42
So what's the first
column of u then?

195
00:11:42 --> 00:11:42
It's A\b.
 

196
00:11:43 --> 00:11:47
And now what's another way to
write, a more mathematical

197
00:11:47 --> 00:11:48
way to write A?
 

198
00:11:48 --> 00:11:50
What's the answer to Au=b.
 

199
00:11:50 --> 00:11:53
 
 

200
00:11:53 --> 00:11:57
It's u equal A
inverse b, right?

201
00:11:57 --> 00:12:00
That's the answer in the first
column and A inverse c would be

202
00:12:00 --> 00:12:03
the answer in the
second column.

203
00:12:03 --> 00:12:06
So it would produce
those answers.

204
00:12:06 --> 00:12:09
Backslash will produce the
answer to the first equation

205
00:12:09 --> 00:12:11
and the answer the
second equation.

206
00:12:11 --> 00:12:16
Will it do that by
finding A inverse?

207
00:12:16 --> 00:12:18
No.
 

208
00:12:18 --> 00:12:22
A inverse, for multiple
reasons, we don't often

209
00:12:22 --> 00:12:28
compute, if it's 3 by 3, 4 by
4, then it's not a bad idea to

210
00:12:28 --> 00:12:30
see what A inverse looks like.
 

211
00:12:30 --> 00:12:35
We'll do that actually, because
it's very enlightening if the

212
00:12:35 --> 00:12:38
matrix is small, you can
see what's going on.

213
00:12:38 --> 00:12:42
But for a large problem
we don't want A inverse.

214
00:12:42 --> 00:12:43
We want the answer.
 

215
00:12:43 --> 00:12:46
And backslash goes
to the answer.

216
00:12:46 --> 00:12:53
It doesn't get there by
computing A inverse.

217
00:12:53 --> 00:12:56
And our examples will show how.
 

218
00:12:56 --> 00:13:05
So I'm giving A inverse a
little bad comments right now.

219
00:13:05 --> 00:13:08
So I maybe should finish that
sentence and then you'll

220
00:13:08 --> 00:13:11
see me turn around and
compute the darn thing.

221
00:13:11 --> 00:13:16
But why do we not
use A inverse?

222
00:13:16 --> 00:13:18
Two reasons.
 

223
00:13:18 --> 00:13:20
One is it's more expensive.
 

224
00:13:20 --> 00:13:23
If I have to compute A
inverse and multiply by b,

225
00:13:23 --> 00:13:26
that's taking too long.
 

226
00:13:26 --> 00:13:30
Second reason is A inverse
could easily be a

227
00:13:30 --> 00:13:32
full, dense matrix.
 

228
00:13:32 --> 00:13:34
All non-zero.
 

229
00:13:34 --> 00:13:37
Where A itself was
like, tridiagonal.

230
00:13:37 --> 00:13:42
So if A is tridiagonal, all the
numbers we need are there in

231
00:13:42 --> 00:13:47
three diagonals, we don't want
A inverse, you'll see,

232
00:13:47 --> 00:13:48
A inverse is full.
 

233
00:13:48 --> 00:13:51
So two reasons for
not using A inverse.

234
00:13:51 --> 00:13:58
Takes too long in the first
place even in the good case and

235
00:13:58 --> 00:14:03
often our matrix A has got lots
is zeroes that are not there in

236
00:14:03 --> 00:14:07
A inverse so we've wasted time.
 

237
00:14:07 --> 00:14:13
Nevertheless let me say
something about A inverse

238
00:14:13 --> 00:14:16
on the next board.
 

239
00:14:16 --> 00:14:17
I don't know if you
ever thought about

240
00:14:17 --> 00:14:20
the inverse matrix.
 

241
00:14:20 --> 00:14:22
Let me ask you this question.
 

242
00:14:22 --> 00:14:27
Suppose I use the command A\I.
 

243
00:14:27 --> 00:14:32
 
 

244
00:14:32 --> 00:14:33
What's that doing?
 

245
00:14:33 --> 00:14:37
That's putting the identity on
the right-hand side instead of

246
00:14:37 --> 00:14:41
a single vector b or instead of
two vectors b and c

247
00:14:41 --> 00:14:44
I'm now putting n.
 

248
00:14:44 --> 00:14:46
That's ok.
 

249
00:14:46 --> 00:14:48
Backslash will work with that.
 

250
00:14:48 --> 00:14:51
That's a shorthand for solving.
 

251
00:14:51 --> 00:14:56
This solves all
these equations.

252
00:14:56 --> 00:14:59
A, it'll get my different
answers, u_1, u_2.

253
00:15:00 --> 00:15:04
It'll get n different answers
from the n right-hand

254
00:15:04 --> 00:15:07
sides, the columns of I.
 

255
00:15:07 --> 00:15:09
Let me take to be three.
 

256
00:15:09 --> 00:15:12
0, 1, 0; 0, 0, 1.
 

257
00:15:12 --> 00:15:19
But I'll take n to be three.
 

258
00:15:19 --> 00:15:21
That's the identity.
 

259
00:15:21 --> 00:15:23
So this solves that equation.
 

260
00:15:23 --> 00:15:29
A\I will output u,
will output u.

261
00:15:29 --> 00:15:31
And here's the question.
 

262
00:15:31 --> 00:15:32
What have I got?
 

263
00:15:32 --> 00:15:38
If you have a matrix A, square
matrix and of course you have

264
00:15:38 --> 00:15:45
to create I as eye(3), that I
would be, this would be eye(3).

265
00:15:47 --> 00:15:52
Cleve Moler's lousy pun.
 

266
00:15:52 --> 00:15:56
So what would I get?
 

267
00:15:56 --> 00:15:57
What would I get?
 

268
00:15:57 --> 00:15:59
The inverse, yes.
 

269
00:15:59 --> 00:16:00
I'd get the inverse.
 

270
00:16:00 --> 00:16:04
I'd get the inverse matrix.
 

271
00:16:04 --> 00:16:06
A backslash-- why's that?
 

272
00:16:06 --> 00:16:11
Because what matrix solves,
what's the solution u to A

273
00:16:11 --> 00:16:13
times something equal I?
 

274
00:16:13 --> 00:16:17
The solution to this
equation is Au=I.

275
00:16:17 --> 00:16:20
 
 

276
00:16:20 --> 00:16:24
The solution to that equation
is the inverse matrix.

277
00:16:24 --> 00:16:28
u will be A inverse.
 

278
00:16:28 --> 00:16:33
So that is a pretty, I mean,
that's pretty, if I wanted

279
00:16:33 --> 00:16:35
A inverse that's a
good way to do it.

280
00:16:35 --> 00:16:40
Other ways to do it would be
inv(A) in MATLAB and other

281
00:16:40 --> 00:16:44
ways, but this is about
as good as you get.

282
00:16:44 --> 00:16:51
Do you see that you get the
inverse matrix that way?

283
00:16:51 --> 00:17:03
And it's worth giving
some words to that fact.

284
00:17:03 --> 00:17:05
How would I describe this?
 

285
00:17:05 --> 00:17:10
This is a set of three
different problems.

286
00:17:10 --> 00:17:14
I would describe <1, 0,
0>, that right-hand

287
00:17:14 --> 00:17:18
side as an impulse.
 

288
00:17:18 --> 00:17:19
That's an impulse.
 

289
00:17:19 --> 00:17:26
A delta vector with an impulse
in the first component.

290
00:17:26 --> 00:17:29
I'd call that an impulse
in the second component.

291
00:17:29 --> 00:17:31
I'd call that an impulse
in the third component.

292
00:17:31 --> 00:17:37
So my inputs are three
impulses and my outputs

293
00:17:37 --> 00:17:39
are u_1, u_2, u_3.
 

294
00:17:39 --> 00:17:42
What words might I use?
 

295
00:17:42 --> 00:17:46
I could call those
impulse response.

296
00:17:46 --> 00:17:48
If I were in Course 6,
I certainly would.

297
00:17:48 --> 00:17:52
These would be impulses, these
would be the responses to that

298
00:17:52 --> 00:17:55
impulse from our system.
 

299
00:17:55 --> 00:18:01
So those are impulse responses
but in linear algebra the words

300
00:18:01 --> 00:18:08
I would use, u_1, u_2, u_3 are
the columns of A

301
00:18:08 --> 00:18:08
inverse, right?
 

302
00:18:08 --> 00:18:12
That's what we just
said. u_1, u_2, u_3.

303
00:18:12 --> 00:18:15
So the columns of-- Let
me write that down.

304
00:18:15 --> 00:18:25
The columns of A inverse are
the responses, the u's,

305
00:18:25 --> 00:18:33
the solutions to the
impulses, to n impulses.

306
00:18:33 --> 00:18:40
And these are the columns of I.
 

307
00:18:40 --> 00:18:41
Do you see?
 

308
00:18:41 --> 00:18:45
Nothing I've said so far
is deep or anything.

309
00:18:45 --> 00:18:51
But it's just, this comes up so
much that it's nice to have

310
00:18:51 --> 00:18:53
different ways to
think about this.

311
00:18:53 --> 00:18:59
And actually, yeah, if I had to
solve by hand, if I had to find

312
00:18:59 --> 00:19:06
the inverse by hand I would
use elimination on this

313
00:19:06 --> 00:19:08
system of equations.
 

314
00:19:08 --> 00:19:10
I would take A and put
the identity next to it.

315
00:19:10 --> 00:19:15
I would do elimination a lot.
 

316
00:19:15 --> 00:19:17
Actually I'll put that in one.
 

317
00:19:17 --> 00:19:24
If I had to find the inverse I
would take this block matrix.

318
00:19:24 --> 00:19:28
Is that the first block
matrix we've seen?

319
00:19:28 --> 00:19:34
Block matrices are really, you
should just get familiar, when

320
00:19:34 --> 00:19:37
you're getting familiar with
matrices, they often

321
00:19:37 --> 00:19:38
come in blocks.
 

322
00:19:38 --> 00:19:41
So here's a three by three
block, here's a three by three

323
00:19:41 --> 00:19:44
block, the whole matrix
is three by six.

324
00:19:44 --> 00:19:48
I can go through the
elimination steps on it.

325
00:19:48 --> 00:19:52
If I really go haywire on
elimination and keep going and

326
00:19:52 --> 00:20:00
going and going all the way
until A gets to the identity,

327
00:20:00 --> 00:20:04
so I do elimination and I get
it triangular then I even clean

328
00:20:04 --> 00:20:07
out up above the pivots and
then I change all the pivots to

329
00:20:07 --> 00:20:10
one, I can get all the way
to the identity if the

330
00:20:10 --> 00:20:12
matrix is invertible.
 

331
00:20:12 --> 00:20:17
And what do you think will
show up in the right half?

332
00:20:17 --> 00:20:19
A inverse, yeah.
 

333
00:20:19 --> 00:20:23
So A inverse will
show up there.

334
00:20:23 --> 00:20:28
So in 18.06 I would explain,
go through examples of

335
00:20:28 --> 00:20:35
this computation just to
see A inverse appear.

336
00:20:35 --> 00:20:38
There's no reason for
us to go through long

337
00:20:38 --> 00:20:40
examples like that.
 

338
00:20:40 --> 00:20:45
M one three by three would be
worth doing, but in the big

339
00:20:45 --> 00:20:49
picture we're going
to use backslash.

340
00:20:49 --> 00:20:51
Questions or discussion.
 

341
00:20:51 --> 00:20:55
So this is the sort of overall
picture about the inverse.

342
00:20:55 --> 00:20:58
Well this is about the inverse.
 

343
00:20:58 --> 00:21:02
This was about elimination.
 

344
00:21:02 --> 00:21:04
I've gotta take a little
time on elimination.

345
00:21:04 --> 00:21:07
No it's just too important.
 

346
00:21:07 --> 00:21:11
It's sort of straightforward
and mechanical but it's like,

347
00:21:11 --> 00:21:13
too important to blow away.
 

348
00:21:13 --> 00:21:23
So let me remove that and
put something better there.

349
00:21:23 --> 00:21:25
So now I'm talking
about elimination.

350
00:21:25 --> 00:21:28
I'm talking about one equation.
 

351
00:21:28 --> 00:21:30
I'm back to Au=b.
 

352
00:21:30 --> 00:21:34
Au=b.
 

353
00:21:34 --> 00:21:41
And notice I'm using the letter
A rather than K or one of

354
00:21:41 --> 00:21:44
our special letters because
right now I don't know that

355
00:21:44 --> 00:21:46
that's a special matrix.
 

356
00:21:46 --> 00:21:50
In a minute the example I do
will be one of our special

357
00:21:50 --> 00:21:54
matrices of course, and
then I'll use it's letter.

358
00:21:54 --> 00:21:58
Just a word, though.
 

359
00:21:58 --> 00:22:02
I thought I would take-- you're
getting a lot of big picture

360
00:22:02 --> 00:22:05
here for a minute and then
we'll get into the details.

361
00:22:05 --> 00:22:08
I thought I would just,
like, this is an

362
00:22:08 --> 00:22:11
occasion to look ahead.
 

363
00:22:11 --> 00:22:17
To say a word about the big
picture of linear algebra.

364
00:22:17 --> 00:22:22
It's got four major
problems, linear algebra.

365
00:22:22 --> 00:22:25
And there are four commands
to solve those problems.

366
00:22:25 --> 00:22:27
And those commands,
why not know?

367
00:22:27 --> 00:22:30
So those commands are LU.
 

368
00:22:30 --> 00:22:36
I'm speaking about MABLAB
notation but Octave, Scilab,

369
00:22:36 --> 00:22:42
Python, R, all other
would have those.

370
00:22:42 --> 00:22:45
Would do these same things.
 

371
00:22:45 --> 00:22:53
So LU is the command that
produces this, but I didn't

372
00:22:53 --> 00:22:56
say what that is yet.
 

373
00:22:56 --> 00:22:57
Ooh.
 

374
00:22:57 --> 00:22:58
So that's my job, right.
 

375
00:22:58 --> 00:23:03
What does this mean?
 

376
00:23:03 --> 00:23:05
What's up there?
 

377
00:23:05 --> 00:23:10
So, okay, to nobody's surprise
MATLAB thought, ok, LU was

378
00:23:10 --> 00:23:12
a good letter for that.
 

379
00:23:12 --> 00:23:15
And what did MATLAB think of
as a good letter for the

380
00:23:15 --> 00:23:17
command that does this?
 

381
00:23:17 --> 00:23:18
QR.
 

382
00:23:18 --> 00:23:30
So if I did lu(A) or qr(A) I
would get-- I mean, this is

383
00:23:30 --> 00:23:34
sometimes associated with
the names of Gram-Schmidt.

384
00:23:34 --> 00:23:36
It makes vectors orthogonal.
 

385
00:23:36 --> 00:23:39
Not to worry about this stuff.
 

386
00:23:39 --> 00:23:44
You can like, close your eyes
for a moment here. lu is the

387
00:23:44 --> 00:23:48
first command and that's what
today's about. qr is the key

388
00:23:48 --> 00:23:51
command for least
squares problems.

389
00:23:51 --> 00:23:55
Maybe the biggest application
of rectangular matrices,

390
00:23:55 --> 00:23:57
I'm sure that's the big.
 

391
00:23:57 --> 00:24:00
Eigenvalues, do you know
about eigenvalues?

392
00:24:00 --> 00:24:04
Well we'll just name
the command eig(A).

393
00:24:04 --> 00:24:09
And the singular value D
composition, which you may

394
00:24:09 --> 00:24:18
never have heard of, but
you will, is svd(A).

395
00:24:18 --> 00:24:21
Can I leave that?
 

396
00:24:21 --> 00:24:23
It's in the videotape now.
 

397
00:24:23 --> 00:24:32
And my point is that when we've
spoken about those four, we

398
00:24:32 --> 00:24:36
really have got numerical
linear algebra and a lot of

399
00:24:36 --> 00:24:42
pure linear algebra explained.
 

400
00:24:42 --> 00:24:44
These are the four big ones.
 

401
00:24:44 --> 00:24:49
And I guess what I'm saying is,
the four big problems of linear

402
00:24:49 --> 00:24:56
algebra turn out to, a good way
to describe the answer is as a

403
00:24:56 --> 00:24:59
factorization of the matrix.
 

404
00:24:59 --> 00:25:00
This is a factor.
 

405
00:25:00 --> 00:25:02
So now let me say
what this one is.

406
00:25:02 --> 00:25:08
I start with a matrix and what
elimination is really doing, if

407
00:25:08 --> 00:25:11
you look to see what is it
doing, it's producing a lower

408
00:25:11 --> 00:25:15
triangular times an
upper triangular.

409
00:25:15 --> 00:25:17
Let's go directly to that.
 

410
00:25:17 --> 00:25:18
Let me go directly to that.
 

411
00:25:18 --> 00:25:21
Let me take an example.
 

412
00:25:21 --> 00:25:22
So here's my matrix.
 

413
00:25:22 --> 00:25:34
Well, I don't have to call it
A because you recognize it.

414
00:25:34 --> 00:25:37
So what's our name
for that matrix?

415
00:25:37 --> 00:25:37
T.
 

416
00:25:37 --> 00:25:41
T because the top boundary
condition is free.

417
00:25:41 --> 00:25:44
Oh, that reminds me.
 

418
00:25:44 --> 00:25:49
Some good comments after class
Friday brought out something

419
00:25:49 --> 00:25:51
that I sloughed over.
 

420
00:25:51 --> 00:25:58
That the free-fixed matrix,
the free-fixed problem is

421
00:25:58 --> 00:26:02
usually one unknown larger
than the fixed-fixed.

422
00:26:02 --> 00:26:04
Because remember the
fixed-fixed problem

423
00:26:04 --> 00:26:05
had both ends fixed.
 

424
00:26:05 --> 00:26:07
They were not unknowns.
 

425
00:26:07 --> 00:26:12
The only unknowns were one,
two, three to n in the middle.

426
00:26:12 --> 00:26:16
But people noticed when I was
talking about the free boundary

427
00:26:16 --> 00:26:21
condition that u_0 came into
it and u_0 is not known.

428
00:26:21 --> 00:26:26
So really, the free boundary
condition like, has an extra

429
00:26:26 --> 00:26:30
unknown, an extra row and
column in the matrix

430
00:26:30 --> 00:26:33
and that's correct.
 

431
00:26:33 --> 00:26:38
We'll see later in Fourier
transforms cosine matrices

432
00:26:38 --> 00:26:42
are one size bigger
than sine matrices.

433
00:26:42 --> 00:26:47
The cosine matrices are
free-free and the sine

434
00:26:47 --> 00:26:48
matrices are fixed-fixed.
 

435
00:26:48 --> 00:26:54
And now here we're
at free-fixed.

436
00:26:54 --> 00:26:59
I want to do elimination
on that matrix.

437
00:26:59 --> 00:27:03
And while I'm at it,
we'll find the inverse.

438
00:27:03 --> 00:27:08
But let's do elimination.
 

439
00:27:08 --> 00:27:10
Just on that matrix.
 

440
00:27:10 --> 00:27:15
Just to see what this
L and U stuff is.

441
00:27:15 --> 00:27:19
What do we do?
 

442
00:27:19 --> 00:27:21
The first pivot is?
 

443
00:27:21 --> 00:27:24
One, it's fine.
 

444
00:27:24 --> 00:27:27
Not going to worry about that.
 

445
00:27:27 --> 00:27:28
We'll use it now.
 

446
00:27:28 --> 00:27:29
So how do I use it?
 

447
00:27:29 --> 00:27:33
I use a pivot, now listen
because here is a convention

448
00:27:33 --> 00:27:36
here, I'm going to use
the word subtract.

449
00:27:36 --> 00:27:41
You would say add that
row to that row, right?

450
00:27:41 --> 00:27:44
Because you want to
get a zero here.

451
00:27:44 --> 00:27:46
Forgive me for making
it sound harder.

452
00:27:46 --> 00:27:50
I'm going to say subtract
because I like subtraction.

453
00:27:50 --> 00:27:54
Subtract minus one of
that row, my multiplier

454
00:27:54 --> 00:27:56
is minus one here.
 

455
00:27:56 --> 00:28:00
I'm going to say subtract minus
one of that row from that.

456
00:28:00 --> 00:28:02
Same thing.
 

457
00:28:02 --> 00:28:04
You'll say okay.
 

458
00:28:04 --> 00:28:05
No problem.
 

459
00:28:05 --> 00:28:07
Let's just do it.
 

460
00:28:07 --> 00:28:14
So there's the pivot row and
now when I-- shall I just add?

461
00:28:14 --> 00:28:25
When I add that to or does my
superego thing subtract minus

462
00:28:25 --> 00:28:27
one of that from that.
 

463
00:28:27 --> 00:28:28
What do I get?
 

464
00:28:28 --> 00:28:33
I get the zero, the one
and the minus one.

465
00:28:33 --> 00:28:34
And then what do I get?
 

466
00:28:34 --> 00:28:36
What's the multiplier?
 

467
00:28:36 --> 00:28:42
So let's just put these L,
these multipliers, the l_21.

468
00:28:42 --> 00:28:44
That's the multiplier.
 

469
00:28:44 --> 00:28:48
2, 1 refers to row
two, column one.

470
00:28:48 --> 00:28:51
And this step got the zero
in row two, column one.

471
00:28:51 --> 00:28:54
And what was the
multiplier that did it?

472
00:28:54 --> 00:28:58
It's the number that I
multiplied row one by and

473
00:28:58 --> 00:29:02
subtracted from row two,
so it was minus one.

474
00:29:02 --> 00:29:05
What's l_31?
 

475
00:29:05 --> 00:29:10
What's the multiplier that
produces a zero in the three,

476
00:29:10 --> 00:29:13
row three, column one position?
 

477
00:29:13 --> 00:29:15
It's zero.
 

478
00:29:15 --> 00:29:21
I take zero of this row
away from this row because

479
00:29:21 --> 00:29:23
it's zero already.
 

480
00:29:23 --> 00:29:26
So I'm not going to change,
that row won't change

481
00:29:26 --> 00:29:30
and l_31 was zero.
 

482
00:29:30 --> 00:29:35
Now I know the next pivot.
 

483
00:29:35 --> 00:29:36
I'm ready to use it.
 

484
00:29:36 --> 00:29:39
I want to get a zero below
it because I'm aiming at

485
00:29:39 --> 00:29:42
this upper triangular u.
 

486
00:29:42 --> 00:29:45
And what's the multiplier now?
 

487
00:29:45 --> 00:29:47
And what's its number?
 

488
00:29:47 --> 00:29:49
What's the multiplier number?
 

489
00:29:49 --> 00:29:55
3, 2 because I'm trying to
fix row three, column two.

490
00:29:55 --> 00:29:58
And what do I multiply
this by and subtract from

491
00:29:58 --> 00:30:02
this to make it zero?
 

492
00:30:02 --> 00:30:04
It's negative one again.
 

493
00:30:04 --> 00:30:05
Negative one, right.
 

494
00:30:05 --> 00:30:06
It's negative one.
 

495
00:30:06 --> 00:30:08
Sorry.
 

496
00:30:08 --> 00:30:09
Right.
 

497
00:30:09 --> 00:30:11
And now what happens
when I do that?

498
00:30:11 --> 00:30:16
Can I just do it in place here?
 

499
00:30:16 --> 00:30:19
Forgive me if I just
add that to that.

500
00:30:19 --> 00:30:25
And I'll get zero and one.
 

501
00:30:25 --> 00:30:31
And now what do I know at this
point, what have I learned?

502
00:30:31 --> 00:30:32
The most important thing
I've learned is the

503
00:30:32 --> 00:30:34
matrix is invertible.
 

504
00:30:34 --> 00:30:39
Because the pivots one, one,
and one, well they're all

505
00:30:39 --> 00:30:40
here on the diagonal.
 

506
00:30:40 --> 00:30:42
This is my matrix U.
 

507
00:30:42 --> 00:30:45
That's my upper
triangular matrix.

508
00:30:45 --> 00:30:48
And-- yeah, of course?
 

509
00:30:48 --> 00:31:05
I'm subtracting from this, so
I've got the two is there,

510
00:31:05 --> 00:31:06
yeah, yeah, that's right.
 

511
00:31:06 --> 00:31:11
So the two is sitting there and
I'm subtracting minus one of

512
00:31:11 --> 00:31:16
that row from it and that would
mean taking, yeah, yeah.

513
00:31:16 --> 00:31:18
Right.
 

514
00:31:18 --> 00:31:21
That would mean I'm subtracting
one from the two and

515
00:31:21 --> 00:31:21
getting the one.
 

516
00:31:21 --> 00:31:22
Yeah, yeah.
 

517
00:31:22 --> 00:31:27
So the row, the typical entry
is, the typical result is the

518
00:31:27 --> 00:31:31
row you have minus L
times the pivot row.

519
00:31:31 --> 00:31:34
The row you have minus the
multiplier times the pivot row.

520
00:31:34 --> 00:31:39
That's the operation that
elimination lives on.

521
00:31:39 --> 00:31:42
Elimination does
that all the time.

522
00:31:42 --> 00:31:45
It's one of the basic linear
algebra subroutines.

523
00:31:45 --> 00:31:48
B L A S.
 

524
00:31:48 --> 00:31:52
Now, this is my U.
 

525
00:31:52 --> 00:31:55
So that's the goal
of elimination, get

526
00:31:55 --> 00:31:57
upper triangular.
 

527
00:31:57 --> 00:31:59
And the reason is, you can
solve upper triangular

528
00:31:59 --> 00:32:02
systems really fast.
 

529
00:32:02 --> 00:32:08
These multipliers l, l_21,
2, and so on, they and

530
00:32:08 --> 00:32:12
go into the L matrix.
 

531
00:32:12 --> 00:32:17
And now, let me just say it
here that in a way, that

532
00:32:17 --> 00:32:20
example is too beautiful.
 

533
00:32:20 --> 00:32:25
Seldom am I sorry to see an
example come out beautifully,

534
00:32:25 --> 00:32:28
but why do I say this
is too beautiful?

535
00:32:28 --> 00:32:30
It's not typical.
 

536
00:32:30 --> 00:32:34
If I had other numbers here, I
would get to other numbers here

537
00:32:34 --> 00:32:37
and what would be the
difference, typically?

538
00:32:37 --> 00:32:44
The pivots wouldn't
be all ones.

539
00:32:44 --> 00:32:48
That's what's too beautiful
here, but let's go with it.

540
00:32:48 --> 00:32:52
I mean, it was worth it because
everything came out simple.

541
00:32:52 --> 00:32:57
But the pivots for another
problem, ooh, let me just

542
00:32:57 --> 00:33:03
do a second problem here.
 

543
00:33:03 --> 00:33:06
I'll do the fixed-fixed guy.
 

544
00:33:06 --> 00:33:09
Ok, so let's just do
elimination on that.

545
00:33:09 --> 00:33:11
That's the first pivot.
 

546
00:33:11 --> 00:33:12
Subtract.
 

547
00:33:12 --> 00:33:17
Now what's the multiplier now?
 

548
00:33:17 --> 00:33:18
You're not as quick as MATLAB.
 

549
00:33:18 --> 00:33:20
MATLAB is ahead of you.
 

550
00:33:20 --> 00:33:26
So the multiplier
is negative 1/2.

551
00:33:26 --> 00:33:32
So the multiplier is, l_21 is
negative 1/2. l_31 will again

552
00:33:32 --> 00:33:36
be zero and let's use it, so
it knocks that guys out.

553
00:33:36 --> 00:33:39
And what did that
number come out to be?

554
00:33:39 --> 00:33:40
Do you remember?
 

555
00:33:40 --> 00:33:42
That was 3/2.
 

556
00:33:42 --> 00:33:44
I think we looked at that once.
 

557
00:33:44 --> 00:33:46
And that would be all the same.
 

558
00:33:46 --> 00:33:52
And then the next multiplier,
l_32 will be negative 2/3

559
00:33:52 --> 00:33:56
because when I multiply that by
2/3 it gives me the negative

560
00:33:56 --> 00:34:02
one and then I subtract and it
kills this and I get 4/3.

561
00:34:02 --> 00:34:04
I just did that quickly.
 

562
00:34:04 --> 00:34:11
And my main point was the
pivots are on the diagonal.

563
00:34:11 --> 00:34:13
They're not all ones now.
 

564
00:34:13 --> 00:34:15
So this is a more typical one.
 

565
00:34:15 --> 00:34:18
This is, again our u.
 

566
00:34:18 --> 00:34:27
And our L matrix will be,
oh, oh, that's the point.

567
00:34:27 --> 00:34:30
That these l's, these
multipliers fit right into

568
00:34:30 --> 00:34:32
a lower triangular matrix.
 

569
00:34:32 --> 00:34:37
All these multipliers, and
we'll put ones on the diagonal

570
00:34:37 --> 00:34:42
of that guy and these lower
triangular ones will fit in

571
00:34:42 --> 00:34:45
just right perfectly in there.
 

572
00:34:45 --> 00:34:51
Over here the L would be,
let me construct the L.

573
00:34:51 --> 00:34:55
Ones on the diagonal
representing the pivot rows

574
00:34:55 --> 00:35:02
that stayed put and minus one,
zero, and minus one as the

575
00:35:02 --> 00:35:10
multipliers that, so this was
L, the multipliers

576
00:35:10 --> 00:35:16
that we used.
 

577
00:35:16 --> 00:35:18
One more.
 

578
00:35:18 --> 00:35:23
We're doing lots of good stuff
here and it's not deep, but

579
00:35:23 --> 00:35:29
it's-- Suppose the matrix
had been singular.

580
00:35:29 --> 00:35:34
We have to realize, okay, this
elimination method is great.

581
00:35:34 --> 00:35:38
But it can break down and it's
going to break down, it has

582
00:35:38 --> 00:35:41
to break down somehow if
the matrix is singular.

583
00:35:41 --> 00:35:46
Now what's our example of
a singular matrix here?

584
00:35:46 --> 00:35:51
The matrix, this is free-fixed
and that by fixing one support

585
00:35:51 --> 00:35:53
it wasn't singular, but if I
want to make it singular,

586
00:35:53 --> 00:35:55
what'll I take?
 

587
00:35:55 --> 00:35:57
Free-free.
 

588
00:35:57 --> 00:35:59
Free-free matrix.
 

589
00:35:59 --> 00:36:04
So can I, if I had thought to
bring colored chalk, I'll just

590
00:36:04 --> 00:36:08
erase for a moment
for the bad case.

591
00:36:08 --> 00:36:12
The bad case would
be free-free.

592
00:36:12 --> 00:36:16
And how would it show up
as bad in elimination.

593
00:36:16 --> 00:36:22
How does a singular
matrix reveal itself as

594
00:36:22 --> 00:36:24
elimination goes forward?
 

595
00:36:24 --> 00:36:30
Because you can't tell
at the beginning.

596
00:36:30 --> 00:36:30
What would have gone wrong?
 

597
00:36:30 --> 00:36:35
We would have had a zero there.
 

598
00:36:35 --> 00:36:38
We had a two that
dropped to one.

599
00:36:38 --> 00:36:41
But if we start with a
one, it'll drop to zero.

600
00:36:41 --> 00:36:43
That would have
been a zero there.

601
00:36:43 --> 00:36:49
The matrix would not
have had three pivots.

602
00:36:49 --> 00:36:53
This upper triangular matrix
is singular, no good.

603
00:36:53 --> 00:36:55
And that tells us back there
that the original matrix

604
00:36:55 --> 00:36:58
is singular, no good.
 

605
00:36:58 --> 00:37:04
So if I can't get to three
pivots somehow, the matrix'll

606
00:37:04 --> 00:37:09
be singular and that's
an example that is.

607
00:37:09 --> 00:37:12
And MATLAB would immediately
tell us, of course.

608
00:37:12 --> 00:37:17
So let's go back to the good
case for the main point.

609
00:37:17 --> 00:37:19
The good case for
the main point.

610
00:37:19 --> 00:37:26
So the good case
was three pivots.

611
00:37:26 --> 00:37:28
In fact it was extra
good because they all

612
00:37:28 --> 00:37:30
turned out to be ones.
 

613
00:37:30 --> 00:37:36
Now, oh, now we're
ready for LU.

614
00:37:36 --> 00:37:38
Here's the magic.
 

615
00:37:38 --> 00:37:40
And I'm not giving a proof.
 

616
00:37:40 --> 00:37:48
The magic is that the result U,
if I multiply the multiplier

617
00:37:48 --> 00:37:55
matrix L times the result
U, I'll bring back A.

618
00:37:55 --> 00:37:57
I'll bring back A.
 

619
00:37:57 --> 00:37:59
So let me just see.
 

620
00:37:59 --> 00:38:05
If I multiply L by U, so this
is now L times U, maybe

621
00:38:05 --> 00:38:07
you can see that I get A.
 

622
00:38:07 --> 00:38:09
So what is U?
 

623
00:38:09 --> 00:38:10
I just have to copy it.
 

624
00:38:10 --> 00:38:15
[1, 1, 1; -1, -1, 0].
 

625
00:38:15 --> 00:38:22
I could fill in the zeroes
but I know they're there.

626
00:38:22 --> 00:38:25
That's L times U.
 

627
00:38:25 --> 00:38:28
And sure enough, if I do the
multiplication, this-- How

628
00:38:28 --> 00:38:29
would you to that
multiplication?

629
00:38:29 --> 00:38:32
I would say this is one of the
first row when I see that.

630
00:38:32 --> 00:38:34
1, 0, 0 multiplying these.
 

631
00:38:34 --> 00:38:37
I'd say get one of
the first row.

632
00:38:37 --> 00:38:39
That's correct in A.
 

633
00:38:39 --> 00:38:43
Here I would say this is minus
one of the first row, plus one

634
00:38:43 --> 00:38:46
of the second row, and
sure enough it's the

635
00:38:46 --> 00:38:48
right part of A.
 

636
00:38:48 --> 00:38:51
And this is minus one of the
second row, plus one of the

637
00:38:51 --> 00:38:54
third row, and sure enough it's
the right third row of A.

638
00:38:54 --> 00:38:57
I get A.
 

639
00:38:57 --> 00:39:01
And that's when elimination
goes through with no zero

640
00:39:01 --> 00:39:07
pivots, no problems, just a
bunch of multipliers, then

641
00:39:07 --> 00:39:15
that wonderful description
of it, A=LU is correct.

642
00:39:15 --> 00:39:17
I don't know how
many that's new to.

643
00:39:17 --> 00:39:19
I should maybe have
thought ahead.

644
00:39:19 --> 00:39:22
How many have seen like,
L times U before?

645
00:39:22 --> 00:39:24
Just to give me an idea?
 

646
00:39:24 --> 00:39:25
Quite a few.
 

647
00:39:25 --> 00:39:25
Ok.
 

648
00:39:25 --> 00:39:31
So it's terrific.
 

649
00:39:31 --> 00:39:34
Oh, here I would get L times U.
 

650
00:39:34 --> 00:39:36
Now this is like a little
more interesting because

651
00:39:36 --> 00:39:43
the pivots were not ones.
 

652
00:39:43 --> 00:39:46
So that's my matrix U.
 

653
00:39:46 --> 00:39:49
And here's my matrix L, right?
 

654
00:39:49 --> 00:39:51
Okay, big point.
 

655
00:39:51 --> 00:39:56
Because we're so interested in
symmetric matrices and this one

656
00:39:56 --> 00:40:00
in particular, or that one,
symmetric matrices are good.

657
00:40:00 --> 00:40:05
Now, I'm unhappy
about one aspect.

658
00:40:05 --> 00:40:10
So now there's just
one part of this.

659
00:40:10 --> 00:40:14
This was great, we got three
non-zero pivots, we got to U,

660
00:40:14 --> 00:40:20
we got the multiplier matrix
all fine and we would be ready

661
00:40:20 --> 00:40:23
for the right-hand side and we
would be ready for two

662
00:40:23 --> 00:40:25
right-hand sides, we would even
be ready for all three

663
00:40:25 --> 00:40:28
right-hand sides, whatever.
 

664
00:40:28 --> 00:40:32
But I have one criticism.
 

665
00:40:32 --> 00:40:37
The matrix A which was
our K, this was really

666
00:40:37 --> 00:40:40
K, was symmetric.
 

667
00:40:40 --> 00:40:43
That was the very first
thing you did, told me

668
00:40:43 --> 00:40:45
on the very first day.
 

669
00:40:45 --> 00:40:48
And now it's equal to L times
U, but what's happened?

670
00:40:48 --> 00:40:51
The symmetry is lost.
 

671
00:40:51 --> 00:40:57
Somehow the L has ones on the
diagonal, the U as we have it

672
00:40:57 --> 00:41:01
has pivots on the diagonal, now
the pivots are not all ones.

673
00:41:01 --> 00:41:06
So you see the symmetry of
the problem got lost, and

674
00:41:06 --> 00:41:07
that shouldn't happen.
 

675
00:41:07 --> 00:41:10
And there ought to be
a way to get back.

676
00:41:10 --> 00:41:11
Ok.
 

677
00:41:11 --> 00:41:15
And now I want to describe the
way to get back to symmetry.

678
00:41:15 --> 00:41:19
So LU doesn't keep
the symmetry.

679
00:41:19 --> 00:41:22
L has ones, U has pivots.
 

680
00:41:22 --> 00:41:24
Different.
 

681
00:41:24 --> 00:41:29
But a very simple idea will
bring back the symmetry.

682
00:41:29 --> 00:41:37
That is peel off the pivots
into a diagonal matrix.

683
00:41:37 --> 00:41:40
In other words, there's
a matrix, I'll call

684
00:41:40 --> 00:41:45
it D, D for diagonal.
 

685
00:41:45 --> 00:41:49
I'll divide those numbers
out of each row.

686
00:41:49 --> 00:41:53
And can I just do that?
 

687
00:41:53 --> 00:41:57
So I'm just going to write this
U as a product of this diagonal

688
00:41:57 --> 00:42:00
D where I'm going to be
dividing the two out.

689
00:42:00 --> 00:42:03
So when I divide the two
out from that row I'm left

690
00:42:03 --> 00:42:06
with one, minus 1/2, zero.
 

691
00:42:06 --> 00:42:11
And when I divide 3/2, the
pivot then, it makes that pivot

692
00:42:11 --> 00:42:15
into a one and what does
it produce for that guy?

693
00:42:15 --> 00:42:19
When I divide 3/2, when
I divide that minus one

694
00:42:19 --> 00:42:22
by 3/2, what do I get?
 

695
00:42:22 --> 00:42:25
I get negative, division
will be 2/3, I'll

696
00:42:25 --> 00:42:28
get a negative 2/3.
 

697
00:42:28 --> 00:42:32
And now, on the last row I'm
dividing that row by 4/3.

698
00:42:32 --> 00:42:35
When I divide that row
by 4/3, what row do I

699
00:42:35 --> 00:42:38
get here? .
 

700
00:42:38 --> 00:42:42
Because I've made the pivots
one, well they're not pivots.

701
00:42:42 --> 00:42:45
What I've done is
separate out the pivots.

702
00:42:45 --> 00:42:51
So I've made the diagonal ones
just by separating it out.

703
00:42:51 --> 00:42:58
And what's happened?
 

704
00:42:58 --> 00:43:02
My goal was to get back
some symmetry that was

705
00:43:02 --> 00:43:03
there at the start.
 

706
00:43:03 --> 00:43:10
Now so I have a pivot matrix
D, and what's that matrix?

707
00:43:10 --> 00:43:12
You could say, well,
it's the rest.

708
00:43:12 --> 00:43:15
But that's not what
I'm looking for.

709
00:43:15 --> 00:43:17
What is it?
 

710
00:43:17 --> 00:43:19
Can everybody have
a look at it?

711
00:43:19 --> 00:43:21
I can't raise it.
 

712
00:43:21 --> 00:43:26
If you look at what we
got there, what is it?

713
00:43:26 --> 00:43:29
What's the right
name to give it?

714
00:43:29 --> 00:43:30
L transpose, exactly!
 

715
00:43:30 --> 00:43:32
That's the right name.
 

716
00:43:32 --> 00:43:34
L transpose.
 

717
00:43:34 --> 00:43:37
So what am I concluding then?
 

718
00:43:37 --> 00:43:42
I'm concluding that, let's
see, where shall I put this?

719
00:43:42 --> 00:43:43
And it'll come back to it.
 

720
00:43:43 --> 00:43:49
Well, here we had just to show
it wasn't an accident, here we

721
00:43:49 --> 00:43:54
had L, L transpose and what
was the pivot matrix in this

722
00:43:54 --> 00:43:56
too beautiful problem case?
 

723
00:43:56 --> 00:43:58
It was the identity.
 

724
00:43:58 --> 00:44:00
So we didn't notice it.
 

725
00:44:00 --> 00:44:02
So can I squeeze
in the identity?

726
00:44:02 --> 00:44:04
That's the pivot matrix there.
 

727
00:44:04 --> 00:44:08
But and again, we had
L times L transpose.

728
00:44:08 --> 00:44:11
The beauty was there,
the symmetry was there.

729
00:44:11 --> 00:44:13
And now what's the usual thing?
 

730
00:44:13 --> 00:44:21
So really I'm completing
this to one more thought.

731
00:44:21 --> 00:44:24
In that case when
A is symmetric.

732
00:44:24 --> 00:44:26
I'm completing, I have the L.
 

733
00:44:26 --> 00:44:28
I'm factoring out the D.
 

734
00:44:28 --> 00:44:33
And what's left is L transpose.
 

735
00:44:33 --> 00:44:39
I hope you like LD*L transpose.
 

736
00:44:39 --> 00:44:45
Seeing a matrix on one side and
the transpose on the other

737
00:44:45 --> 00:44:49
side, the matrix L at the left
and L transpose at the

738
00:44:49 --> 00:44:52
right is just right.
 

739
00:44:52 --> 00:45:10
So the point of symmetric case
we have, and I'll use the

740
00:45:10 --> 00:45:13
letter K rather than A because
now we're getting the matrix

741
00:45:13 --> 00:45:14
that's more special.
 

742
00:45:14 --> 00:45:19
It's that K or it's this
T or it's any other

743
00:45:19 --> 00:45:20
symmetric matrix.
 

744
00:45:20 --> 00:45:27
The elimination leads to, uses
multipliers L and if I factor

745
00:45:27 --> 00:45:33
out the pivot matrix then the
other part is L transpose.

746
00:45:33 --> 00:45:36
We've seen that
just by example.

747
00:45:36 --> 00:45:39
By two examples.
 

748
00:45:39 --> 00:45:42
Now I want to just
look at that.

749
00:45:42 --> 00:45:50
Because this that describes not
only the result of elimination

750
00:45:50 --> 00:45:56
which is the key operation, but
it also keeps the symmetry.

751
00:45:56 --> 00:46:03
In fact every matrix of
that sort is symmetric.

752
00:46:03 --> 00:46:07
No, yeah, that's important.
 

753
00:46:07 --> 00:46:13
This is sure to be symmetric.
 

754
00:46:13 --> 00:46:21
We will often see matrices
multiplied by their transpose.

755
00:46:21 --> 00:46:27
So what I'm saying is that if
you gave me any matrix L, any

756
00:46:27 --> 00:46:32
diagonal matrix D, and then the
transpose of L, if I multiplied

757
00:46:32 --> 00:46:34
those out, I would get
a symmetric matrix.

758
00:46:34 --> 00:46:38
And going the other way, if I
started with a symmetric matrix

759
00:46:38 --> 00:46:43
and I did elimination and got
an L, then the D factoring out

760
00:46:43 --> 00:46:44
would leave me L transpose.
 

761
00:46:44 --> 00:46:49
So what you've seen by
example is what will

762
00:46:49 --> 00:46:51
happen all the time.
 

763
00:46:51 --> 00:46:57
Now why is that
matrix symmetric?

764
00:46:57 --> 00:47:01
Here we get a chance to
show the power of matrix

765
00:47:01 --> 00:47:03
notation, really.
 

766
00:47:03 --> 00:47:08
I just think that if I have any
matrix L, in this case it

767
00:47:08 --> 00:47:10
happened to be lower
triangular, but if I have any

768
00:47:10 --> 00:47:14
matrix L and I have a nice,
symmetric diagonal guy in the

769
00:47:14 --> 00:47:18
middle and I have the transpose
of this matrix on the other

770
00:47:18 --> 00:47:24
side I think the result is a
symmetric matrix

771
00:47:24 --> 00:47:25
when I multiply.
 

772
00:47:25 --> 00:47:33
So it's these symmetric
factorizations that we're

773
00:47:33 --> 00:47:36
getting to and are important
problems because our important

774
00:47:36 --> 00:47:39
problems are symmetric.
 

775
00:47:39 --> 00:47:42
Ok, why is that sure
to be symmetric?

776
00:47:42 --> 00:47:49
Suppose I asked you as a
exercise, prove that L times a

777
00:47:49 --> 00:47:55
diagonal times L transpose is
always a symmetric matrix.

778
00:47:55 --> 00:47:57
How could you do that?
 

779
00:47:57 --> 00:47:59
How could you do that?
 

780
00:47:59 --> 00:48:02
You could certainly create an
example that did it and check

781
00:48:02 --> 00:48:05
it out, multiply,
it would work.

782
00:48:05 --> 00:48:11
But we want to see that this
is going to be true always.

783
00:48:11 --> 00:48:16
So how would you do that?
 

784
00:48:16 --> 00:48:19
I guess, let me get started.
 

785
00:48:19 --> 00:48:22
I would take its transpose.
 

786
00:48:22 --> 00:48:26
If I want to show something's
symmetric, I transpose it

787
00:48:26 --> 00:48:28
and see if I get the
same matrix again.

788
00:48:28 --> 00:48:32
So let me take the
transpose of this.

789
00:48:32 --> 00:48:35
So I'm answering, Why is
it sure to be symmetric?

790
00:48:35 --> 00:48:37
So let me take K transpose.
 

791
00:48:37 --> 00:48:42
So this is the transpose of, I
have K equals something times

792
00:48:42 --> 00:48:48
something times something, A
times B times C, you could say.

793
00:48:48 --> 00:48:56
If I transpose a matrix, how
can I create transposes

794
00:48:56 --> 00:48:59
out of a, B and C.
 

795
00:48:59 --> 00:49:02
Do you remember what happens?
 

796
00:49:02 --> 00:49:04
They reverse the order.
 

797
00:49:04 --> 00:49:05
It's like inverses.
 

798
00:49:05 --> 00:49:14
Transposes and inverses
both have that key rule.

799
00:49:14 --> 00:49:18
When you have a product and
you invert it, they come

800
00:49:18 --> 00:49:19
in the opposite order.
 

801
00:49:19 --> 00:49:22
When you transpose it, they
come in the opposite order.

802
00:49:22 --> 00:49:26
So let me try put these
separate transposes in

803
00:49:26 --> 00:49:31
the opposite order.
 

804
00:49:31 --> 00:49:34
So I've used the most
important fact there.

805
00:49:34 --> 00:49:43
Which is just a fact about
transposing a product.

806
00:49:43 --> 00:49:47
Ok, what have I got now?
 

807
00:49:47 --> 00:49:50
What's L transpose transposed?
 

808
00:49:50 --> 00:49:52
It's L, great.
 

809
00:49:52 --> 00:49:54
What's L transpose
L transposed?

810
00:49:54 --> 00:49:55
Nothing but L.
 

811
00:49:55 --> 00:49:57
Transpose twice and
I'm back to L.

812
00:49:57 --> 00:50:00
What about D transpose?
 

813
00:50:00 --> 00:50:01
Same as D.
 

814
00:50:01 --> 00:50:04
Because D was symmetric,
in fact diagonal.

815
00:50:04 --> 00:50:08
So what have I learned?
 

816
00:50:08 --> 00:50:11
The proof is done.
 

817
00:50:11 --> 00:50:13
I've got K back again.
 

818
00:50:13 --> 00:50:15
This was the original K.
 

819
00:50:15 --> 00:50:17
So I've learned that
K transpose is K.

820
00:50:17 --> 00:50:22
So you're going to see time
after time, let me just

821
00:50:22 --> 00:50:25
put these things there,
you're going to see an

822
00:50:25 --> 00:50:28
A transpose times an A.
 

823
00:50:28 --> 00:50:32
That's the most important,
most highly important

824
00:50:32 --> 00:50:33
multiplication.
 

825
00:50:33 --> 00:50:36
Take a matrix, maybe
rectangular, multiply

826
00:50:36 --> 00:50:38
by A transpose.
 

827
00:50:38 --> 00:50:44
So this matrix is certainly
square because A

828
00:50:44 --> 00:50:47
could be m by n.
 

829
00:50:47 --> 00:50:51
And then A transpose
would be n by m and the

830
00:50:51 --> 00:50:55
result would be n by n.
 

831
00:50:55 --> 00:50:57
So it's certainly square.
 

832
00:50:57 --> 00:51:00
But now what's the new
property we now know?

833
00:51:00 --> 00:51:02
It's symmetric.
 

834
00:51:02 --> 00:51:06
It's symmetric because if I
transpose it, the transpose of

835
00:51:06 --> 00:51:09
A will go on this side, the
double transpose will go on

836
00:51:09 --> 00:51:12
this side, but the double
transpose is A again,

837
00:51:12 --> 00:51:16
so symmetric.
 

838
00:51:16 --> 00:51:24
So I'm plugging away here on
symmetric matrices because

839
00:51:24 --> 00:51:31
they're just-- yeah, what does
symmetry mean in, yeah, can I

840
00:51:31 --> 00:51:42
just come back to this
idea of responses?

841
00:51:42 --> 00:51:50
And by the way, if this
was symmetric, would it's

842
00:51:50 --> 00:51:52
inverse be symmetric?
 

843
00:51:52 --> 00:51:54
The answer is yes.
 

844
00:51:54 --> 00:51:59
If a matrix is symmetric,
it's inverse is symmetric.

845
00:51:59 --> 00:52:04
These symmetric matrices
are a fantastic family.

846
00:52:04 --> 00:52:07
So I could add that to this.
 

847
00:52:07 --> 00:52:17
K inverse will also symmetric
without having yet said why.

848
00:52:17 --> 00:52:19
But maybe in words, I'll
just say a few words

849
00:52:19 --> 00:52:27
here at the end.
 

850
00:52:27 --> 00:52:31
So what's a typical entry?
 

851
00:52:31 --> 00:52:36
Say the 2, 1 entry, just
to carry on with this

852
00:52:36 --> 00:52:39
language one more moment.
 

853
00:52:39 --> 00:52:43
This is A inverse here.
 

854
00:52:43 --> 00:52:49
Now the 2, 1 entry in the
inverse is an impulse is in

855
00:52:49 --> 00:52:56
the first, the first mass,
whatever gets an impulse.

856
00:52:56 --> 00:53:01
And that is the response
of the second mass.

857
00:53:01 --> 00:53:06
The response in position two to
the impulse in position one.

858
00:53:06 --> 00:53:09
Now my matrix is symmetric,
thinking about symmetric

859
00:53:09 --> 00:53:10
matrices here.
 

860
00:53:10 --> 00:53:12
So what about here?
 

861
00:53:12 --> 00:53:17
Here, if I take the impulse in
position two and look at the

862
00:53:17 --> 00:53:22
response in position one, so
do you see the difference?

863
00:53:22 --> 00:53:24
In general, those
could be different.

864
00:53:24 --> 00:53:28
This is the response
at position two to

865
00:53:28 --> 00:53:30
an impulse at one.
 

866
00:53:30 --> 00:53:34
This is the response at
one to an impulse at two.

867
00:53:34 --> 00:53:37
You see that I'm multiplying
those columns and

868
00:53:37 --> 00:53:38
getting these columns.
 

869
00:53:38 --> 00:53:42
And what's the point
about symmetry?

870
00:53:42 --> 00:53:43
Those are the same.
 

871
00:53:43 --> 00:53:49
Symmetry is expressing this
physical meaning that the

872
00:53:49 --> 00:53:54
response that i, to an impulse
at j is the same as the

873
00:53:54 --> 00:53:58
response at j to
an impulse at i.

874
00:53:58 --> 00:54:03
And that's sort of, that's
such an important property,

875
00:54:03 --> 00:54:06
you want to notice it.
 

876
00:54:06 --> 00:54:09
And it goes into symmetry.
 

877
00:54:09 --> 00:54:14
So many, many problems will be
symmetric and then some won't.

878
00:54:14 --> 00:54:19
We'll have to admit this won't
cover everything, but it

879
00:54:19 --> 00:54:22
covers such an important
and beautiful part.

880
00:54:22 --> 00:54:30
So that's today's lecture on LU
elimination, solving linear

881
00:54:30 --> 00:54:38
systems, and then let's move
forward to understanding

882
00:54:38 --> 00:54:40
the actual inverses.