1
00:00:07,840 --> 00:00:08,460
OK.
2
00:00:08,460 --> 00:00:09,050
Good.
3
00:00:09,050 --> 00:00:15,540
The final class in linear
algebra at MIT this Fall
4
00:00:15,540 --> 00:00:18,430
is to review the whole course.
5
00:00:18,430 --> 00:00:23,890
And, you know the best
way I know how to review
6
00:00:23,890 --> 00:00:30,410
is to take old exams and just
think through the problems.
7
00:00:30,410 --> 00:00:33,680
So it will be a three-hour
exam next Thursday.
8
00:00:37,380 --> 00:00:41,240
Nobody will be able to take an
exam before Thursday, anybody
9
00:00:41,240 --> 00:00:44,120
who needs to take it
in some different way
10
00:00:44,120 --> 00:00:47,380
after Thursday should
see me next Monday.
11
00:00:47,380 --> 00:00:49,310
I'll be in my office Monday.
12
00:00:49,310 --> 00:00:52,190
OK.
13
00:00:52,190 --> 00:00:54,330
May I just read
out some problems
14
00:00:54,330 --> 00:01:06,400
and, let me bring the board
down, and let's start.
15
00:01:06,400 --> 00:01:08,270
OK.
16
00:01:08,270 --> 00:01:12,140
Here's a question.
17
00:01:12,140 --> 00:01:18,100
This is about a 3-by-n matrix.
18
00:01:18,100 --> 00:01:21,900
And we're given --
so we're given --
19
00:01:21,900 --> 00:01:30,160
given -- A x equals 1
0 0 has no solution.
20
00:01:33,380 --> 00:01:45,010
And we're also given A x equals
0 1 0 has exactly one solution.
21
00:01:45,010 --> 00:01:45,510
OK.
22
00:01:48,090 --> 00:01:51,970
So you can probably
anticipate my first question,
23
00:01:51,970 --> 00:01:55,170
what can you tell me about m?
24
00:01:55,170 --> 00:01:59,190
It's an m-by-n matrix
of rank r, as always,
25
00:01:59,190 --> 00:02:02,910
what can you tell me
about those three numbers?
26
00:02:02,910 --> 00:02:12,160
So what can you tell me about
m, the number of rows, n,
27
00:02:12,160 --> 00:02:16,931
the number of columns,
and r, the rank?
28
00:02:16,931 --> 00:02:17,430
OK.
29
00:02:20,100 --> 00:02:23,430
See, do you want to
tell me first what m is?
30
00:02:23,430 --> 00:02:25,275
How many rows in this matrix?
31
00:02:28,610 --> 00:02:32,940
Must be three, right?
32
00:02:32,940 --> 00:02:40,610
We can't tell what n
is, but we can certainly
33
00:02:40,610 --> 00:02:43,030
tell that m is three.
34
00:02:43,030 --> 00:02:43,530
OK.
35
00:02:43,530 --> 00:02:46,830
And, what do these
things tell us?
36
00:02:46,830 --> 00:02:49,750
Let's take them one at a time.
37
00:02:49,750 --> 00:02:53,740
When I discover that some
equation has no solution,
38
00:02:53,740 --> 00:02:56,880
that there's some right-hand
side with no answer,
39
00:02:56,880 --> 00:03:01,950
what does that tell me about
the rank of the matrix?
40
00:03:06,430 --> 00:03:11,850
It's smaller m.
41
00:03:11,850 --> 00:03:13,820
Is that right?
42
00:03:13,820 --> 00:03:19,840
If there is no
solution, that tells me
43
00:03:19,840 --> 00:03:26,060
that some rows of the matrix
are combinations of other rows.
44
00:03:26,060 --> 00:03:30,850
Because if I had a
pivot in every row,
45
00:03:30,850 --> 00:03:34,440
then I would certainly be
able to solve the system.
46
00:03:34,440 --> 00:03:38,410
I would have particular
solutions and all the good
47
00:03:38,410 --> 00:03:43,170
So any time that there's a
system with no solutions,
48
00:03:43,170 --> 00:03:46,260
stuff. that tells me
that r must be below m.
49
00:03:46,260 --> 00:03:50,450
What about the fact that if,
when there is a solution,
50
00:03:50,450 --> 00:03:52,330
there's only one?
51
00:03:52,330 --> 00:03:53,320
What does that tell me?
52
00:03:55,830 --> 00:03:58,570
Well, normally there
would be one solution,
53
00:03:58,570 --> 00:04:03,070
and then we could add in
anything in the null space.
54
00:04:03,070 --> 00:04:07,110
So this is telling me the null
space only has the 0 vector in
55
00:04:07,110 --> 00:04:07,790
it.
56
00:04:07,790 --> 00:04:10,550
There's just one
solution, period,
57
00:04:10,550 --> 00:04:11,820
so what does that tell me?
58
00:04:11,820 --> 00:04:15,000
The null space has only
the zero vector in it?
59
00:04:15,000 --> 00:04:18,959
What does that tell me about
the relation of r to n?
60
00:04:18,959 --> 00:04:22,640
So this one solution
only, that means
61
00:04:22,640 --> 00:04:29,120
the null space of the matrix
must be just the zero vector,
62
00:04:29,120 --> 00:04:33,720
and what does that
tell me about r and n?
63
00:04:33,720 --> 00:04:34,410
They're equal.
64
00:04:34,410 --> 00:04:36,020
The columns are independent.
65
00:04:36,020 --> 00:04:39,730
So I've got, now, r equals
n, and r less than m,
66
00:04:39,730 --> 00:04:42,760
and now I also know m is three.
67
00:04:42,760 --> 00:04:45,580
So those are really
the facts I know.
68
00:04:45,580 --> 00:04:51,030
n=r and those numbers
are smaller than three.
69
00:04:51,030 --> 00:04:52,820
Sorry, yes, yes.
70
00:04:52,820 --> 00:04:58,030
r is smaller than m, and
n, of course, is also.
71
00:04:58,030 --> 00:05:01,830
So I guess this summarizes
what we can tell.
72
00:05:01,830 --> 00:05:05,870
In fact, why not
give me a matrix --
73
00:05:05,870 --> 00:05:09,380
because I would often ask for
an example of such a matrix --
74
00:05:09,380 --> 00:05:13,150
can you give me a matrix
A that's an example?
75
00:05:13,150 --> 00:05:16,670
That shows this possibility?
76
00:05:16,670 --> 00:05:20,780
Exactly, that
there's no solution
77
00:05:20,780 --> 00:05:26,310
with that right-hand side, but
there's exactly one solution
78
00:05:26,310 --> 00:05:28,420
with this right-hand side.
79
00:05:28,420 --> 00:05:33,400
Anybody want to suggest
a matrix that does that?
80
00:05:33,400 --> 00:05:34,560
Let's see.
81
00:05:34,560 --> 00:05:39,610
What do I -- what vector do
I want in the column space?
82
00:05:39,610 --> 00:05:42,120
I want zero, one, zero,
to be in the column space,
83
00:05:42,120 --> 00:05:45,180
because I'm able
to solve for that.
84
00:05:45,180 --> 00:05:47,930
So let's put zero, one,
zero in the column space.
85
00:05:47,930 --> 00:05:51,290
Actually, I could
stop right there.
86
00:05:51,290 --> 00:05:56,480
That would be a matrix with
m equal three, three rows,
87
00:05:56,480 --> 00:06:04,160
and n and r are both one,
rank one, one column,
88
00:06:04,160 --> 00:06:08,170
and, of course, there's
no solution to that one.
89
00:06:08,170 --> 00:06:10,290
So that's perfectly
good as it is.
90
00:06:10,290 --> 00:06:12,990
Or if you, kind of,
have a prejudice
91
00:06:12,990 --> 00:06:15,580
against matrices that
only have one column,
92
00:06:15,580 --> 00:06:17,620
I'll accept a second
93
00:06:17,620 --> 00:06:18,310
column.
94
00:06:18,310 --> 00:06:20,970
So what could I include
as a second column
95
00:06:20,970 --> 00:06:26,020
that would just be a different
answer but equally good?
96
00:06:26,020 --> 00:06:31,420
I could put this vector in the
column space, too, if I wanted.
97
00:06:31,420 --> 00:06:39,180
That would now be a case
with r=n=2, but, of course,
98
00:06:39,180 --> 00:06:43,850
three m eq- m is still
three, and this vector is not
99
00:06:43,850 --> 00:06:45,540
in the column space.
100
00:06:45,540 --> 00:06:49,470
So you're -- this is just like
prompting us to remember all
101
00:06:49,470 --> 00:06:53,660
those things, column space,
null space, all that stuff.
102
00:06:53,660 --> 00:06:56,250
Now, I probably asked
a second question
103
00:06:56,250 --> 00:06:58,200
about this type of thing.
104
00:07:01,370 --> 00:07:01,940
OK.
105
00:07:01,940 --> 00:07:04,260
Oh, I even asked, write
down an example of a
106
00:07:04,260 --> 00:07:07,080
Ah. matrix that fits
the description.
107
00:07:07,080 --> 00:07:07,580
Hm.
108
00:07:07,580 --> 00:07:13,520
I guess I haven't learned
anything in twenty-six years.
109
00:07:13,520 --> 00:07:15,340
CK.
110
00:07:15,340 --> 00:07:19,910
Cross out all statements that
are false about any matrix with
111
00:07:19,910 --> 00:07:23,480
these -- so again, these are
-- this is the preliminary sta-
112
00:07:23,480 --> 00:07:27,600
these are the facts about my
matrix, this is one example.
113
00:07:27,600 --> 00:07:29,430
But, of course, by
having an example,
114
00:07:29,430 --> 00:07:33,290
it will be easy to check some
of these facts, or non-facts.
115
00:07:33,290 --> 00:07:40,400
Let me, let me write
down some, facts.
116
00:07:40,400 --> 00:07:42,980
Some possible facts.
117
00:07:42,980 --> 00:07:45,580
So this is really true or false.
118
00:07:45,580 --> 00:07:49,510
The determinant --
this is part one,
119
00:07:49,510 --> 00:07:55,300
the determinant of A transpose
A is the same as the determinant
120
00:07:55,300 --> 00:07:57,670
of A A transpose.
121
00:07:57,670 --> 00:07:59,820
Is that true or not?
122
00:07:59,820 --> 00:08:05,230
Second one, A transpose
A, is invertible.
123
00:08:05,230 --> 00:08:06,150
Is invertible.
124
00:08:09,810 --> 00:08:17,270
Third possible fact, A A
transpose is positive definite.
125
00:08:22,070 --> 00:08:24,090
So you see how, on
an exam question,
126
00:08:24,090 --> 00:08:28,490
I try to connect the
different parts of the course.
127
00:08:28,490 --> 00:08:33,360
So, well, I mean,
the simplest way
128
00:08:33,360 --> 00:08:38,260
would be to try it with that
matrix as a good example,
129
00:08:38,260 --> 00:08:43,730
but maybe we can
answer, even directly.
130
00:08:43,730 --> 00:08:47,100
Let me take number two first.
131
00:08:47,100 --> 00:08:51,910
Because I'm -- you know, I'm
very, very fond of that matrix,
132
00:08:51,910 --> 00:08:55,460
A transpose A.
133
00:08:55,460 --> 00:09:01,170
And when is it invertible?
134
00:09:01,170 --> 00:09:03,335
When is the matrix A
transpose A, invertible?
135
00:09:08,130 --> 00:09:12,500
The great thing is that I
can tell from the rank of A
136
00:09:12,500 --> 00:09:15,790
that I don't have to
multiply out A transpose A.
137
00:09:15,790 --> 00:09:18,990
A transpose A, is invertible --
138
00:09:18,990 --> 00:09:24,550
well, if A has a null space
other than the zero vector,
139
00:09:24,550 --> 00:09:27,410
then it -- it's -- no way
it's going to be invertible.
140
00:09:27,410 --> 00:09:31,440
But the beauty is, if the null
space of A is just the zero
141
00:09:31,440 --> 00:09:34,480
vector, so the fact
-- the key fact is,
142
00:09:34,480 --> 00:09:40,030
this is invertible if
r=n, by which I mean,
143
00:09:40,030 --> 00:09:46,290
independent columns of A.
144
00:09:46,290 --> 00:09:47,230
In A.
145
00:09:47,230 --> 00:09:49,340
In the matrix A.
146
00:09:49,340 --> 00:09:53,320
If r=n -- if the matrix A
has independent columns,
147
00:09:53,320 --> 00:09:56,200
then this combination,
A transpose A,
148
00:09:56,200 --> 00:10:00,000
is square and still
that same null space,
149
00:10:00,000 --> 00:10:03,200
only the zero vector,
independent columns all good,
150
00:10:03,200 --> 00:10:07,320
and so, what's the true/false?
151
00:10:07,320 --> 00:10:14,310
Is it -- is this middle one T
or F for this, in this setup?
152
00:10:14,310 --> 00:10:17,650
Well, we discovered that --
153
00:10:17,650 --> 00:10:22,560
we discovered that -- that r
was n, from that second fact.
154
00:10:22,560 --> 00:10:24,610
So this is a true.
155
00:10:24,610 --> 00:10:26,190
That's a true.
156
00:10:26,190 --> 00:10:29,330
And, of course, A transpose
A, in this example,
157
00:10:29,330 --> 00:10:32,190
would probably be -- what
would A transpose A, be,
158
00:10:32,190 --> 00:10:32,870
for that matrix?
159
00:10:35,610 --> 00:10:38,000
Can you multiply A
transpose A, and see what
160
00:10:38,000 --> 00:10:39,380
it looks like for that matrix?
161
00:10:39,380 --> 00:10:40,600
What shape would it be?
162
00:10:43,200 --> 00:10:44,990
It will be two by two.
163
00:10:44,990 --> 00:10:47,810
And what matrix will it be?
164
00:10:47,810 --> 00:10:48,660
The identity.
165
00:10:48,660 --> 00:10:51,080
So, it checks out.
166
00:10:51,080 --> 00:10:52,925
OK, what about A A transpose?
167
00:10:55,510 --> 00:10:58,870
Well, depending
on the shape of A,
168
00:10:58,870 --> 00:11:02,770
it could be good or not so good.
169
00:11:02,770 --> 00:11:04,980
It's always symmetric,
it's always square,
170
00:11:04,980 --> 00:11:06,400
but what's the size, now?
171
00:11:06,400 --> 00:11:11,220
This is three by n,
and this is n by three,
172
00:11:11,220 --> 00:11:13,270
so the result is three by three.
173
00:11:17,550 --> 00:11:18,665
Is it positive definite?
174
00:11:21,760 --> 00:11:23,160
I don't think so.
175
00:11:23,160 --> 00:11:24,940
False.
176
00:11:24,940 --> 00:11:29,870
If I multiply that by A
transpose, A A transpose,
177
00:11:29,870 --> 00:11:31,480
what would the rank be?
178
00:11:31,480 --> 00:11:36,620
It would be the same as
the rank of A, that's --
179
00:11:36,620 --> 00:11:39,430
it would be just rank two.
180
00:11:39,430 --> 00:11:42,200
And if it's three-by-three,
and it's only rank two,
181
00:11:42,200 --> 00:11:44,270
it's certainly not
positive definite.
182
00:11:44,270 --> 00:11:47,410
So what could I say
about A A transpose,
183
00:11:47,410 --> 00:11:52,220
if I wanted to, like, say
something true about it?
184
00:11:52,220 --> 00:11:56,360
It's true that it is
positive semi-definite.
185
00:11:56,360 --> 00:12:01,550
If I made this semi-definite,
it would always be true, always.
186
00:12:01,550 --> 00:12:03,950
But if I'm looking
for positive definite,
187
00:12:03,950 --> 00:12:08,990
then I'm looking at the null
space of whatever's here,
188
00:12:08,990 --> 00:12:14,720
and, in this case,
it's got a null space.
189
00:12:14,720 --> 00:12:17,200
So A, A -- eh, shall
we just figure it out,
190
00:12:17,200 --> 00:12:17,720
here?
191
00:12:17,720 --> 00:12:22,090
A A transpose, for that
matrix, will be three-by-three.
192
00:12:22,090 --> 00:12:25,340
If I multiplied
A by A transpose,
193
00:12:25,340 --> 00:12:26,600
what would the first row be?
194
00:12:29,990 --> 00:12:31,960
All zeroes, right?
195
00:12:31,960 --> 00:12:34,970
First row of A A
transpose, could only
196
00:12:34,970 --> 00:12:40,000
be all zeroes, so it's probably
a one there and a one there,
197
00:12:40,000 --> 00:12:41,950
or something like that.
198
00:12:41,950 --> 00:12:45,860
But, I don't even
know if that's right.
199
00:12:45,860 --> 00:12:48,570
But it's all zeroes
there, so it's certainly
200
00:12:48,570 --> 00:12:49,700
not positive definite.
201
00:12:49,700 --> 00:12:53,550
Let me not put anything up
I'm not sh- don't check.
202
00:12:53,550 --> 00:12:54,930
What about this determinant?
203
00:12:54,930 --> 00:12:58,290
Oh, well, I guess --
204
00:12:58,290 --> 00:13:01,470
that's a sort of
tricky question.
205
00:13:01,470 --> 00:13:04,620
Is it true or
false in this case?
206
00:13:04,620 --> 00:13:08,810
It's false, apparently, because
A transpose A, is invertible,
207
00:13:08,810 --> 00:13:13,500
we just got a true for this
one, and we got a false,
208
00:13:13,500 --> 00:13:15,960
we got a z- we got
a non-invertible one
209
00:13:15,960 --> 00:13:16,530
for this one.
210
00:13:16,530 --> 00:13:22,060
So actually, this one
is false, number one.
211
00:13:22,060 --> 00:13:24,590
That surprises us,
actually, because it's,
212
00:13:24,590 --> 00:13:25,930
I mean, why was it tricky?
213
00:13:25,930 --> 00:13:30,120
Because what is true
about determinants?
214
00:13:30,120 --> 00:13:34,110
This would be true if
those matrices were square.
215
00:13:34,110 --> 00:13:39,530
If I have two square matrices,
A and any other matrix B,
216
00:13:39,530 --> 00:13:43,670
could be A transpose, could
be somebody else's matrix.
217
00:13:43,670 --> 00:13:47,090
Then it would be true that
the determinant of B A
218
00:13:47,090 --> 00:13:49,610
would equal the
determinant of A B.
219
00:13:49,610 --> 00:13:54,420
But if the matrices are not
square and it would actually be
220
00:13:54,420 --> 00:13:59,010
true that it would be equal
-- that this would equal
221
00:13:59,010 --> 00:14:03,940
the determinant of A times the
determinant of A transpose.
222
00:14:03,940 --> 00:14:06,610
We could even split up those
two separate determinants.
223
00:14:06,610 --> 00:14:09,080
And, of course,
those would be equal.
224
00:14:09,080 --> 00:14:11,920
But only when A is square.
225
00:14:11,920 --> 00:14:15,860
So that's just, that's a
question that rests on the,
226
00:14:15,860 --> 00:14:18,970
the falseness rests on the
fact that the matrix isn't
227
00:14:18,970 --> 00:14:20,530
square in the first place.
228
00:14:20,530 --> 00:14:22,700
OK, good.
229
00:14:22,700 --> 00:14:25,470
Let's see.
230
00:14:25,470 --> 00:14:28,860
Oh, now, even asks more.
231
00:14:28,860 --> 00:14:34,370
Prove that A transpose
y equals c --
232
00:14:34,370 --> 00:14:37,760
hah-God, it's -- this
question goes on and on.
233
00:14:40,830 --> 00:14:44,130
now I ask you about
A transpose y=c.
234
00:14:46,780 --> 00:14:48,650
So I'm asking you
about the equation --
235
00:14:48,650 --> 00:14:51,100
about the matrix A transpose.
236
00:14:51,100 --> 00:14:57,890
And I want you to prove that
it has at least one solution --
237
00:14:57,890 --> 00:15:04,160
one solution for every c,
every right-hand side c,
238
00:15:04,160 --> 00:15:13,640
and, in fact -- in fact,
infinitely many solutions
239
00:15:13,640 --> 00:15:16,630
for every c.
240
00:15:16,630 --> 00:15:17,910
OK.
241
00:15:17,910 --> 00:15:20,170
Well, none -- none
of this is difficult,
242
00:15:20,170 --> 00:15:25,470
but, it's been a little while.
243
00:15:25,470 --> 00:15:27,900
So we just have to think again.
244
00:15:27,900 --> 00:15:30,980
When I have a system of
equations -- this is --
245
00:15:30,980 --> 00:15:37,690
this matrix A transpose is now,
instead of being three by n,
246
00:15:37,690 --> 00:15:40,970
it's n by three, it's n by m.
247
00:15:40,970 --> 00:15:43,470
Of course.
248
00:15:43,470 --> 00:15:53,900
To show that a system has
at least one solution,
249
00:15:53,900 --> 00:15:56,020
when does this, when
does this system --
250
00:15:56,020 --> 00:15:57,760
when is the system
always solvable?
251
00:16:01,040 --> 00:16:06,810
When it has full row rank,
when the rows are independent.
252
00:16:06,810 --> 00:16:12,100
Here, we have n rows,
and that's the rank.
253
00:16:12,100 --> 00:16:19,450
So at least one solution,
because the number
254
00:16:19,450 --> 00:16:22,500
of rows, which is n,
for the transpose,
255
00:16:22,500 --> 00:16:24,300
is equal to r, the rank.
256
00:16:27,490 --> 00:16:30,860
This A transpose
had independent rows
257
00:16:30,860 --> 00:16:34,530
because A had independent
columns, right?
258
00:16:34,530 --> 00:16:41,480
The original A had independent
columns, when we transpose it,
259
00:16:41,480 --> 00:16:44,430
it has independent rows, so
there's at least one solution.
260
00:16:44,430 --> 00:16:46,660
But now, how do I even know
that there are infinitely
261
00:16:46,660 --> 00:16:48,050
many solutions?
262
00:16:48,050 --> 00:16:49,260
Oh, what do I --
263
00:16:49,260 --> 00:16:52,280
I want to know something
about the null space.
264
00:16:52,280 --> 00:16:57,390
What's the dimension of the
null space of A transpose?
265
00:16:57,390 --> 00:17:00,100
So the answer has got
to be the dimension
266
00:17:00,100 --> 00:17:03,850
of the null space of
A transpose, what's
267
00:17:03,850 --> 00:17:06,119
the general fact?
268
00:17:06,119 --> 00:17:10,790
If A is an m by n
matrix of rank r,
269
00:17:10,790 --> 00:17:14,260
what's the dimension
of A transpose?
270
00:17:14,260 --> 00:17:15,970
The null space of A transpose?
271
00:17:15,970 --> 00:17:20,470
Do you remember that
little fourth subspace
272
00:17:20,470 --> 00:17:24,050
that's tagging along
down in our big picture?
273
00:17:24,050 --> 00:17:26,497
It's dimension was m-r.
274
00:17:30,780 --> 00:17:32,630
And, that's bigger than zero.
275
00:17:32,630 --> 00:17:35,610
m is bigger than r.
276
00:17:35,610 --> 00:17:37,520
So there's a lot
in that null space.
277
00:17:44,520 --> 00:17:47,790
So there's always one
solution because n i- this
278
00:17:47,790 --> 00:17:49,230
is speaking about A transpose.
279
00:17:52,880 --> 00:17:55,730
So for A transpose, the roles
of m and n are reversed,
280
00:17:55,730 --> 00:17:57,540
of course, so I'm --
281
00:17:57,540 --> 00:18:01,430
keep in mind that this
board was about A transpose,
282
00:18:01,430 --> 00:18:04,880
so the roles -- so it's the
null space of a transpose,
283
00:18:04,880 --> 00:18:08,590
and there are m-r
free variables.
284
00:18:08,590 --> 00:18:13,630
OK, that's, like,
just some, review.
285
00:18:13,630 --> 00:18:17,030
Can I take another problem
that's also sort of --
286
00:18:17,030 --> 00:18:24,710
suppose the matrix A has
three columns, v1, v2, v3.
287
00:18:24,710 --> 00:18:28,880
Those are the columns
of the matrix.
288
00:18:28,880 --> 00:18:31,140
All right.
289
00:18:31,140 --> 00:18:33,500
Question A.
290
00:18:33,500 --> 00:18:36,827
Solve Ax=v1-v2+v3.
291
00:18:44,260 --> 00:18:45,670
Tell me what x is.
292
00:18:53,960 --> 00:18:57,520
Well, there, you're
seeing the most --
293
00:18:57,520 --> 00:19:04,380
the one absolutely
essential fact about matrix
294
00:19:04,380 --> 00:19:06,530
multiplication,
how does it work,
295
00:19:06,530 --> 00:19:10,760
when we do it a column at a
time, the very, very first day,
296
00:19:10,760 --> 00:19:13,800
way back in September, we
did multiplication a column
297
00:19:13,800 --> 00:19:14,590
at a time.
298
00:19:14,590 --> 00:19:16,190
So what's x?
299
00:19:16,190 --> 00:19:18,070
Just tell me?
300
00:19:18,070 --> 00:19:19,250
One minus one, one.
301
00:19:19,250 --> 00:19:19,820
Thanks.
302
00:19:19,820 --> 00:19:20,740
OK.
303
00:19:20,740 --> 00:19:22,950
Everybody's got that.
304
00:19:22,950 --> 00:19:23,550
OK?
305
00:19:23,550 --> 00:19:26,750
Then the next question is,
suppose that combination is
306
00:19:26,750 --> 00:19:28,130
zero --
307
00:19:28,130 --> 00:19:31,560
oh, yes, OK, so
question (b) says --
308
00:19:31,560 --> 00:19:42,730
part (b) says, suppose
this thing is zero.
309
00:19:42,730 --> 00:19:45,380
Suppose that's zero.
310
00:19:45,380 --> 00:19:48,520
Then the solution is not unique.
311
00:19:48,520 --> 00:19:51,730
Suppose I want true or false.
312
00:19:51,730 --> 00:19:54,440
-- and a reason.
313
00:19:54,440 --> 00:20:00,330
Suppose this
combination is zero.
314
00:20:00,330 --> 00:20:02,170
v1-v2+v3.
315
00:20:02,170 --> 00:20:06,640
Show that -- what
does that tell me?
316
00:20:06,640 --> 00:20:08,210
So it's a separate
question, maybe
317
00:20:08,210 --> 00:20:11,880
I sort of saved time
by writing it that way,
318
00:20:11,880 --> 00:20:14,620
but it's a totally
separate question.
319
00:20:14,620 --> 00:20:19,280
If I have a matrix, and I know
that column one minus column
320
00:20:19,280 --> 00:20:23,160
two plus column
three is zero, what
321
00:20:23,160 --> 00:20:33,810
does that tell me about whether
the solution is unique or not?
322
00:20:33,810 --> 00:20:36,420
Is there more than one solution?
323
00:20:36,420 --> 00:20:40,330
What's uniqueness about?
324
00:20:40,330 --> 00:20:42,420
Uniqueness is about,
is there anything
325
00:20:42,420 --> 00:20:44,720
in the null space, right?
326
00:20:44,720 --> 00:20:46,490
The solution is
unique when there's
327
00:20:46,490 --> 00:20:49,550
nobody in the null space
except the zero vector.
328
00:20:49,550 --> 00:20:57,670
And, if that's zero, then this
guy would be in the null space.
329
00:20:57,670 --> 00:21:08,020
So if this were zero, then this
x is in the null space of A.
330
00:21:08,020 --> 00:21:18,750
So solutions are never
unique, because I could always
331
00:21:18,750 --> 00:21:26,890
add that to any solution,
and Ax wouldn't change.
332
00:21:26,890 --> 00:21:29,610
So it's always that question.
333
00:21:29,610 --> 00:21:31,760
Is there somebody
in the null space?
334
00:21:31,760 --> 00:21:33,620
OK.
335
00:21:33,620 --> 00:21:37,370
Oh, now, here's a totally
different question.
336
00:21:37,370 --> 00:21:43,770
Suppose those three vectors,
v1, v2, v3, are orthonormal.
337
00:21:43,770 --> 00:21:48,340
So this isn't going to happen
for orthonormal vectors.
338
00:21:48,340 --> 00:21:51,680
OK, so part (c),
forget part (b).
339
00:21:51,680 --> 00:21:52,410
c.
340
00:21:52,410 --> 00:22:05,040
If v1, v2, v3,
are orthonormal --
341
00:22:05,040 --> 00:22:08,725
so that I would usually
have called them q1, q2, q3.
342
00:22:11,540 --> 00:22:16,420
Now, what combination --
oh, here's a nice question,
343
00:22:16,420 --> 00:22:19,020
if I say so myself --
344
00:22:19,020 --> 00:22:24,280
what combination of v1
and v2 is closest to v3?
345
00:22:24,280 --> 00:22:28,120
What point on the
plane of v1 and v2
346
00:22:28,120 --> 00:22:33,100
is the closest point to v3 if
these vectors are orthonormal?
347
00:22:33,100 --> 00:22:33,750
So let me --
348
00:22:33,750 --> 00:22:41,610
I'll start the sentence -- then
the combination something times
349
00:22:41,610 --> 00:22:51,570
v1 plus something times v2 is
the closest combination to v3?
350
00:22:51,570 --> 00:22:53,860
And what's the answer?
351
00:22:53,860 --> 00:22:57,150
What's the closest vector
on that plane to v3?
352
00:22:57,150 --> 00:23:00,030
Zeroes.
353
00:23:00,030 --> 00:23:01,480
Right.
354
00:23:01,480 --> 00:23:07,240
We just imagine the x, y,
z axes, the v1, v2, th- v3
355
00:23:07,240 --> 00:23:13,380
could be the standard
basis, the x, y, z vectors,
356
00:23:13,380 --> 00:23:19,090
and, of course, the point on
the xy plane that's closest
357
00:23:19,090 --> 00:23:24,090
to v3 on the z axis is zero.
358
00:23:24,090 --> 00:23:28,880
So if we're orthonormal,
then the projection
359
00:23:28,880 --> 00:23:34,690
of v3 onto that plane
is perpendicular,
360
00:23:34,690 --> 00:23:36,440
it hits right at zero.
361
00:23:36,440 --> 00:23:39,850
OK, so that's like a quick --
362
00:23:39,850 --> 00:23:43,890
you know, an easy question,
but still brings it out.
363
00:23:43,890 --> 00:23:45,110
OK.
364
00:23:45,110 --> 00:23:56,720
Let me see what, shall I
write down a Markov matrix,
365
00:23:56,720 --> 00:24:01,870
and I'll ask you
for its eigenvalues.
366
00:24:01,870 --> 00:24:02,680
OK.
367
00:24:02,680 --> 00:24:06,070
Here's a Markov matrix --
368
00:24:06,070 --> 00:24:14,210
this -- and, tell
me its eigenvalues.
369
00:24:14,210 --> 00:24:16,020
So here -- I'll
call the matrix A,
370
00:24:16,020 --> 00:24:19,990
and I'll call this as point
two, point four, point four,
371
00:24:19,990 --> 00:24:25,480
point four, point four, point
two, point four, point three,
372
00:24:25,480 --> 00:24:28,391
point three, point four.
373
00:24:28,391 --> 00:24:28,890
OK.
374
00:24:33,340 --> 00:24:39,020
Let's see -- it helps out to
notice that column one plus
375
00:24:39,020 --> 00:24:41,840
column two --
376
00:24:41,840 --> 00:24:46,870
what's interesting about
column one plus column two?
377
00:24:46,870 --> 00:24:50,530
It's twice as much
as column three.
378
00:24:50,530 --> 00:24:53,790
So column one plus column two
equals two times column three.
379
00:24:53,790 --> 00:24:57,300
I put that in there,
column one plus column two
380
00:24:57,300 --> 00:24:59,730
equals twice column three.
381
00:24:59,730 --> 00:25:01,430
That's observation.
382
00:25:01,430 --> 00:25:02,230
OK.
383
00:25:02,230 --> 00:25:04,500
Tell me the eigenvalues
of the matrix.
384
00:25:07,080 --> 00:25:08,455
OK, tell me one eigenvalue?
385
00:25:11,770 --> 00:25:15,230
Because the matrix is singular.
386
00:25:15,230 --> 00:25:17,570
Tell me another eigenvalue?
387
00:25:17,570 --> 00:25:20,320
One, because it's
a Markov matrix,
388
00:25:20,320 --> 00:25:25,120
the columns add to
the all ones vector,
389
00:25:25,120 --> 00:25:31,940
and that will be an
eigenvector of A transpose.
390
00:25:31,940 --> 00:25:33,435
And tell me the
third eigenvalue?
391
00:25:36,740 --> 00:25:38,770
Let's see, to make
the trace come out
392
00:25:38,770 --> 00:25:42,570
right, which is point eight,
we need minus point two.
393
00:25:45,200 --> 00:25:46,270
OK.
394
00:25:46,270 --> 00:25:52,750
And now, suppose I start
the Markov process.
395
00:25:52,750 --> 00:25:56,460
Suppose I start with u(0) --
396
00:25:56,460 --> 00:26:01,490
so I'm going to look at the
powers of A applied to u(0).
397
00:26:01,490 --> 00:26:04,840
This is uk.
398
00:26:04,840 --> 00:26:09,930
And there's my matrix, and
I'm going to let u(0) be --
399
00:26:09,930 --> 00:26:14,860
this is going to
be zero, ten, zero.
400
00:26:14,860 --> 00:26:21,060
And my question is,
what does that approach?
401
00:26:21,060 --> 00:26:24,910
If u(0) is equal to
this -- there is u(0).
402
00:26:24,910 --> 00:26:26,420
Shall I write it in?
403
00:26:26,420 --> 00:26:27,800
Maybe I'll just write in u(0).
404
00:26:30,680 --> 00:26:40,430
A to the k, starting with
ten people in state two,
405
00:26:40,430 --> 00:26:48,210
and every step follows
the Markov rule,
406
00:26:48,210 --> 00:26:52,920
what does the solution
look like after k steps?
407
00:26:52,920 --> 00:26:54,980
Let me just ask you that.
408
00:26:54,980 --> 00:26:58,500
And then, what happens
as k goes to infinity?
409
00:26:58,500 --> 00:27:00,360
This is a steady-state
question, right?
410
00:27:00,360 --> 00:27:02,770
I'm looking for
the steady state.
411
00:27:02,770 --> 00:27:06,090
Actually, the question doesn't
ask for the k step answer,
412
00:27:06,090 --> 00:27:08,380
it just jumps right
away to infinity --
413
00:27:08,380 --> 00:27:15,170
but how would I express
the solution after k steps?
414
00:27:15,170 --> 00:27:23,340
It would be some multiple of
the first eigenvalue to the k-th
415
00:27:23,340 --> 00:27:26,160
power -- times the
first eigenvector,
416
00:27:26,160 --> 00:27:30,210
plus some other multiple
of the second eigenvalue,
417
00:27:30,210 --> 00:27:34,410
times its eigenvector, and
some multiple of the third
418
00:27:34,410 --> 00:27:38,440
eigenvalue, times
its eigenvector.
419
00:27:38,440 --> 00:27:39,285
OK.
420
00:27:39,285 --> 00:27:39,785
Good.
421
00:27:42,290 --> 00:27:49,140
And these eigenvalues are
zero, one, and minus point two.
422
00:27:52,810 --> 00:27:55,290
So what happens as
k goes to infinity?
423
00:27:58,480 --> 00:28:01,550
The only thing that
survives the steady state --
424
00:28:01,550 --> 00:28:07,860
so at u infinity, this
is gone, this is gone,
425
00:28:07,860 --> 00:28:16,640
all that's left is c2x2.
426
00:28:16,640 --> 00:28:18,550
So I'd better find x2.
427
00:28:18,550 --> 00:28:20,700
I've got to find
that eigenvector
428
00:28:20,700 --> 00:28:22,710
to complete the answer.
429
00:28:22,710 --> 00:28:25,260
What's the eigenvector
that corresponds
430
00:28:25,260 --> 00:28:26,400
to lambda equal one?
431
00:28:26,400 --> 00:28:29,650
That's the key eigenvector
in any Markov process,
432
00:28:29,650 --> 00:28:32,430
is that eigenvector.
433
00:28:32,430 --> 00:28:34,550
Lambda equal one
is an eigenvalue,
434
00:28:34,550 --> 00:28:38,280
I need its eigenvector
x2, and then
435
00:28:38,280 --> 00:28:44,480
I need to know how much of it
is in the starting vector u0.
436
00:28:44,480 --> 00:28:45,240
OK.
437
00:28:45,240 --> 00:28:47,880
So, how do I find
that eigenvector?
438
00:28:47,880 --> 00:28:51,500
I guess I subtract one
from the diagonal, right?
439
00:28:51,500 --> 00:28:56,280
So I have minus point
eight, minus point eight,
440
00:28:56,280 --> 00:28:59,660
minus point six, and the
rest, of course, is just --
441
00:28:59,660 --> 00:29:04,360
still point four, point
four, point four, point four,
442
00:29:04,360 --> 00:29:07,150
point three, point
three, and hopefully,
443
00:29:07,150 --> 00:29:19,230
that's a singular matrix, so
I'm looking to solve A minus Ix
444
00:29:19,230 --> 00:29:19,980
equal zero.
445
00:29:19,980 --> 00:29:22,972
Let's see -- can anybody
spot the solution here?
446
00:29:22,972 --> 00:29:24,930
I don't know, I didn't
make it easy for myself.
447
00:29:27,900 --> 00:29:30,630
What do you think there?
448
00:29:30,630 --> 00:29:39,080
Maybe those first two
entries might be --
449
00:29:39,080 --> 00:29:41,500
oh, no, what do you think?
450
00:29:41,500 --> 00:29:44,640
Anybody see it?
451
00:29:44,640 --> 00:29:46,680
We could use elimination
if we were desperate.
452
00:29:49,210 --> 00:29:51,810
Are we that desperate?
453
00:29:51,810 --> 00:29:55,320
Anybody just call out if
you see the vector that's
454
00:29:55,320 --> 00:29:57,980
in that null space.
455
00:29:57,980 --> 00:30:00,390
Eh, there better be a
vector in that null space,
456
00:30:00,390 --> 00:30:03,220
or I'm quitting.
457
00:30:03,220 --> 00:30:11,555
Uh, ha- OK, well, I guess
we could use elimination.
458
00:30:15,200 --> 00:30:18,010
I thought maybe somebody might
see it from further away.
459
00:30:21,100 --> 00:30:23,820
Is there a chance
that these guys are --
460
00:30:23,820 --> 00:30:28,140
could it be that these two are
equal and this is whatever it
461
00:30:28,140 --> 00:30:31,500
takes, like, something
like three, three, two?
462
00:30:31,500 --> 00:30:33,870
Would that possibly work?
463
00:30:33,870 --> 00:30:37,170
I mean, that's great for this
-- no, it's not that great.
464
00:30:37,170 --> 00:30:39,810
Three, three, four --
465
00:30:39,810 --> 00:30:43,930
this is, deeper mathematics
you're watching now.
466
00:30:43,930 --> 00:30:47,460
Three, three, four, is that --
467
00:30:47,460 --> 00:30:48,000
it works!
468
00:30:48,000 --> 00:30:49,030
Don't mess with it!
469
00:30:49,030 --> 00:30:49,910
It works!
470
00:30:49,910 --> 00:30:52,420
Uh, yes.
471
00:30:52,420 --> 00:30:54,140
OK, it works, all right.
472
00:30:54,140 --> 00:31:01,710
And, yes, OK, and, so that's
x2, three, three, four,
473
00:31:01,710 --> 00:31:11,870
and, how much of that vector
is in the starting vector?
474
00:31:11,870 --> 00:31:16,420
Well, we could go through
a complicated process.
475
00:31:16,420 --> 00:31:19,430
But what's the beauty
of Markov things?
476
00:31:19,430 --> 00:31:23,460
That the total number
of the total population,
477
00:31:23,460 --> 00:31:27,870
the sum of these doesn't change.
478
00:31:27,870 --> 00:31:30,200
That the total number of
people, they're moving around,
479
00:31:30,200 --> 00:31:35,130
but they don't get born
or die or get dead.
480
00:31:35,130 --> 00:31:38,520
So there's ten of them at the
start, so there's ten of them
481
00:31:38,520 --> 00:31:41,570
there, so c2 is
actually one, yes.
482
00:31:41,570 --> 00:31:45,380
So that would be the
correct solution.
483
00:31:45,380 --> 00:31:45,880
OK.
484
00:31:45,880 --> 00:31:48,490
That would be the u infinity.
485
00:31:48,490 --> 00:31:49,120
OK.
486
00:31:49,120 --> 00:31:50,890
So I used there,
in that process,
487
00:31:50,890 --> 00:31:53,190
sort of, the main
facts about Markov
488
00:31:53,190 --> 00:31:58,160
matrices to, to get
a jump on the answer.
489
00:31:58,160 --> 00:31:59,090
OK. let's see.
490
00:31:59,090 --> 00:32:05,810
OK, here's some, kind of
quick, short questions.
491
00:32:05,810 --> 00:32:09,770
Uh, maybe I'll move over to
this board, and leave that for
492
00:32:09,770 --> 00:32:11,260
the moment.
493
00:32:11,260 --> 00:32:16,020
I'm looking for
two-by-two matrices.
494
00:32:16,020 --> 00:32:19,610
And I'll read out the property
I want, and you give me
495
00:32:19,610 --> 00:32:23,670
an example, or tell me
there isn't such a matrix.
496
00:32:23,670 --> 00:32:24,640
All right.
497
00:32:24,640 --> 00:32:25,180
Here we go.
498
00:32:25,180 --> 00:32:28,240
First -- so two-by-twos.
499
00:32:28,240 --> 00:32:36,550
First, I want the
projection onto the line
500
00:32:36,550 --> 00:32:41,820
through A equals
four minus three.
501
00:32:46,190 --> 00:32:49,600
So it's a one-dimensional
projection matrix
502
00:32:49,600 --> 00:32:50,430
I'm looking for.
503
00:32:53,520 --> 00:32:56,280
And what's the formula for it?
504
00:32:56,280 --> 00:33:00,950
What's the formula for the
projection matrix P onto a line
505
00:33:00,950 --> 00:33:03,960
through A. And then we'd just
plug in this particular A.
506
00:33:03,960 --> 00:33:08,050
Do you remember that formula?
507
00:33:08,050 --> 00:33:14,150
There's an A and an A
transpose, and normally we
508
00:33:14,150 --> 00:33:17,030
would have an A transpose
A inverse in the middle,
509
00:33:17,030 --> 00:33:20,730
but here we've just got numbers,
so we just divide by it.
510
00:33:20,730 --> 00:33:25,780
And then plug in A
and we've got it.
511
00:33:25,780 --> 00:33:26,850
So, equals.
512
00:33:26,850 --> 00:33:28,290
OK.
513
00:33:28,290 --> 00:33:30,390
You can put in the numbers.
514
00:33:30,390 --> 00:33:31,790
Trivial, right.
515
00:33:31,790 --> 00:33:32,370
OK.
516
00:33:32,370 --> 00:33:33,190
Number two.
517
00:33:38,440 --> 00:33:39,900
So this is a new problem.
518
00:33:39,900 --> 00:33:45,910
The matrix with eigenvalue zero
and three and eigenvectors --
519
00:33:45,910 --> 00:33:49,740
well, let me write these
down. eigenvalue zero,
520
00:33:49,740 --> 00:33:56,320
eigenvector one, two, eigenvalue
three, eigenvector two, one.
521
00:33:59,170 --> 00:34:01,970
I'm giving you the
eigenvalues and eigenvectors
522
00:34:01,970 --> 00:34:04,090
instead of asking for them.
523
00:34:04,090 --> 00:34:05,485
Now I'm asking for the matrix.
524
00:34:10,080 --> 00:34:11,590
What's the matrix, then?
525
00:34:11,590 --> 00:34:12,210
What's A?
526
00:34:17,310 --> 00:34:19,090
Here was a formula,
then we just put
527
00:34:19,090 --> 00:34:21,610
in some numbers, what's
the formula here,
528
00:34:21,610 --> 00:34:26,270
into which we'll just
put the given numbers?
529
00:34:26,270 --> 00:34:30,840
It's the S lambda
S inverse, right?
530
00:34:30,840 --> 00:34:35,290
So it's S, which is
this eigenvector matrix,
531
00:34:35,290 --> 00:34:41,090
it's the lambda, which
is the eigenvalue matrix,
532
00:34:41,090 --> 00:34:44,790
it's the S inverse, whatever
that turns out to be,
533
00:34:44,790 --> 00:34:46,369
let me just leave it as inverse.
534
00:34:49,489 --> 00:34:52,000
That has to be it, right?
535
00:34:52,000 --> 00:34:54,409
Because if we went in
the other direction,
536
00:34:54,409 --> 00:35:00,150
that matrix S would diagonalize
A to produce lambda.
537
00:35:00,150 --> 00:35:02,340
So it's S lambda S inverse.
538
00:35:02,340 --> 00:35:03,200
Good.
539
00:35:03,200 --> 00:35:06,150
OK, ready for number three.
540
00:35:06,150 --> 00:35:13,120
A real matrix that cannot
be factored into A --
541
00:35:13,120 --> 00:35:17,570
I'm looking for a matrix
A that never could
542
00:35:17,570 --> 00:35:24,000
equal B transpose B, for any B.
543
00:35:24,000 --> 00:35:27,840
A two-by-two matrix that could
not be factored in the form B
544
00:35:27,840 --> 00:35:30,540
transpose B.
545
00:35:30,540 --> 00:35:33,470
So all you have to do is think,
well, what does B transpose B,
546
00:35:33,470 --> 00:35:37,060
look like, and then pick
something different.
547
00:35:37,060 --> 00:35:38,530
What do you suggest?
548
00:35:42,690 --> 00:35:43,910
Let's see.
549
00:35:43,910 --> 00:35:46,850
What shall we take for
a matrix that could not
550
00:35:46,850 --> 00:35:49,430
have this form, B transpose B.
551
00:35:49,430 --> 00:35:51,770
Well, what do we know
about B transpose B?
552
00:35:51,770 --> 00:35:54,090
It's always symmetric.
553
00:35:54,090 --> 00:35:56,280
So just give me any
non-symmetric matrix,
554
00:35:56,280 --> 00:35:58,460
it couldn't possibly
have that form.
555
00:35:58,460 --> 00:35:59,160
OK.
556
00:35:59,160 --> 00:36:02,030
And let me ask the fourth
part of this question --
557
00:36:02,030 --> 00:36:07,310
a matrix that has
orthogonal eigenvectors,
558
00:36:07,310 --> 00:36:12,820
but it's not symmetric.
559
00:36:12,820 --> 00:36:15,900
What matrices have
orthogonal eigenvectors,
560
00:36:15,900 --> 00:36:17,655
but they're not
symmetric matrices?
561
00:36:20,560 --> 00:36:28,240
What other families of matrices
have orthogonal eigenvectors?
562
00:36:28,240 --> 00:36:33,130
We know symmetric matrices
do, but others, also.
563
00:36:33,130 --> 00:36:40,030
So I'm looking for
orthogonal eigenvectors,
564
00:36:40,030 --> 00:36:42,540
and, what do you suggest?
565
00:36:47,220 --> 00:36:51,460
The matrix could
be skew-symmetric.
566
00:36:51,460 --> 00:36:55,100
It could be an
orthogonal matrix.
567
00:36:55,100 --> 00:36:59,590
It could be symmetric,
but that was too easy,
568
00:36:59,590 --> 00:37:01,140
so I ruled that out.
569
00:37:01,140 --> 00:37:11,050
It could be skew-symmetric
like one minus one, like that.
570
00:37:11,050 --> 00:37:19,680
Or it could be an orthogonal
matrix like cosine sine,
571
00:37:19,680 --> 00:37:22,250
minus sine, cosine.
572
00:37:22,250 --> 00:37:28,240
All those matrices would
have complex orthogonal
573
00:37:28,240 --> 00:37:30,620
eigenvectors.
574
00:37:30,620 --> 00:37:35,540
But they would be orthogonal,
and so those examples are fine.
575
00:37:35,540 --> 00:37:36,270
OK.
576
00:37:36,270 --> 00:37:45,660
We can continue a little longer
if you would like to, with
577
00:37:45,660 --> 00:37:47,260
these --
578
00:37:47,260 --> 00:37:48,620
from this exam.
579
00:37:48,620 --> 00:37:49,990
From these exams.
580
00:37:49,990 --> 00:37:50,630
Least squares?
581
00:37:53,570 --> 00:37:56,180
OK, here's a least
squares problem in which,
582
00:37:56,180 --> 00:38:00,130
to make life quick,
I've given the answer --
583
00:38:00,130 --> 00:38:03,240
it's like Jeopardy!, right?
584
00:38:03,240 --> 00:38:06,240
I just give the answer,
and you give the question.
585
00:38:06,240 --> 00:38:07,480
OK.
586
00:38:07,480 --> 00:38:12,540
Whoops, sorry.
587
00:38:12,540 --> 00:38:17,710
Let's see, can I stay over
here for the next question?
588
00:38:17,710 --> 00:38:18,210
OK.
589
00:38:22,300 --> 00:38:22,980
least squares.
590
00:38:22,980 --> 00:38:28,370
So I'm giving you the problem,
one, one, one, zero, one, two,
591
00:38:28,370 --> 00:38:35,890
c d equals three, four, one,
and that's b, of course,
592
00:38:35,890 --> 00:38:37,290
this is Ax=b.
593
00:38:40,310 --> 00:38:43,040
And the least
squares solution --
594
00:38:43,040 --> 00:38:46,350
Maybe I put c hat
d hat to emphasize
595
00:38:46,350 --> 00:38:49,670
it's not the true solution.
596
00:38:49,670 --> 00:38:54,520
So the least square solution
-- the hats really go here --
597
00:38:54,520 --> 00:38:58,360
is eleven-thirds and minus one.
598
00:38:58,360 --> 00:39:01,470
Of course, you could have
figured that out in no time.
599
00:39:01,470 --> 00:39:05,390
So this year, I'll ask
you to do it, probably.
600
00:39:05,390 --> 00:39:09,250
But, suppose we're
given the answer,
601
00:39:09,250 --> 00:39:14,240
then let's just
remember what happened.
602
00:39:14,240 --> 00:39:16,050
OK, good question.
603
00:39:16,050 --> 00:39:21,070
What's the projection P of
this vector onto the column
604
00:39:21,070 --> 00:39:22,110
space of that matrix?
605
00:39:25,090 --> 00:39:29,350
So I'll write that
question down, one.
606
00:39:29,350 --> 00:39:30,530
What is P?
607
00:39:30,530 --> 00:39:31,490
The projection.
608
00:39:31,490 --> 00:39:41,290
The projection of b onto the
column space of A is what?
609
00:39:44,860 --> 00:39:50,030
Hopefully, that's what the
least squares problem solved.
610
00:39:50,030 --> 00:39:51,430
What is it?
611
00:39:54,430 --> 00:40:02,220
This was the best solution, it's
eleven-thirds times column one,
612
00:40:02,220 --> 00:40:07,760
plus -- or rather, minus
one times column two.
613
00:40:07,760 --> 00:40:08,260
Right?
614
00:40:08,260 --> 00:40:10,280
That's what least squares did.
615
00:40:10,280 --> 00:40:14,470
It found the combination
of the columns that
616
00:40:14,470 --> 00:40:16,350
was as close as possible to b.
617
00:40:16,350 --> 00:40:18,740
That's what least
squares was doing.
618
00:40:18,740 --> 00:40:20,450
It found the projection.
619
00:40:20,450 --> 00:40:21,960
OK?
620
00:40:21,960 --> 00:40:26,440
Secondly, draw the straight
line problem that corresponds to
621
00:40:26,440 --> 00:40:27,590
this system.
622
00:40:27,590 --> 00:40:32,060
So I guess that the straight
line fitting a straight line
623
00:40:32,060 --> 00:40:35,110
problem, we kind of recognize.
624
00:40:35,110 --> 00:40:37,410
So we recognize,
these are the heights,
625
00:40:37,410 --> 00:40:41,380
and these are the points,
and so at zero, one, two,
626
00:40:41,380 --> 00:40:46,420
the heights are three,
and at t equal to one,
627
00:40:46,420 --> 00:40:50,720
the height is four,
one, two, three, four,
628
00:40:50,720 --> 00:40:53,640
and at t equal to two,
the height is one.
629
00:40:56,180 --> 00:41:01,880
So I'm trying to fit
the best straight line
630
00:41:01,880 --> 00:41:04,470
through those points.
631
00:41:04,470 --> 00:41:04,970
God.
632
00:41:07,640 --> 00:41:10,360
I could fit a
triangle very well,
633
00:41:10,360 --> 00:41:16,130
but, I don't even know which
way the best straight line goes.
634
00:41:16,130 --> 00:41:19,340
Oh, I do know how it goes,
because there's the answer,yes.
635
00:41:19,340 --> 00:41:25,860
It has a height eleven-thirds,
and it has slope minus one,
636
00:41:25,860 --> 00:41:28,590
so it's something like that.
637
00:41:28,590 --> 00:41:29,370
Great.
638
00:41:29,370 --> 00:41:29,930
OK.
639
00:41:29,930 --> 00:41:36,680
Now, finally -- and this
completes the course --
640
00:41:36,680 --> 00:41:41,140
find a different vector
b, not all zeroes,
641
00:41:41,140 --> 00:41:45,310
for which the least square
solution would be zero.
642
00:41:45,310 --> 00:41:50,140
So I want you to
find a different B
643
00:41:50,140 --> 00:41:54,915
so that the least square
solution changes to all zeroes.
644
00:42:00,530 --> 00:42:04,360
So tell me what I'm
really looking for here.
645
00:42:04,360 --> 00:42:08,640
I'm looking for a b where the
best combination of these two
646
00:42:08,640 --> 00:42:11,780
columns is the zero combination.
647
00:42:11,780 --> 00:42:15,200
So what kind of a
vector b I looking for?
648
00:42:15,200 --> 00:42:16,780
I'm looking for
a vector b that's
649
00:42:16,780 --> 00:42:19,230
orthogonal to those columns.
650
00:42:19,230 --> 00:42:20,810
It's orthogonal
to those columns,
651
00:42:20,810 --> 00:42:22,810
it's orthogonal to
the column space,
652
00:42:22,810 --> 00:42:24,880
the best possible answer is
653
00:42:24,880 --> 00:42:25,550
zero.
654
00:42:25,550 --> 00:42:29,870
So a vector b that's orthogonal
to those columns -- let's see,
655
00:42:29,870 --> 00:42:35,830
maybe one of those minus two
of those, and one of those?
656
00:42:35,830 --> 00:42:38,460
That would be orthogonal
to those columns,
657
00:42:38,460 --> 00:42:42,720
and the best vector
would be zero, zero.
658
00:42:42,720 --> 00:42:43,470
OK.
659
00:42:43,470 --> 00:42:46,590
So that's as many questions
as I can do in an hour,
660
00:42:46,590 --> 00:42:50,490
but you get three hours,
and, let me just say,
661
00:42:50,490 --> 00:42:56,100
as I've said by e-mail, thanks
very much for your patience
662
00:42:56,100 --> 00:43:00,060
as this series of
lectures was videotaped,
663
00:43:00,060 --> 00:43:04,490
and, thanks for filling
out these forms,
664
00:43:04,490 --> 00:43:08,250
maybe just leave them on the
table up there as you go out --
665
00:43:08,250 --> 00:43:10,901
and above all, thanks
for taking the course.
666
00:43:10,901 --> 00:43:11,400
Thank you.
667
00:43:11,400 --> 00:43:12,950
Thanks.