1 00:00:00,000 --> 00:00:01,964 [SQUEAKING] 2 00:00:01,964 --> 00:00:03,928 [RUSTLING] 3 00:00:03,928 --> 00:00:05,892 [CLICKING] 4 00:00:12,770 --> 00:00:15,350 GILBERT STRANG: OK, this is about finding 5 00:00:15,350 --> 00:00:17,480 the null space of a matrix A-- 6 00:00:17,480 --> 00:00:20,630 any matrix, square or rectangular. 7 00:00:20,630 --> 00:00:22,560 And what does that mean? 8 00:00:22,560 --> 00:00:25,700 That means, well, in algebra, we're 9 00:00:25,700 --> 00:00:29,210 solving the equation Ax equals 0. 10 00:00:29,210 --> 00:00:33,830 So A is our matrix, x is a vector that we're looking for, 11 00:00:33,830 --> 00:00:38,780 and Ax is a combination of the columns of A. 12 00:00:38,780 --> 00:00:40,880 So we're looking for combinations 13 00:00:40,880 --> 00:00:44,090 of the columns that give the zero vector-- 14 00:00:44,090 --> 00:00:46,520 dependent columns, we'll be saying. 15 00:00:46,520 --> 00:00:49,500 So that's the goal. 16 00:00:49,500 --> 00:00:55,730 And the new start for linear algebra that I've suggested 17 00:00:55,730 --> 00:00:57,710 solves that problem for small matrices. 18 00:00:57,710 --> 00:01:01,370 It kind of just does it for a small matrix. 19 00:01:01,370 --> 00:01:05,580 But for any matrix, or a big one, we need a system. 20 00:01:05,580 --> 00:01:08,270 So this kind of completes the idea 21 00:01:08,270 --> 00:01:15,530 by giving the system, which uses the algorithm 22 00:01:15,530 --> 00:01:18,260 of linear algebra, which is elimination. 23 00:01:18,260 --> 00:01:20,840 So elimination is going to be the key 24 00:01:20,840 --> 00:01:23,990 to solving this problem-- 25 00:01:23,990 --> 00:01:25,800 to finding the null space. 26 00:01:25,800 --> 00:01:27,980 So that's my picture. 27 00:01:27,980 --> 00:01:29,390 Oh, these are the-- 28 00:01:29,390 --> 00:01:36,410 I'll just mention-- the three books which discuss this. 29 00:01:36,410 --> 00:01:40,190 The Introduction to Linear Algebra is the main textbook. 30 00:01:40,190 --> 00:01:44,150 Then the Learning from Data book-- 31 00:01:44,150 --> 00:01:47,330 actually, that's where this new idea got started. 32 00:01:47,330 --> 00:01:49,670 And the Linear Algebra for Everyone 33 00:01:49,670 --> 00:01:53,370 has got the idea more completely. 34 00:01:53,370 --> 00:01:59,450 So I'm sort of speaking about a section out of that third book. 35 00:01:59,450 --> 00:02:00,200 OK. 36 00:02:00,200 --> 00:02:02,900 And they all have websites. 37 00:02:02,900 --> 00:02:11,720 Math.mit.edu, the addresses are there to see more. 38 00:02:11,720 --> 00:02:13,250 Actually, I should say, so these are 39 00:02:13,250 --> 00:02:15,020 the key ideas of this lecture. 40 00:02:15,020 --> 00:02:20,030 And yesterday, I looked at my lecture from a few years 41 00:02:20,030 --> 00:02:22,910 ago-- quite a few years ago-- lecture seven 42 00:02:22,910 --> 00:02:29,270 in the OpenCourseWare series for Math 18.06. 43 00:02:29,270 --> 00:02:33,830 So that was on the same topic, elimination and Ax equals 0. 44 00:02:33,830 --> 00:02:37,670 So a lot of good stuff is there, but there's a little more 45 00:02:37,670 --> 00:02:38,690 to say now. 46 00:02:38,690 --> 00:02:41,700 And that's what this is about today. 47 00:02:41,700 --> 00:02:45,450 So OK, here are key ideas of the lecture. 48 00:02:45,450 --> 00:02:46,740 So the null space-- 49 00:02:46,740 --> 00:02:50,180 so now you see it in writing-- the null space is all 50 00:02:50,180 --> 00:02:53,030 the solutions x to Ax equals 0. 51 00:02:53,030 --> 00:02:55,010 Remember, x is a vector. 52 00:02:55,010 --> 00:03:00,170 And elimination is the key and it keeps the same null space. 53 00:03:00,170 --> 00:03:05,556 And the Matlab command is rref of A. 54 00:03:05,556 --> 00:03:09,650 That's the command that does elimination. 55 00:03:09,650 --> 00:03:18,050 And we'll see the identity matrix, we'll see a matrix F, 56 00:03:18,050 --> 00:03:23,630 and the new idea was to factor the matrix into a column matrix 57 00:03:23,630 --> 00:03:27,050 C times a row matrix R. So this is really 58 00:03:27,050 --> 00:03:29,120 putting all the ideas together. 59 00:03:29,120 --> 00:03:33,980 And we learn that if a matrix is short and wide, 60 00:03:33,980 --> 00:03:36,110 then there are certainly-- 61 00:03:36,110 --> 00:03:38,120 if we've got lots of columns, then some 62 00:03:38,120 --> 00:03:42,200 of them will be dependent and that means that there'll 63 00:03:42,200 --> 00:03:44,690 be solutions to Ax equals 0. 64 00:03:44,690 --> 00:03:47,420 So here we go. 65 00:03:47,420 --> 00:03:50,680 Oh OK, I'm going to start with an example. 66 00:03:50,680 --> 00:03:55,430 And elimination has already done its job. 67 00:03:55,430 --> 00:03:57,520 So what elimination did was to get 68 00:03:57,520 --> 00:04:00,610 to that-- you see that identity matrix in the first two 69 00:04:00,610 --> 00:04:01,300 columns? 70 00:04:01,300 --> 00:04:03,310 1, 0 and 0, 1. 71 00:04:03,310 --> 00:04:07,090 So you can't ask for a simpler matrix than that. 72 00:04:07,090 --> 00:04:12,310 So once we've got that, we can't mess with it, so 3, 5, 4, 6, 73 00:04:12,310 --> 00:04:14,140 we're stuck with-- 74 00:04:14,140 --> 00:04:16,100 those are our other columns. 75 00:04:16,100 --> 00:04:18,970 So the first two columns are independent. 76 00:04:18,970 --> 00:04:23,380 Columns of the identity matrix are very independent. 77 00:04:23,380 --> 00:04:29,130 But then 3, 4 is a combination of those two columns, right? 78 00:04:29,130 --> 00:04:33,760 3, 4 is 3 times the first column plus 4 times the second. 79 00:04:33,760 --> 00:04:38,227 So there is an x in the null space. 80 00:04:38,227 --> 00:04:39,810 That's one of the vectors we're after. 81 00:04:39,810 --> 00:04:42,930 I'll call it a special solution because it just 82 00:04:42,930 --> 00:04:45,030 came specially from one column. 83 00:04:45,030 --> 00:04:48,330 And you see it down below as s1. 84 00:04:48,330 --> 00:04:53,220 So if we took minus 3 of the first column, minus 4 85 00:04:53,220 --> 00:04:57,060 of the second column, and then plus 1 of the third column, 86 00:04:57,060 --> 00:04:58,875 we would have 0. 87 00:04:58,875 --> 00:05:01,600 Ax equals 0-- what we're looking for. 88 00:05:01,600 --> 00:05:04,910 And the second special solution would 89 00:05:04,910 --> 00:05:08,870 come from the last column of the matrix, 5, 6. 90 00:05:08,870 --> 00:05:12,710 So again, that's 5 of the 1, 0 column and 6 91 00:05:12,710 --> 00:05:14,550 of the 0, 1 column. 92 00:05:14,550 --> 00:05:20,540 And if we put that together into a special solution s2, 93 00:05:20,540 --> 00:05:25,280 we want minus 5 of column 1, minus 6 of column 2, 94 00:05:25,280 --> 00:05:30,440 plus nothing of column 3, plus column 4, giving 0. 95 00:05:30,440 --> 00:05:37,280 So in this case, elimination has produced a simple matrix R-- 96 00:05:37,280 --> 00:05:40,140 simple because it's got the identity there. 97 00:05:40,140 --> 00:05:44,510 So can I show you one more example of R 98 00:05:44,510 --> 00:05:47,990 before I begin to talk about how do we get to R? 99 00:05:47,990 --> 00:05:53,000 OK, so here's another R. A little different, though. 100 00:05:53,000 --> 00:05:56,990 It's different because it's got a row of 0's. 101 00:05:56,990 --> 00:05:59,180 Well that won't pose any problem, 102 00:05:59,180 --> 00:06:02,040 but we just have to think what to do with it. 103 00:06:02,040 --> 00:06:04,880 We always move 0 rows to the bottom. 104 00:06:04,880 --> 00:06:06,320 And there it is. 105 00:06:06,320 --> 00:06:09,590 And it does have an identity matrix 106 00:06:09,590 --> 00:06:16,370 as the first example did, but you see that the 0, 1 is often 107 00:06:16,370 --> 00:06:18,230 column 3. 108 00:06:18,230 --> 00:06:21,440 The identity is in columns 1 and 3 here 109 00:06:21,440 --> 00:06:25,190 and that makes a small change. 110 00:06:25,190 --> 00:06:27,530 But the idea is still the same. 111 00:06:27,530 --> 00:06:30,320 Those two columns are the independent ones. 112 00:06:30,320 --> 00:06:32,390 They're very simple. 113 00:06:32,390 --> 00:06:37,100 The 1, 0, 0 and 0, 1, 0, those are totally different 114 00:06:37,100 --> 00:06:38,330 directions. 115 00:06:38,330 --> 00:06:44,530 Then the 7, 0, 0 is 7 of the first column. 116 00:06:44,530 --> 00:06:51,720 So that spotted a special solution-- an x that I call s1. 117 00:06:51,720 --> 00:06:54,480 At the bottom line, you see the minus 7 118 00:06:54,480 --> 00:06:59,220 of the first column plus 1 of the second column produces 0. 119 00:06:59,220 --> 00:07:01,950 If you just look at those columns, minus 7 120 00:07:01,950 --> 00:07:05,040 of the first plus 1 of the second, everything cancels. 121 00:07:05,040 --> 00:07:10,800 And the other one is going to come from the 8, 9, 0 column. 122 00:07:10,800 --> 00:07:14,400 That'll be 8 of the first column and 9 of the third-- 123 00:07:14,400 --> 00:07:16,470 the two bits of the identity. 124 00:07:16,470 --> 00:07:25,920 So you see that when we get to this reduced row echelon form-- 125 00:07:25,920 --> 00:07:28,650 horrible set of words, but what it means 126 00:07:28,650 --> 00:07:33,780 is elimination has made it as simple as it can be. 127 00:07:33,780 --> 00:07:36,480 OK, I might have something more to say. 128 00:07:36,480 --> 00:07:40,090 Yeah, so this is summarizing what you've just seen. 129 00:07:40,090 --> 00:07:42,930 So we have the simplest case where the identity just 130 00:07:42,930 --> 00:07:47,010 sits there at the left or the more general case 131 00:07:47,010 --> 00:07:50,520 where the identity is mixed in with the other two columns. 132 00:07:50,520 --> 00:07:52,170 We can live with both. 133 00:07:52,170 --> 00:07:55,980 And I'm using F for the other columns-- 134 00:07:55,980 --> 00:07:58,510 the columns that are not part of the identity. 135 00:07:58,510 --> 00:08:01,650 And then it has that extra 0 row. 136 00:08:01,650 --> 00:08:02,310 OK. 137 00:08:02,310 --> 00:08:06,330 And now, I want to write it-- 138 00:08:06,330 --> 00:08:15,660 a key part of this lecture is to see the result this matrix R-- 139 00:08:15,660 --> 00:08:20,010 see it in matrix form instead of a bunch of numbers. 140 00:08:20,010 --> 00:08:24,870 Often, in the computing, you're just pages full of numbers 141 00:08:24,870 --> 00:08:26,650 and you don't see what's happening. 142 00:08:26,650 --> 00:08:31,980 So what we're looking for is the identity matrix 143 00:08:31,980 --> 00:08:36,450 and the non-zero matrix F. 144 00:08:36,450 --> 00:08:40,650 And now, in that R0 case, that second example, 145 00:08:40,650 --> 00:08:44,039 there's a P. What is that matrix P doing there? 146 00:08:44,039 --> 00:08:47,340 Well, it's a permutation matrix-- 147 00:08:47,340 --> 00:08:49,140 an exchange matrix. 148 00:08:49,140 --> 00:08:53,490 Because the identity in this second example 149 00:08:53,490 --> 00:08:55,380 is not in columns 1 and 2. 150 00:08:55,380 --> 00:08:56,640 It's in 1 and 3. 151 00:08:56,640 --> 00:08:59,070 So that P has to put it there. 152 00:08:59,070 --> 00:09:04,990 So that P is-- and because P is on the right, 153 00:09:04,990 --> 00:09:06,480 it'll move columns. 154 00:09:06,480 --> 00:09:10,440 And there I wrote down what that P is. 155 00:09:10,440 --> 00:09:15,360 That exchanges column 2 and 3 and puts the identity 156 00:09:15,360 --> 00:09:18,150 in where we want it. 157 00:09:18,150 --> 00:09:18,720 OK. 158 00:09:18,720 --> 00:09:23,880 So that's the linear algebra-- the matrix notation part. 159 00:09:23,880 --> 00:09:31,420 Oh, and this is-- here, now you're seeing the new start, 160 00:09:31,420 --> 00:09:39,790 this C times R. So that's what I'm suggesting that if you give 161 00:09:39,790 --> 00:09:43,360 me a matrix, the first thing I would do would be to find 162 00:09:43,360 --> 00:09:44,710 independent columns-- 163 00:09:44,710 --> 00:09:48,940 those would go into C. And then the matrix R would-- 164 00:09:48,940 --> 00:09:52,240 well, we've seen the matrix R and that would tell me 165 00:09:52,240 --> 00:09:55,390 the combinations of those C columns 166 00:09:55,390 --> 00:09:57,790 to get A. Yeah, you'll see it. 167 00:09:57,790 --> 00:10:03,460 Well, this box is around all the matrix formulas. 168 00:10:03,460 --> 00:10:08,170 So we're sorting every matrix into independent columns 169 00:10:08,170 --> 00:10:12,190 followed by dependent columns and then this permutation-- 170 00:10:12,190 --> 00:10:14,230 this exchange matrix P-- 171 00:10:14,230 --> 00:10:17,090 if they don't really come in that order. 172 00:10:17,090 --> 00:10:18,530 If we need a new order. 173 00:10:18,530 --> 00:10:23,770 So all this is about A equals CR, the new-- 174 00:10:23,770 --> 00:10:24,790 well, not new. 175 00:10:24,790 --> 00:10:31,420 Not completely new, I'm sure, but it's the factorization 176 00:10:31,420 --> 00:10:33,580 of any matrix. 177 00:10:33,580 --> 00:10:34,080 OK. 178 00:10:34,080 --> 00:10:36,990 And now how do we actually do it if the matrix is big 179 00:10:36,990 --> 00:10:40,920 and we need the computer to help? 180 00:10:40,920 --> 00:10:44,050 Well, we do it by elimination. 181 00:10:44,050 --> 00:10:48,410 OK, this is what elimination is about. 182 00:10:48,410 --> 00:10:52,160 What are we allowed to do in elimination? 183 00:10:52,160 --> 00:10:55,730 We don't want to mess up the equation Ax equals 0. 184 00:10:55,730 --> 00:11:00,200 I'm operating on A, but I'm not creating or losing 185 00:11:00,200 --> 00:11:02,130 any of those solutions. 186 00:11:02,130 --> 00:11:09,140 So if I subtract one row from another row, 187 00:11:09,140 --> 00:11:11,960 or any multiple of one row from another row, 188 00:11:11,960 --> 00:11:14,120 that's the main operation of elimination. 189 00:11:14,120 --> 00:11:17,330 And obviously, the equations are still true. 190 00:11:17,330 --> 00:11:22,010 If I take 3 of equation one away from equation two, 191 00:11:22,010 --> 00:11:24,650 I still have a correct equation. 192 00:11:24,650 --> 00:11:29,750 And I could multiply a row by 15 or any non-zero number. 193 00:11:29,750 --> 00:11:30,770 No change. 194 00:11:30,770 --> 00:11:33,560 And I can switch any two rows. 195 00:11:33,560 --> 00:11:37,460 That third one of switching rows, exchanging rows, 196 00:11:37,460 --> 00:11:41,160 is just to move the 0 row down to the bottom. 197 00:11:41,160 --> 00:11:44,180 This is the example. 198 00:11:44,180 --> 00:11:45,330 Can you see those numbers? 199 00:11:45,330 --> 00:11:49,340 So I'm giving an example of a matrix that leads to that very 200 00:11:49,340 --> 00:11:52,610 same R that we worked out. 201 00:11:52,610 --> 00:11:54,500 So you see what I'm doing now. 202 00:11:54,500 --> 00:11:55,910 I'm backing up. 203 00:11:55,910 --> 00:12:01,180 I know the R I want, but I'm given some matrix A, 204 00:12:01,180 --> 00:12:06,360 and it's these elimination steps that will lead me to R. OK. 205 00:12:06,360 --> 00:12:09,540 So you see A over on the left? 206 00:12:09,540 --> 00:12:13,410 So it's a full matrix A. No identity there. 207 00:12:13,410 --> 00:12:16,530 But I'm allowed to do elimination. 208 00:12:16,530 --> 00:12:19,920 I'm allowed to do these three steps when I want to. 209 00:12:19,920 --> 00:12:21,540 So what shall I do? 210 00:12:21,540 --> 00:12:25,080 Probably, at this point, you've seen elimination 211 00:12:25,080 --> 00:12:26,820 as much as you want to, but allow me 212 00:12:26,820 --> 00:12:29,190 to say it once or twice more. 213 00:12:29,190 --> 00:12:34,050 So I would take 3 of row 1 away from row 2. 214 00:12:34,050 --> 00:12:38,640 That would produce-- that 3 that's in the original matrix 215 00:12:38,640 --> 00:12:40,500 would turn into a 0. 216 00:12:40,500 --> 00:12:44,290 So when you take 3 of that first row, 217 00:12:44,290 --> 00:12:47,910 that would be 3, 6, 33, 51. 218 00:12:47,910 --> 00:12:52,080 I subtract it and I get just that 0, 1, 4, 6. 219 00:12:52,080 --> 00:12:55,020 So I've got a much better second row. 220 00:12:55,020 --> 00:12:59,370 Now, I'm going to use the second row going upwards. 221 00:12:59,370 --> 00:13:03,240 So I'll take 2 of the second row away from the first row. 222 00:13:03,240 --> 00:13:06,390 That'll knock out the 2 that I don't want 223 00:13:06,390 --> 00:13:09,720 and it'll produce the identity that I do want. 224 00:13:09,720 --> 00:13:10,800 So do you see that? 225 00:13:10,800 --> 00:13:15,900 If I take 2 of that last row, that 0, 2, 8, 12, 226 00:13:15,900 --> 00:13:19,170 when I take 0, 2, 8, 12 away from the top row, 227 00:13:19,170 --> 00:13:21,120 I'm left with 1 and 0. 228 00:13:21,120 --> 00:13:25,200 And 3 and 5 are the numbers that happen to be left. 229 00:13:25,200 --> 00:13:27,360 OK. 230 00:13:27,360 --> 00:13:30,130 So really, what has elimination done? 231 00:13:30,130 --> 00:13:32,280 This is an important idea that I think 232 00:13:32,280 --> 00:13:35,970 I never caught onto when I was learning linear algebra. 233 00:13:35,970 --> 00:13:41,595 Elimination has found the inverse of this 1, 2, 3, 234 00:13:41,595 --> 00:13:44,220 7-- of this 2 by 2 matrix. 235 00:13:44,220 --> 00:13:47,340 Because it's started with 1, 2, 3, 7 and it's 236 00:13:47,340 --> 00:13:49,320 ended with the identity. 237 00:13:49,320 --> 00:13:51,810 So that must have inverted that matrix. 238 00:13:51,810 --> 00:13:55,470 And then it had to apply the inverse 239 00:13:55,470 --> 00:14:00,810 to the second half of the matrix and that produced the 3, 4, 5, 240 00:14:00,810 --> 00:14:02,520 6. 241 00:14:02,520 --> 00:14:04,830 So you see, this is-- oh, I think 242 00:14:04,830 --> 00:14:09,090 there's probably an example that leads to number two. 243 00:14:09,090 --> 00:14:11,280 Here, I must have thought ahead. 244 00:14:11,280 --> 00:14:15,090 So it says on the screen what I just 245 00:14:15,090 --> 00:14:19,590 said, that elimination has inverted the leading matrix. 246 00:14:19,590 --> 00:14:25,650 And then it's written out the step 247 00:14:25,650 --> 00:14:32,280 H equal WF that connects the original dependent columns 248 00:14:32,280 --> 00:14:37,680 to the final dependent columns in F, 3, 4, 5, 6. 249 00:14:37,680 --> 00:14:38,520 OK. 250 00:14:38,520 --> 00:14:42,270 I think elimination is just a matter of practice 251 00:14:42,270 --> 00:14:44,550 and you probably have done it. 252 00:14:44,550 --> 00:14:48,740 And here is an important point about elimination, 253 00:14:48,740 --> 00:14:52,830 that you could do things in different orders. 254 00:14:52,830 --> 00:14:56,210 I spoke and I did them in a sort of natural order, 255 00:14:56,210 --> 00:14:59,130 but you could do it other ways. 256 00:14:59,130 --> 00:15:02,910 But you wouldn't get a different outcome. 257 00:15:02,910 --> 00:15:05,430 You'd get the same matrix R. 258 00:15:05,430 --> 00:15:09,960 Well, because the equations are still true at every point. 259 00:15:09,960 --> 00:15:13,020 The x's, the null space of a matrix, 260 00:15:13,020 --> 00:15:17,970 is not changing by my elimination steps. 261 00:15:17,970 --> 00:15:19,500 Yeah. 262 00:15:19,500 --> 00:15:21,240 So let's just repeat this. 263 00:15:21,240 --> 00:15:28,290 When you finally get to R, when you've done the elimination 264 00:15:28,290 --> 00:15:31,110 and you get to R with its identity matrix, 265 00:15:31,110 --> 00:15:35,910 that identity matrix in R tells you which columns at the start 266 00:15:35,910 --> 00:15:38,290 were independent. 267 00:15:38,290 --> 00:15:44,850 So you need to have a way to do that and now we have a way. 268 00:15:44,850 --> 00:15:49,960 In the other opening talks in this series-- 269 00:15:49,960 --> 00:15:52,990 in this series about the new start-- 270 00:15:52,990 --> 00:15:55,720 the matrices were nice and small so I 271 00:15:55,720 --> 00:15:57,160 didn't need much of a system. 272 00:15:57,160 --> 00:15:58,900 I could practically wing it. 273 00:15:58,900 --> 00:16:01,730 But now, we need a system. 274 00:16:01,730 --> 00:16:08,080 And of course, if we want to use the Matlab or other code, 275 00:16:08,080 --> 00:16:09,740 it has to have a system. 276 00:16:09,740 --> 00:16:16,120 So anyway, we now have a system of row elimination steps 277 00:16:16,120 --> 00:16:19,990 and we know what it ends with. 278 00:16:19,990 --> 00:16:25,000 It ends with an identity matrix of the right size, a matrix F-- 279 00:16:25,000 --> 00:16:26,920 that's the other columns-- 280 00:16:26,920 --> 00:16:30,910 and then possibly a permutation to tell us the order. 281 00:16:30,910 --> 00:16:34,450 That's elimination. 282 00:16:34,450 --> 00:16:37,410 OK and here's the second example. 283 00:16:37,410 --> 00:16:39,470 So we're moving along. 284 00:16:39,470 --> 00:16:42,230 You've got to pay attention because this 285 00:16:42,230 --> 00:16:46,040 is all there is is two examples and then 286 00:16:46,040 --> 00:16:48,270 put the ideas together. 287 00:16:48,270 --> 00:16:50,420 So here is an example of a matrix A 288 00:16:50,420 --> 00:16:52,970 with numbers as big as 97. 289 00:16:52,970 --> 00:16:54,680 I don't know how I got there. 290 00:16:54,680 --> 00:16:55,610 OK. 291 00:16:55,610 --> 00:17:00,770 So there are no 0's in the original matrix A 292 00:17:00,770 --> 00:17:04,260 but elimination aims to get 0's. 293 00:17:04,260 --> 00:17:04,760 Oh, here. 294 00:17:04,760 --> 00:17:08,990 It got 0's very quickly because the second row 295 00:17:08,990 --> 00:17:15,200 of that matrix, 2, 14, 6, 70, that's 296 00:17:15,200 --> 00:17:16,859 just twice the first row. 297 00:17:16,859 --> 00:17:20,599 So when I subtract 2 times the first row from the second row, 298 00:17:20,599 --> 00:17:22,050 I get that 0 row. 299 00:17:22,050 --> 00:17:25,310 So I'm now one step along with a row of 0's. 300 00:17:25,310 --> 00:17:26,690 That's great. 301 00:17:26,690 --> 00:17:31,250 And the third row improved too because I 302 00:17:31,250 --> 00:17:34,340 took 2 of the first row away from the third row 303 00:17:34,340 --> 00:17:40,590 and that gave me a 0, 0 at the start and then a 3, 27. 304 00:17:40,590 --> 00:17:41,720 So are we OK? 305 00:17:41,720 --> 00:17:44,120 We're at the second matrix now. 306 00:17:44,120 --> 00:17:45,350 It's not finished. 307 00:17:45,350 --> 00:17:50,030 It's not in its R form, in its echelon form, 308 00:17:50,030 --> 00:17:52,190 but it's a lot closer. 309 00:17:52,190 --> 00:17:52,730 OK. 310 00:17:52,730 --> 00:17:55,100 So what do we still have to do? 311 00:17:55,100 --> 00:17:59,870 Well I look at that 3, and I'm looking to get the identity 312 00:17:59,870 --> 00:18:02,390 matrix, so I'll divide that-- 313 00:18:02,390 --> 00:18:05,900 sooner or later, I'm going to divide that last row by 3 314 00:18:05,900 --> 00:18:08,370 and get 0, 0, 1, 9. 315 00:18:08,370 --> 00:18:12,360 But another thing I want to do is clean up the first row. 316 00:18:12,360 --> 00:18:16,130 So I subtract that last row from the first row. 317 00:18:16,130 --> 00:18:17,330 You see that? 318 00:18:17,330 --> 00:18:20,220 See, I'm much closer to the identity. 319 00:18:20,220 --> 00:18:23,840 So I'm now moved on to the third matrix. 320 00:18:23,840 --> 00:18:26,900 And if you've done a lot of these, you know what I'm doing. 321 00:18:29,870 --> 00:18:32,300 I'm working a column at a time. 322 00:18:32,300 --> 00:18:37,970 And notice, column 2, I can't do anything with. 323 00:18:37,970 --> 00:18:39,950 Column 2 is a dependent column. 324 00:18:39,950 --> 00:18:41,930 It's seven times column 1. 325 00:18:41,930 --> 00:18:43,880 I can't improve column 2. 326 00:18:43,880 --> 00:18:47,720 But I can improve column 3 and that's what we do. 327 00:18:47,720 --> 00:18:55,730 Divide that third, that 0, 0, 3, 27, by 3 and exchange rows. 328 00:18:55,730 --> 00:18:59,930 Then you see that the result is the R0. 329 00:18:59,930 --> 00:19:04,110 So it's the R0 with the 0 row. 330 00:19:04,110 --> 00:19:04,610 Yeah. 331 00:19:04,610 --> 00:19:10,310 So just to repeat, this is a matrix A 332 00:19:10,310 --> 00:19:13,400 that leads through the steps of elimination, 333 00:19:13,400 --> 00:19:17,360 which we're remembering to R0. 334 00:19:17,360 --> 00:19:21,980 So we're really finished with this matrix A 335 00:19:21,980 --> 00:19:24,950 because this shows how to get it to R0, 336 00:19:24,950 --> 00:19:28,280 and then we've already seen with R0, 337 00:19:28,280 --> 00:19:32,540 we've figured out which were the special solutions-- 338 00:19:32,540 --> 00:19:35,910 the vectors in the null space. 339 00:19:35,910 --> 00:19:37,370 So we're good. 340 00:19:37,370 --> 00:19:39,980 We really have done the job. 341 00:19:39,980 --> 00:19:46,400 And it remains to just see a little bit, what did we do 342 00:19:46,400 --> 00:19:55,040 and what's going on with these other two columns? 343 00:19:55,040 --> 00:19:58,280 The F columns that are not the identity. 344 00:19:58,280 --> 00:20:01,880 And you see, what we've done is-- 345 00:20:01,880 --> 00:20:10,490 well, we started with columns 1 and 3. 346 00:20:10,490 --> 00:20:13,130 Yeah, so that's our matrix C. Those are the two 347 00:20:13,130 --> 00:20:15,330 independent columns. 348 00:20:15,330 --> 00:20:16,590 Well how do I know? 349 00:20:16,590 --> 00:20:18,720 Where did C come from? 350 00:20:18,720 --> 00:20:20,520 Because that's part of this search, 351 00:20:20,520 --> 00:20:22,410 where are the independent columns. 352 00:20:22,410 --> 00:20:25,410 They're the ones that end up with the identity 353 00:20:25,410 --> 00:20:27,660 because the identity is the way to go 354 00:20:27,660 --> 00:20:30,180 to the right independent columns. 355 00:20:30,180 --> 00:20:34,150 So they're columns 1 and 3 of the original matrix. 356 00:20:34,150 --> 00:20:35,550 So you see C there? 357 00:20:35,550 --> 00:20:37,920 1, 2, 2 and 3, 6, 9. 358 00:20:37,920 --> 00:20:44,660 Then F is the part of R from the dependent column. 359 00:20:44,660 --> 00:20:46,830 So that's the 7, 8, 0, 9. 360 00:20:46,830 --> 00:20:50,340 And then I'm seeing the dependent columns. 361 00:20:50,340 --> 00:20:50,840 Yeah. 362 00:20:50,840 --> 00:20:57,770 You have to do this a few times, but all the ideas now are-- 363 00:20:57,770 --> 00:21:00,860 and there I've put it together. 364 00:21:00,860 --> 00:21:04,550 So the idea is, with a small matrix, 365 00:21:04,550 --> 00:21:08,450 like only three columns and maybe only two rows, 366 00:21:08,450 --> 00:21:13,380 we could find C and R almost by sight. 367 00:21:13,380 --> 00:21:19,220 But now, even where we're up to four columns and three rows, 368 00:21:19,220 --> 00:21:20,510 we need a method. 369 00:21:20,510 --> 00:21:23,630 And that's the point of this talk and it was also 370 00:21:23,630 --> 00:21:28,520 the point of lecture seven in the original 18.06-- 371 00:21:28,520 --> 00:21:31,940 Math 18.06 on OpenCourseWare. 372 00:21:31,940 --> 00:21:35,000 OK, so we've found C and R. Good. 373 00:21:35,000 --> 00:21:36,170 Good. 374 00:21:36,170 --> 00:21:40,430 I think it's just congratulating ourselves from here in. 375 00:21:40,430 --> 00:21:43,370 These are remembering the special solutions. 376 00:21:43,370 --> 00:21:46,080 Oh, and then why not-- 377 00:21:46,080 --> 00:21:48,110 you remember those? 378 00:21:48,110 --> 00:21:51,240 That second column was 7 of the first column, 379 00:21:51,240 --> 00:21:55,160 so that gave that special solution, that combination, 380 00:21:55,160 --> 00:21:56,450 to give 0. 381 00:21:56,450 --> 00:22:00,140 And then the other combination gave the 0 vector. 382 00:22:00,140 --> 00:22:03,060 So we've got two special solutions. 383 00:22:03,060 --> 00:22:05,630 Oh, and here is the general principle. 384 00:22:05,630 --> 00:22:06,330 Yeah. 385 00:22:06,330 --> 00:22:06,830 Yeah. 386 00:22:06,830 --> 00:22:07,730 It's very simple. 387 00:22:07,730 --> 00:22:12,850 After you've got this matrix down to identity and F, 388 00:22:12,850 --> 00:22:19,520 then the solutions x should have a minus FI. 389 00:22:19,520 --> 00:22:22,430 You remember the minus signs that we saw? 390 00:22:22,430 --> 00:22:24,830 So all I'm saying here on the first line 391 00:22:24,830 --> 00:22:32,340 is that the matrix IF times minus FI gives the 0 matrix. 392 00:22:32,340 --> 00:22:33,750 Of course it does. 393 00:22:33,750 --> 00:22:38,400 And then the second line is the little bit of special trick 394 00:22:38,400 --> 00:22:43,380 that you have to do if there's a P, a permutation, 395 00:22:43,380 --> 00:22:45,840 an exchange involved here. 396 00:22:45,840 --> 00:22:49,140 When the identity matrix is not sitting where you want it, 397 00:22:49,140 --> 00:22:51,660 you have to have a P to put it there. 398 00:22:51,660 --> 00:22:56,610 Then P transpose has to come into the solutions. 399 00:22:56,610 --> 00:22:57,330 Yeah. 400 00:22:57,330 --> 00:22:59,940 Yeah, because P times P transpose. 401 00:22:59,940 --> 00:23:03,330 So remember about permutation matrices. 402 00:23:03,330 --> 00:23:08,580 Linear algebra is about learning different special character 403 00:23:08,580 --> 00:23:09,900 of different matrices. 404 00:23:09,900 --> 00:23:16,230 And permutations just take the identity matrix and mess around 405 00:23:16,230 --> 00:23:21,810 with the rows or mess around with the columns, same thing. 406 00:23:21,810 --> 00:23:25,740 And then P transpose unmesses it. 407 00:23:25,740 --> 00:23:27,940 Brings it back to the identity. 408 00:23:27,940 --> 00:23:31,020 So P times P transpose is the identity. 409 00:23:31,020 --> 00:23:34,380 OK, so we're just about there. 410 00:23:34,380 --> 00:23:38,130 Oh, then this is the little point I made-- 411 00:23:38,130 --> 00:23:39,960 well, it's an important point. 412 00:23:39,960 --> 00:23:45,980 Suppose I have five columns and only three rows. 413 00:23:45,980 --> 00:23:49,430 That means I've got five unknowns and only 414 00:23:49,430 --> 00:23:51,150 three equations. 415 00:23:51,150 --> 00:23:53,360 So there are going to be solutions to that. 416 00:23:53,360 --> 00:23:58,520 There are going to be solutions other than the 0 solution. 417 00:23:58,520 --> 00:24:02,510 The 0 combination will surely give 418 00:24:02,510 --> 00:24:09,020 0, but if I have five unknowns, five columns in there, 419 00:24:09,020 --> 00:24:13,260 and I have only three equations, three rows-- 420 00:24:13,260 --> 00:24:16,460 so five greater than three, n greater than m-- 421 00:24:16,460 --> 00:24:20,640 then there will be some columns in F. 422 00:24:20,640 --> 00:24:22,850 There'll be some non-zero solutions. 423 00:24:22,850 --> 00:24:25,590 And these examples show. 424 00:24:25,590 --> 00:24:30,020 So take that matrix M at the end there. 425 00:24:30,020 --> 00:24:36,020 You see there's a matrix with four columns and only two rows. 426 00:24:36,020 --> 00:24:38,320 So I've got four vectors in the plane. 427 00:24:38,320 --> 00:24:40,630 If I have four vectors in the plane, 428 00:24:40,630 --> 00:24:44,470 there are combinations that give 0. 429 00:24:44,470 --> 00:24:47,210 They can complete triangles. 430 00:24:47,210 --> 00:24:51,850 So in that case, rank would be 2. 431 00:24:51,850 --> 00:24:58,000 So we're really seeing this part of linear algebra, 432 00:24:58,000 --> 00:25:02,950 to simplify the matrix by elimination so that all 433 00:25:02,950 --> 00:25:07,250 the main facts are clear. 434 00:25:07,250 --> 00:25:07,750 Yeah. 435 00:25:07,750 --> 00:25:14,020 And oh, if you really want to see it in shorthand-- 436 00:25:14,020 --> 00:25:18,010 I'm not necessarily pushing this last idea, 437 00:25:18,010 --> 00:25:20,620 but if you want to see it in a shorthand, 438 00:25:20,620 --> 00:25:25,570 we could think of the matrix as having just four blocks. 439 00:25:25,570 --> 00:25:28,060 So it's a 2 by 2 matrix, but unfortunately, 440 00:25:28,060 --> 00:25:31,510 each of those guys is a block. 441 00:25:31,510 --> 00:25:35,770 So W is the one in the corner that we talked about earlier 442 00:25:35,770 --> 00:25:38,230 that gets inverted. 443 00:25:38,230 --> 00:25:41,200 So if we look at the last line, that 444 00:25:41,200 --> 00:25:47,130 W in the corner, that ended up as the identity. 445 00:25:47,130 --> 00:25:53,550 The second block row, the J, K, ended up as 0's. 446 00:25:53,550 --> 00:26:00,180 And the one remaining guy, it tells us the combinations 447 00:26:00,180 --> 00:26:02,130 that we are looking for-- 448 00:26:02,130 --> 00:26:05,880 is this W inverse H. 449 00:26:05,880 --> 00:26:09,530 So let me look at that equation in the box 450 00:26:09,530 --> 00:26:13,520 W, H, J, K, that's the matrix I'm starting with. 451 00:26:13,520 --> 00:26:17,900 I'm going to invert W and I know that J, K is 452 00:26:17,900 --> 00:26:24,470 some combination of W, H. So when I invert W, I get-- 453 00:26:24,470 --> 00:26:29,000 the first elimination produces the first row, 454 00:26:29,000 --> 00:26:30,630 starting with the identity. 455 00:26:30,630 --> 00:26:34,850 And when I apply that to the second, the J, K row, 456 00:26:34,850 --> 00:26:36,620 those go to 0. 457 00:26:36,620 --> 00:26:42,380 So that's what elimination is doing in real shorthand. 458 00:26:42,380 --> 00:26:46,850 It's taking a 2 by 2 block matrix to that matrix R 459 00:26:46,850 --> 00:26:51,080 at the end, with a 0 block row and an identity 460 00:26:51,080 --> 00:26:53,360 block in the upper corner. 461 00:26:53,360 --> 00:26:58,700 So that is elimination and the solution of-- 462 00:26:58,700 --> 00:27:00,600 and finding the null space. 463 00:27:00,600 --> 00:27:04,670 So really, this completes the job 464 00:27:04,670 --> 00:27:10,800 of the first topic in linear algebra, is identifying-- 465 00:27:10,800 --> 00:27:15,120 understanding these four fundamental subspaces. 466 00:27:15,120 --> 00:27:16,980 The null space being one of them. 467 00:27:16,980 --> 00:27:18,900 The column space being another. 468 00:27:18,900 --> 00:27:22,320 We understand now which columns are independent. 469 00:27:22,320 --> 00:27:25,440 And the row space being another. 470 00:27:25,440 --> 00:27:30,330 And so we're really understanding 471 00:27:30,330 --> 00:27:35,100 the idea of these four fundamental subspaces that 472 00:27:35,100 --> 00:27:38,160 go into the big picture of linear algebra. 473 00:27:38,160 --> 00:27:45,920 So this completes the first major stage of a linear algebra 474 00:27:45,920 --> 00:27:46,820 course. 475 00:27:46,820 --> 00:27:50,960 And then what's to follow will come. 476 00:27:50,960 --> 00:27:55,790 Eigenvalues, singular values, applications of all kinds. 477 00:27:55,790 --> 00:27:57,050 Least squares. 478 00:27:57,050 --> 00:27:57,560 Good. 479 00:27:57,560 --> 00:27:58,220 Good. 480 00:27:58,220 --> 00:27:59,300 Good math. 481 00:27:59,300 --> 00:28:01,220 And thank you very much. 482 00:28:01,220 --> 00:28:07,790 So that's my summary of finding the null space. 483 00:28:07,790 --> 00:28:09,340 Good.