1 00:00:00 --> 00:00:06 OK, here's linear algebra lecture seven. 2 00:00:06 --> 00:00:15 I've been talking about vector spaces and specially the null 3 00:00:15 --> 00:00:23 space of a matrix and the column space of a matrix. 4 00:00:23 --> 00:00:27 What's in those spaces. 5 00:00:27 --> 00:00:34 Now I want to actually describe them. 6 00:00:34 --> 00:00:39 How do you describe all the vectors that are in those 7 00:00:39 --> 00:00:40 spaces? 8 00:00:40 --> 00:00:43 How do you compute these things? 9 00:00:43 --> 00:00:46 So this is the, turning the idea, 10 00:00:46 --> 00:00:49 the definition, into an algorithm. 11 00:00:49 --> 00:00:53 What's the algorithm for solving A x =0? 12 00:00:53 --> 00:00:58 So that's the null space that I'm interested in. 13 00:00:58 --> 00:01:03 So can I take a particular matrix A and describe the 14 00:01:03 --> 00:01:06 natural algorithm, and I'll execute it for that 15 00:01:06 --> 00:01:08 matrix -- here we go. 16 00:01:08 --> 00:01:11 So let me take the matrix as an example. 17 00:01:11 --> 00:01:16 So we're definitely talking rectangular matrices in this 18 00:01:16 --> 00:01:16 chapter. 19 00:01:16 --> 00:01:20 So I'll make, I'll have four columns. 20 00:01:20 --> 00:01:22 And three rows. 21 00:01:22 --> 00:01:28 Two four six eight and three six eight ten. 22 00:01:28 --> 00:01:28 OK. 23 00:01:28 --> 00:01:34 If I just look at those columns, and rows, 24 00:01:34 --> 00:01:41 well, I notice right away that column two is a multiple of 25 00:01:41 --> 00:01:43.09 column one. 26 00:01:43.09 --> 00:01:49 It's in the same direction as column one. 27 00:01:49 --> 00:01:52.03 It's not independent. 28 00:01:52.03 --> 00:01:56 I'll expect to discover that in the process. 29 00:01:56 --> 00:02:01 Actually, with rows, I notice that that row plus 30 00:02:01 --> 00:02:04 this row gives the third row. 31 00:02:04 --> 00:02:07.82 So the third row is not independent. 32 00:02:07.82 --> 00:02:12 So, all that should come out of elimination. 33 00:02:12 --> 00:02:17 So now what I -- my algorithm is elimination, 34 00:02:17 --> 00:02:22 but extended now to the rectangular case, 35 00:02:22 --> 00:02:28 where we have to continue even if there's zeros in the pivot 36 00:02:28 --> 00:02:30 position, we go on. 37 00:02:30 --> 00:02:35 OK, so let me execute elimination for that matrix. 38 00:02:35 --> 00:02:41 My goal is always, while I'm doing elimination -- 39 00:02:41 --> 00:02:44 I'm not changing the null space. 40 00:02:44 --> 00:02:46 That's very important, right? 41 00:02:46 --> 00:02:50 I'm solving A x equals zero by elimination, and when I do these 42 00:02:50 --> 00:02:55 operations that you already know, when I subtract a multiple 43 00:02:55 --> 00:02:59.69 of one equation from another equation, I'm not changing the 44 00:02:59.69 --> 00:03:00 solutions. 45 00:03:00 --> 00:03:03 So I'm not changing the null space. 46 00:03:03 --> 00:03:07 Actually, I changing the column space, as you'll see. 47 00:03:07 --> 00:03:09 So you have to pay attention. 48 00:03:09 --> 00:03:11 What does elimination leave unchanged? 49 00:03:11 --> 00:03:15.55 And the answer is the solutions to the system are not changed 50 00:03:15.55 --> 00:03:19.45 because I'm doing the same thing to -- I'm doing a legitimate 51 00:03:19.45 --> 00:03:21 operations on the equations. 52 00:03:21 --> 00:03:26 Of course, on the right hand side it's always zero, 53 00:03:26 --> 00:03:29 and I don't plan to write zero all the time. 54 00:03:29 --> 00:03:33.65 OK, so I'm really just working on the left side, 55 00:03:33.65 --> 00:03:37 but the right side is, is keeping up always zeros. 56 00:03:37 --> 00:03:40 OK, so what's elimination? 57 00:03:40 --> 00:03:46 Well, you know where the first pivot is and you know what to do 58 00:03:46 --> 00:03:46 with it. 59 00:03:46 --> 00:03:50 So can I just take the first step below here? 60 00:03:50 --> 00:03:53 So that pivot row is fine. 61 00:03:53 --> 00:03:58 I take two times that row away from this one and I get zero 62 00:03:58 --> 00:03:58 zero. 63 00:03:58 --> 00:04:01 That's signaling a difficulty. 64 00:04:01 --> 00:04:06 Two, two twos away from the six leaves me with a two. 65 00:04:06 --> 00:04:10 Two twos away from the eight leaves me with a four. 66 00:04:10 --> 00:04:13 And now three of those away from here is zero, 67 00:04:13 --> 00:04:17 again another zero, three twos away from that eight 68 00:04:17 --> 00:04:20 is the two, three twos away from that ten is a four. 69 00:04:20 --> 00:04:20 OK. 70 00:04:20 --> 00:04:24 That's the first stage of elimination. 71 00:04:24 --> 00:04:27 I've got the first column straight. 72 00:04:27 --> 00:04:32 So of course I move on to the second column. 73 00:04:32 --> 00:04:36 I look in that position, I see a zero. 74 00:04:36 --> 00:04:41 I look below it, hoping for a non-zero that I 75 00:04:41 --> 00:04:43 can do a row exchange. 76 00:04:43 --> 00:04:46 But it's zero below. 77 00:04:46 --> 00:04:50 So that's telling me that that column is -- well, 78 00:04:50 --> 00:04:55 what it's really going to be telling me is that that column 79 00:04:55 --> 00:04:59 is a combination of the earlier columns. 80 00:04:59 --> 00:05:03 It's that second column is dependent on the earlier 81 00:05:03 --> 00:05:04 columns. 82 00:05:04 --> 00:05:06.68 But I don't stop to think here. 83 00:05:06.68 --> 00:05:10 In that column there's nothing to do. 84 00:05:10 --> 00:05:12 I go on to the next. 85 00:05:12 --> 00:05:14 So here's the next pivot. 86 00:05:14 --> 00:05:19 So there's the first pivot and there's the second pivot, 87 00:05:19 --> 00:05:23 and I just keep this elimination going downwards. 88 00:05:23 --> 00:05:29 So, so the next step keeps the first row, keeps the second row 89 00:05:29 --> 00:05:32 with its pivot, so I've got my two pivots, 90 00:05:32 --> 00:05:39 and does elimination to clear out the column below that pivot. 91 00:05:39 --> 00:05:43 So actually you see the multiplier is one. 92 00:05:43 --> 00:05:49 It subtracts row two from row three and produces a row of 93 00:05:49 --> 00:05:50 zeros. 94 00:05:50 --> 00:05:50 OK. 95 00:05:50 --> 00:05:54 That I would call that matrix U, right? 96 00:05:54 --> 00:05:59 That's our upper -- well, I can't quite say upper 97 00:05:59 --> 00:06:02 triangular. 98 00:06:02 --> 00:06:08 Maybe upper -- I don't know -- upper something. 99 00:06:08 --> 00:06:12 It's in this so-called echelon form. 100 00:06:12 --> 00:06:17 The word echelon means, like, staircase form. 101 00:06:17 --> 00:06:25 It's the, the non-zeros come in that staircase form. 102 00:06:25 --> 00:06:30 If there was another pivot here, then the staircase would 103 00:06:30 --> 00:06:31 include that. 104 00:06:31 --> 00:06:35 But here's a case where we have two pivots only. 105 00:06:35 --> 00:06:40 OK, so actually we've already discovered the most important 106 00:06:40 --> 00:06:42.12 number about this matrix. 107 00:06:42.12 --> 00:06:44 The number of pivots is two. 108 00:06:44 --> 00:06:49 That number we will call the rank of the matrix. 109 00:06:49 --> 00:06:52 So let me put immediately. 110 00:06:52 --> 00:07:00 The rank of A -- so I'm telling you what this word rank means in 111 00:07:00 --> 00:07:02 the algorithm. 112 00:07:02 --> 00:07:06 It's equal to the number of pivots. 113 00:07:06 --> 00:07:09 And in this case, two. 114 00:07:09 --> 00:07:16 OK, for me that number two is the crucial number. 115 00:07:16 --> 00:07:21 OK, now I go to -- you remember I'm always solving A x equals 116 00:07:21 --> 00:07:25 zero, but now I can solve U x equals zero, right? 117 00:07:25 --> 00:07:28 Same solution, same null space. 118 00:07:28 --> 00:07:28 OK. 119 00:07:28 --> 00:07:33 So I could stop here -- why don't I stop here and do the 120 00:07:33 --> 00:07:36.04 back substitution. 121 00:07:36.04 --> 00:07:39 So now I have to ask you, how do I describe the 122 00:07:39 --> 00:07:40 solutions? 123 00:07:40 --> 00:07:44 There are solutions, right, to A x equals zero. 124 00:07:44 --> 00:07:45 I knew there would be. 125 00:07:45 --> 00:07:48 I had three equations in four unknowns. 126 00:07:48 --> 00:07:51 I certainly expected some solutions. 127 00:07:51 --> 00:07:53 Now I want to see what are they. 128 00:07:53 --> 00:07:57 OK, here's the critical step. 129 00:07:57 --> 00:08:01 I refer to it up here as separating out the pivot 130 00:08:01 --> 00:08:06 variables, the pivot columns, which are these two. 131 00:08:06 --> 00:08:09 Here I have two pivot columns. 132 00:08:09 --> 00:08:14 Those, obviously, they're the columns with the 133 00:08:14 --> 00:08:15 pivots. 134 00:08:15 --> 00:08:17 So I have two pivot columns. 135 00:08:17 --> 00:08:23 And I have the other columns, I'll call free. 136 00:08:23 --> 00:08:26 These are free columns, OK. 137 00:08:26 --> 00:08:29 Why do I use those words? 138 00:08:29 --> 00:08:33 Why do I use that word free? 139 00:08:33 --> 00:08:40 Because now I want to write, I want to find the solutions to 140 00:08:40 --> 00:08:42 U x equals zero. 141 00:08:42 --> 00:08:44 Here is the way I do it. 142 00:08:44 --> 00:08:53 These free columns I can assign any number freely to those -- 143 00:08:53 --> 00:09:00 to the variables x2 and x4, the ones that multiply columns 144 00:09:00 --> 00:09:02 two and four. 145 00:09:02 --> 00:09:07 So I can assign anything I like to x2 and x4. 146 00:09:07 --> 00:09:13 And then I can solve the equations for x1 and x3. 147 00:09:13 --> 00:09:16 Let me say that again. 148 00:09:16 --> 00:09:21 If I give -- let me, let me assign. 149 00:09:21 --> 00:09:27 So, so one particular x is to assign, say, the value one to 150 00:09:27 --> 00:09:30.05 the, to x2, and zero to x4. 151 00:09:30.05 --> 00:09:35 Those are -- that was a free choice, but it's a convenient 152 00:09:35 --> 00:09:36 choice. 153 00:09:36 --> 00:09:36 OK. 154 00:09:36 --> 00:09:42 Now I want to solve U x equals zero and find numbers one and 155 00:09:42 --> 00:09:46.45 three, complete the solution. 156 00:09:46.45 --> 00:09:46 OK. 157 00:09:46 --> 00:09:49 Can I write down -- let's see. 158 00:09:49 --> 00:09:55 Shall we just remember what U x equals zero represents? 159 00:09:55 --> 00:09:57 What are my equations? 160 00:09:57 --> 00:10:03 That first equation is x1 plus just -- I'm just saying what are 161 00:10:03 --> 00:10:05.71 these matrices meaning. 162 00:10:05.71 --> 00:10:08 That's the first equation. 163 00:10:08 --> 00:10:13 And the second equation was 2x3 + 4x4=0. 164 00:10:13 --> 00:10:15 Those are my two equations. 165 00:10:15 --> 00:10:16.24 OK. 166 00:10:16.24 --> 00:10:22 Now the point is I can find x1 and x3 by back substitution. 167 00:10:22 --> 00:10:26 So we're building on what we already know. 168 00:10:26 --> 00:10:31 The new thing is that there were some free variables that I 169 00:10:31 --> 00:10:35 could give any numbers to. 170 00:10:35 --> 00:10:40.1 And I'm systematically going to make a choice like this, 171 00:10:40.1 --> 00:10:40 1 and 0. 172 00:10:40 --> 00:10:42 Now what is x3? 173 00:10:42 --> 00:10:45 Let's, let's go backwards, back up. 174 00:10:45 --> 00:10:49 I look at the last equation. x3 is zero, from the last 175 00:10:49 --> 00:10:53 equation, because, because x4 we've set x4 to 176 00:10:53 --> 00:10:57 zero, and then we get x3 as zero. 177 00:10:57 --> 00:10:57 OK. 178 00:10:57 --> 00:11:00 Now we set x2 to be one, so what is x1? 179 00:11:00 --> 00:11:02 Negative two, right? 180 00:11:02 --> 00:11:07 So then I have negative two plus two, zero and zero, 181 00:11:07 --> 00:11:09 correctly giving zero. 182 00:11:09 --> 00:11:12 There is a vector in the null space. 183 00:11:12 --> 00:11:16 There is a solution to A x equals zero. 184 00:11:16 --> 00:11:19 In fact, what solution is that? 185 00:11:19 --> 00:11:23 That simply says that minus two times the first column plus one 186 00:11:23 --> 00:11:26 times the second column is the zero column. 187 00:11:26 --> 00:11:28 Of course that's right. 188 00:11:28 --> 00:11:32 Minus two of that column plus one of that, or minus two of 189 00:11:32 --> 00:11:34 that plus one of that. 190 00:11:34 --> 00:11:38 That solution is -- that, that's just what we saw 191 00:11:38 --> 00:11:43.81 immediately, that the second column is twice as big as the 192 00:11:43.81 --> 00:11:44 first column. 193 00:11:44 --> 00:11:49 OK, tell me some more vectors in the null space. 194 00:11:49 --> 00:11:50 I found one. 195 00:11:50 --> 00:11:55 Tell me, how to get a bunch more immediately out of that 196 00:11:55 --> 00:11:56 one. 197 00:11:56 --> 00:11:58 Just take multiples of it. 198 00:11:58 --> 00:12:01.8 Any multiple of -- I could multiply this by anything. 199 00:12:01.8 --> 00:12:04 I might as well call it, I could say, 200 00:12:04 --> 00:12:06 C, some multiple of this. 201 00:12:06 --> 00:12:09 So let me -- so X could be any multiple of this one. 202 00:12:09 --> 00:12:13 OK, that, that describes now a line, an infinitely long line in 203 00:12:13 --> 00:12:16.37 four dimensional space. 204 00:12:16.37 --> 00:12:20.14 But -- which is in the null space. 205 00:12:20.14 --> 00:12:23 Is that the whole null space? 206 00:12:23 --> 00:12:23 No. 207 00:12:23 --> 00:12:27 I've got two free variables here. 208 00:12:27 --> 00:12:31.11 I made this choice, one and zero, 209 00:12:31.11 --> 00:12:37 for the free variables, but I could have made another 210 00:12:37 --> 00:12:39 choice. 211 00:12:39 --> 00:12:42 Let me make the other choice zero and one. 212 00:12:42 --> 00:12:43 You see my system. 213 00:12:43 --> 00:12:45 So let me repeat the system. 214 00:12:45 --> 00:12:49 This is the algorithm that you, you just learned to do. 215 00:12:49 --> 00:12:50 Do elimination. 216 00:12:50 --> 00:12:54 Decide which are the pivot columns and which are the free 217 00:12:54 --> 00:12:55 columns. 218 00:12:55 --> 00:12:58.72 That tells you that, that variables one and three 219 00:12:58.72 --> 00:13:01.72 are pivot variables, two and four are free 220 00:13:01.72 --> 00:13:03 variables. 221 00:13:03 --> 00:13:09 Then those free variables, you assign them -- you give one 222 00:13:09 --> 00:13:16 of them the value one and the others the value zero -- in this 223 00:13:16 --> 00:13:22 case, we had a one and a zero -- and complete the solution. 224 00:13:22 --> 00:13:26 And you do -- you give the other one the 225 00:13:26 --> 00:13:27 value one and zero. 226 00:13:27 --> 00:13:29 And now complete the solution. 227 00:13:29 --> 00:13:31 So let's complete that solution. 228 00:13:31 --> 00:13:35 I'm looking for a vector in the null space, and it's absolutely 229 00:13:35 --> 00:13:38 going to be different from this guy, because, 230 00:13:38 --> 00:13:42 because any multiple of that zero is never going to give the 231 00:13:42 --> 00:13:43 one. 232 00:13:43 --> 00:13:48 So I have somebody new in the null space, and let me finish it 233 00:13:48 --> 00:13:48 out. 234 00:13:48 --> 00:13:50 What's x3 here? 235 00:13:50 --> 00:13:53 So we're going by back substitution, 236 00:13:53 --> 00:13:55 looking at this equation. 237 00:13:55 --> 00:13:59 Now x4 we've changed, we're doing the other 238 00:13:59 --> 00:14:03 possibility, where x2 is zero and x4 is one. 239 00:14:03 --> 00:14:07 So x3 will happen to be minus two. 240 00:14:07 --> 00:14:12 And now what do I get for that first equation? 241 00:14:12 --> 00:14:13 x1 -- let's see. 242 00:14:13 --> 00:14:19 Two x3s is minus four plus two -- do I get a two there? 243 00:14:19 --> 00:14:20 Perhaps, yeah. 244 00:14:20 --> 00:14:23 Two for x1, minus four, and two. 245 00:14:23 --> 00:14:24 OK. 246 00:14:24 --> 00:14:26 That's in the null space. 247 00:14:26 --> 00:14:31 What does that say about the columns? 248 00:14:31 --> 00:14:36 That says that two of this column minus two of this column 249 00:14:36 --> 00:14:39 plus this column gives zero, which it does. 250 00:14:39 --> 00:14:44.44 Two of that minus two of that and one of that gives the zero 251 00:14:44.44 --> 00:14:45 column. 252 00:14:45 --> 00:14:49 OK, now, now I've found another vector in the null space. 253 00:14:49 --> 00:14:53 Now we're ready to tell me the whole null space. 254 00:14:53 --> 00:14:57 What are all the solutions to Ax=0? 255 00:14:57 --> 00:15:02 I've got this guy and when I have him, what else is, 256 00:15:02 --> 00:15:07 goes into the null space along with that? 257 00:15:07 --> 00:15:11 These are my two special solutions. 258 00:15:11 --> 00:15:16 I call them special -- I just invented that name. 259 00:15:16 --> 00:15:19 Special solutions. 260 00:15:19 --> 00:15:24 What's special about them is the special numbers I gave to 261 00:15:24 --> 00:15:28 the free variables, the values zero and one for the 262 00:15:28 --> 00:15:29.55 free variables. 263 00:15:29.55 --> 00:15:34 I could have given the free variables any values and got 264 00:15:34 --> 00:15:36 vectors in the null space. 265 00:15:36 --> 00:15:41 But this was a good way to be sure I got t- got everybody. 266 00:15:41 --> 00:15:45 OK, so once I have him, I also have any multiple, 267 00:15:45 --> 00:15:46 right? 268 00:15:46 --> 00:15:51 So I could take any multiple of that and it's in the null space. 269 00:15:51 --> 00:15:55 And now what else -- I left a little space for what? 270 00:15:55 --> 00:15:56 What -- a plus sign. 271 00:15:56 --> 00:15:59 I can take any combination. 272 00:15:59 --> 00:16:02 Here is a line of vectors in the null space. 273 00:16:02 --> 00:16:05 A bunch of solutions. 274 00:16:05 --> 00:16:11.76 Would you rather I say in the null space or would you rather I 275 00:16:11.76 --> 00:16:14.53 say, OK, I'm solving Ax=0? 276 00:16:14.53 --> 00:16:17 Well, really I'm solving Ux=0. 277 00:16:17 --> 00:16:22 Well, OK, let me put in that crucial plus sign. 278 00:16:22 --> 00:16:28 I'm taking all the combinations of my two special solutions. 279 00:16:28 --> 00:16:32 That's my conclusion there. 280 00:16:32 --> 00:16:36 The null space contains, contains exactly all the 281 00:16:36 --> 00:16:39 combinations of the special solutions. 282 00:16:39 --> 00:16:42 And how many special solutions are there? 283 00:16:42 --> 00:16:44 There's one for every free variable. 284 00:16:44 --> 00:16:47.54 And how many free variables are there? 285 00:16:47.54 --> 00:16:52 Oh, I mean, we can see all the whole picture now. 286 00:16:52 --> 00:16:55 If the rank R was two, this is the, 287 00:16:55 --> 00:17:00 this is the number of pivot variables, right, 288 00:17:00 --> 00:17:03 because it counted the pivots. 289 00:17:03 --> 00:17:06 So how many free variables? 290 00:17:06 --> 00:17:10 Well, you know it's two, right? 291 00:17:10 --> 00:17:14 What is it in -- for a matrix that's m rows, 292 00:17:14 --> 00:17:21 n columns, n variables that means, with rank r? 293 00:17:21 --> 00:17:27 How many free variables have we got left? 294 00:17:27 --> 00:17:36 If r of the variables are pivot variables, we have n-r -- in 295 00:17:36 --> 00:17:42 this case four minus two -- free variables. 296 00:17:42 --> 00:17:52 Do you see that first of all we get clean answers here? 297 00:17:52 --> 00:17:56 We get r pivot variables -- so there really were r equations 298 00:17:56 --> 00:17:57 here. 299 00:17:57 --> 00:18:01.09 There looked like three equations, but there were really 300 00:18:01.09 --> 00:18:03 only two independent equations. 301 00:18:03 --> 00:18:07 And there were n-r variables that we could choose freely, 302 00:18:07 --> 00:18:10 and we gave them those special zero one values, 303 00:18:10 --> 00:18:13 and we got the special solutions. 304 00:18:13 --> 00:18:14 OK. 305 00:18:14 --> 00:18:20 For me -- we could stop at that point. 306 00:18:20 --> 00:18:28 That gives you a complete algorithm for finding all the 307 00:18:28 --> 00:18:33 solutions to A x equals zero. 308 00:18:33 --> 00:18:33 OK. 309 00:18:33 --> 00:18:41 Again, you do elimination -- going onward when a column, 310 00:18:41 --> 00:18:46 when there's nothing to be done on one column, 311 00:18:46 --> 00:18:48.03 you just continue. 312 00:18:48.03 --> 00:18:52 There's this number r, the number of pivots, 313 00:18:52 --> 00:18:56 is crucial, and it leaves n-r free variables, 314 00:18:56 --> 00:19:00 which you give values zero and one to. 315 00:19:00 --> 00:19:04 I would like to take one more step. 316 00:19:04 --> 00:19:09 I would like to clean up this matrix even more. 317 00:19:09 --> 00:19:14 So now I'm going to go to -- this is in its, 318 00:19:14 --> 00:19:20 this is in echelon form, upper triangular if you like. 319 00:19:20 --> 00:19:26 I want to go one more step to make it as good as it can be. 320 00:19:26 --> 00:19:32 OK, so now I'm going to speak about the reduced row echelon 321 00:19:32 --> 00:19:34.3 form. 322 00:19:34.3 --> 00:19:41 OK, so now I'm going to speak about the matrix R, 323 00:19:41 --> 00:19:46 which is the reduced row echelon form. 324 00:19:46 --> 00:19:50 So what does that mean? 325 00:19:50 --> 00:19:57.83 That means I just -- I can, I can work harder on U. 326 00:19:57.83 --> 00:20:04 So let me start, let me suppose I got as far as 327 00:20:04 --> 00:20:08 U, which was good. 328 00:20:08 --> 00:20:12 Notice how that row of zeros appeared. 329 00:20:12 --> 00:20:16 I didn't comment on that, but I should have. 330 00:20:16 --> 00:20:21.08 That row of zeros up here is because row three was a 331 00:20:21.08 --> 00:20:26 combination of rows one and two, and elimination discovered that 332 00:20:26 --> 00:20:27 fact. 333 00:20:27 --> 00:20:33 When we get a row of zeros, that's telling us that the -- 334 00:20:33 --> 00:20:40 original row that was there was a combination of other rows, 335 00:20:40 --> 00:20:44 and elimination knocked it out. 336 00:20:44 --> 00:20:46 OK, so we got this far. 337 00:20:46 --> 00:20:51.1 Now how can I clean that up further? 338 00:20:51.1 --> 00:20:54.63 I can do, elimination upwards. 339 00:20:54.63 --> 00:20:58 I can get zero above the pivots. 340 00:20:58 --> 00:21:04 So this reduced row echelon form has zeros above and below 341 00:21:04 --> 00:21:06.28 the pivots. 342 00:21:06.28 --> 00:21:09 So let me do that. 343 00:21:09 --> 00:21:14 So now I'll subtract one of this from the row above. 344 00:21:14 --> 00:21:19 That will leave a zero and a minus two in there. 345 00:21:19 --> 00:21:21 And that's good. 346 00:21:21 --> 00:21:25 OK, and I can clean it up even one more step. 347 00:21:25 --> 00:21:31 I can make the pivots -- the pivots I'm going to make equal 348 00:21:31 --> 00:21:37 to one, because I can divide equation two by the pivot. 349 00:21:37 --> 00:21:41 That won't change the solutions. 350 00:21:41 --> 00:21:43 So let me do that. 351 00:21:43 --> 00:21:47 And then I really -- I'm ready to stop. 352 00:21:47 --> 00:21:51 One, two, zero, minus two, zero, 353 00:21:51 --> 00:21:52 zero, one, two. 354 00:21:52 --> 00:21:59 I divided the second equation by two, because now I have a one 355 00:21:59 --> 00:22:04 in the pivot and zeros below. 356 00:22:04 --> 00:22:04 OK. 357 00:22:04 --> 00:22:07 This is my matrix R. 358 00:22:07 --> 00:22:13 I guess I'm hoping that you could now execute the whole 359 00:22:13 --> 00:22:15 algorithm. 360 00:22:15 --> 00:22:22 Matlab will do it immediately with the command -- reduced row 361 00:22:22 --> 00:22:24 echelon form of A. 362 00:22:24 --> 00:22:31 So if I input that original matrix A and then I write, 363 00:22:31 --> 00:22:35 then I type that command, press return, 364 00:22:35 --> 00:22:40 that matrix will appear. 365 00:22:40 --> 00:22:47 That's the reduced row echelon form, and it's got all the 366 00:22:47 --> 00:22:51 information as clear as can be. 367 00:22:51 --> 00:22:55 What, what information has it got? 368 00:22:55 --> 00:23:02 Well, of course it immediately tells me the pivot rows, 369 00:23:02 --> 00:23:07 pivot rows, one and two, pivot columns, 370 00:23:07 --> 00:23:10 one and three. 371 00:23:10 --> 00:23:15.21 And in fact it's got the identity matrix in there, 372 00:23:15.21 --> 00:23:15 right? 373 00:23:15 --> 00:23:20 It's, it's got zeros above and below the pivots, 374 00:23:20 --> 00:23:25 and the pivots are one, so it's, so it's got a -- so 375 00:23:25 --> 00:23:31 notice the two by two identity matrix that's sitting in the 376 00:23:31 --> 00:23:36 pivot rows and pivot columns. it's I in the pivot rows and 377 00:23:36 --> 00:23:38 columns. 378 00:23:38 --> 00:23:42 It's got zero rows below. 379 00:23:42 --> 00:23:49 Those are always indicating that original rows were, 380 00:23:49 --> 00:23:53 were combinations of other rows. 381 00:23:53 --> 00:23:58 So we really only had two rows there. 382 00:23:58 --> 00:24:04 And now it also -- so there's the identity. 383 00:24:04 --> 00:24:10 Now it's also got its free columns. 384 00:24:10 --> 00:24:15 And, they're cleaned up as much as possible. 385 00:24:15 --> 00:24:20 Actually, actually it's now so cleaned up that the special 386 00:24:20 --> 00:24:26 solutions, I can read off -- you remember I went through the 387 00:24:26 --> 00:24:31 steps of computing this -- doing back substitution? 388 00:24:31 --> 00:24:35 Let me, let me, instead of that system, 389 00:24:35 --> 00:24:38.09 let me take this improved system. 390 00:24:38.09 --> 00:24:41.71 So I'm going to use these numbers, right. 391 00:24:41.71 --> 00:24:44 In these equations, what did I do? 392 00:24:44 --> 00:24:49 I divided this equation by two and, oh yeah and I had 393 00:24:49 --> 00:24:54.2 subtracted two of this, which knocked out this guy and 394 00:24:54.2 --> 00:24:57 made that a minus sign. 395 00:24:57 --> 00:25:03 Is that what -- I've now written Rx equals zero. 396 00:25:03 --> 00:25:08 Now I guess I'm hoping everybody in this room 397 00:25:08 --> 00:25:15 understands the solutions to the original A x equals zero, 398 00:25:15 --> 00:25:19 the midway, halfway, U x equals zero, 399 00:25:19 --> 00:25:26 and the final R x equals zero are all the same. 400 00:25:26 --> 00:25:32 Because going from one of those to another one I didn't mess up. 401 00:25:32 --> 00:25:38 I just multiplied equations and subtracted from other equations, 402 00:25:38 --> 00:25:40 which I'm allowed to do. 403 00:25:40 --> 00:25:40 OK. 404 00:25:40 --> 00:25:46 But my point is that now if I do this free variables and back 405 00:25:46 --> 00:25:49 substitution, it's just, the numbers are 406 00:25:49 --> 00:25:51 there. 407 00:25:51 --> 00:25:57 When I let x -- so in this guy, I let x2 be one and x4 be zero. 408 00:25:57 --> 00:26:00 I, I guess, what I seeing here? 409 00:26:00 --> 00:26:05 Let me, let me sort of separate this out here. 410 00:26:05 --> 00:26:09 I'm seeing in the pivot, in the pivot columns, 411 00:26:09 --> 00:26:13 if I, if I put the pivot columns here, 412 00:26:13 --> 00:26:15 I'm seeing those. 413 00:26:15 --> 00:26:21 And I'm -- in the free columns I'm seeing -- what I seeing in 414 00:26:21 --> 00:26:23 the free columns? 415 00:26:23 --> 00:26:28 A two, zero in that first free column, the x2 column, 416 00:26:28 --> 00:26:31 and a minus two, two in the fourth column, 417 00:26:31 --> 00:26:33 the other free column. 418 00:26:33 --> 00:26:39 And the row of zeros below, which of course have no -- that 419 00:26:39 --> 00:26:43 equation is zero equals zero. 420 00:26:43 --> 00:26:44 That's satisfied. 421 00:26:44 --> 00:26:46 Here's my point. 422 00:26:46 --> 00:26:48 That when I do back substitution, 423 00:26:48 --> 00:26:52 these numbers are exactly what shows up -- oh, 424 00:26:52 --> 00:26:55 their signs get switched. 425 00:26:55 --> 00:26:58 I was going to say those numbers, two, 426 00:26:58 --> 00:27:02 minus two, zero, two, can I just circle the -- 427 00:27:02 --> 00:27:05 this is the free part of the matrix. 428 00:27:05 --> 00:27:08 This is the identity part. 429 00:27:08 --> 00:27:12 This is the free part, maybe I'll call it F. 430 00:27:12 --> 00:27:15 This, of course, I call I, because it's the 431 00:27:15 --> 00:27:16 identity. 432 00:27:16 --> 00:27:19 The free part is a, I mean, I'm just doing back 433 00:27:19 --> 00:27:20 substitution. 434 00:27:20 --> 00:27:24 And those free numbers will show up, with a minus sign, 435 00:27:24 --> 00:27:29 because they pop onto the other side of the equation -- 436 00:27:29 --> 00:27:34 so I see minus two, zero, and I see two, 437 00:27:34 --> 00:27:35 minus two. 438 00:27:35 --> 00:27:38 So that wasn't magic. 439 00:27:38 --> 00:27:40 It had to happen. 440 00:27:40 --> 00:27:44.87 Let me, show you clearly why it happened. 441 00:27:44.87 --> 00:27:52 OK, so that's -- this is what I'm interested in here. 442 00:27:52 --> 00:27:55 And now let me, let me just, 443 00:27:55 --> 00:27:59 like, do it, do it for -- let's suppose 444 00:27:59 --> 00:28:06.22 we've, we've got to -- let's suppose we've got this system 445 00:28:06.22 --> 00:28:09 already in, in rref form. 446 00:28:09 --> 00:28:14 So my matrix R is -- what does it look like? 447 00:28:14 --> 00:28:20 OK, and I'll -- let me pretend that the pivot columns come 448 00:28:20 --> 00:28:27 first and then whatever's in the free columns. 449 00:28:27 --> 00:28:34 And there might be some zero rows below. 450 00:28:34 --> 00:28:45 There's a typical -- a pretty typical reduced row echelon 451 00:28:45 --> 00:28:46 form. 452 00:28:46 --> 00:28:50 You see what's typical. 453 00:28:50 --> 00:28:55 It's got -- this is r by r. 454 00:28:55 --> 00:29:01 This is r pivot rows. 455 00:29:01 --> 00:29:04 This is r pivot columns. 456 00:29:04 --> 00:29:08 And here are n-r free columns. 457 00:29:08 --> 00:29:09 OK. 458 00:29:09 --> 00:29:14 Tell me what are the special solutions? 459 00:29:14 --> 00:29:18 What are the -- what's x? 460 00:29:18 --> 00:29:25 If I want to solve R x equals zero -- in fact, 461 00:29:25 --> 00:29:33 let me -- I'm really going to, do the whole -- 462 00:29:33 --> 00:29:37 since these -- this is now block matrices, 463 00:29:37 --> 00:29:42 I might as well do all of the special solutions at once. 464 00:29:42 --> 00:29:48 So I want to solve R x equals zero, and I'll have some special 465 00:29:48 --> 00:29:49 solutions. 466 00:29:49 --> 00:29:54 Let me, actually -- can I do them all at once? 467 00:29:54 --> 00:29:58 I'm going to create a null space matrix, 468 00:29:58 --> 00:29:59 OK. 469 00:29:59 --> 00:30:00 A matrix. 470 00:30:00 --> 00:30:07 Its, its, its columns are the special -- the columns are the 471 00:30:07 --> 00:30:09 special solutions. 472 00:30:09 --> 00:30:15 This is, I'm making it sound harder, it's going to be totally 473 00:30:15 --> 00:30:16 easy. 474 00:30:16 --> 00:30:19 N will be this null space matrix. 475 00:30:19 --> 00:30:23 I want R N to be the zero matrix. 476 00:30:23 --> 00:30:30 These columns of N are supposed to multipl- to get multiplied by 477 00:30:30 --> 00:30:34 R and give zero columns. 478 00:30:34 --> 00:30:36 So what N will do the job? 479 00:30:36 --> 00:30:42 Let me put -- I'm going to put the identity in the free 480 00:30:42 --> 00:30:47 variable part and then minus F will show up in the pivot 481 00:30:47 --> 00:30:52 variables, just the way it did in that example. 482 00:30:52 --> 00:30:55 There we had the identity and F. 483 00:30:55 --> 00:30:59 Here -- in the special solution. 484 00:30:59 --> 00:31:05 So these columns are -- there's the matrix of special solutions. 485 00:31:05 --> 00:31:09 And actually, there -- so there's a Matlab 486 00:31:09 --> 00:31:13 command or a teaching code command, NULL -- N equal, 487 00:31:13 --> 00:31:18 so this is the -- produces the null basis, the null space 488 00:31:18 --> 00:31:22 matrix, NULL of A, and there it is. 489 00:31:22 --> 00:31:27 And how does that command actually work? 490 00:31:27 --> 00:31:33.88 It uses Matlab to compute R, then it picks out the pivot 491 00:31:33.88 --> 00:31:40 variables, the free variables, puts, puts ones and zeros in 492 00:31:40 --> 00:31:46.15 for the free variables, and copies out the pivot 493 00:31:46.15 --> 00:31:48 variables. 494 00:31:48 --> 00:31:55 It, it does back substitution, but back substitution for this 495 00:31:55 --> 00:31:58 system is totally simple. 496 00:31:58 --> 00:32:01 What is this system? 497 00:32:01 --> 00:32:03 R x equals zero. 498 00:32:03 --> 00:32:09 So this is R is I F, and x is the pivot variables 499 00:32:09 --> 00:32:16 and the free variables, and it's supposed to give zero. 500 00:32:16 --> 00:32:19 So what does that mean? 501 00:32:19 --> 00:32:24 That means that the pivot variables plus F times the free 502 00:32:24 --> 00:32:26 variables give zero. 503 00:32:26 --> 00:32:32.56 So let me put F times the free variables on the other side. 504 00:32:32.56 --> 00:32:36 I get minus F times the free variables. 505 00:32:36 --> 00:32:41 There's my, equation, as simple as it can be. 506 00:32:41 --> 00:32:47 That's what back substitution comes to when I've reduced and 507 00:32:47 --> 00:32:52 reduced and reduced this system to the, to the best form, 508 00:32:52 --> 00:32:52 OK. 509 00:32:52 --> 00:32:57 And, then if the free variables, I make this special 510 00:32:57 --> 00:33:02 choice of the identity, then the pivot variables are 511 00:33:02 --> 00:33:02 minus F. 512 00:33:02 --> 00:33:05 OK, can I do, another example? 513 00:33:05 --> 00:33:09 Could you do another example? 514 00:33:09 --> 00:33:13 Can I -- let me just take another matrix and, 515 00:33:13 --> 00:33:18 and let's go through this algorithm once more, 516 00:33:18 --> 00:33:18 OK. 517 00:33:18 --> 00:33:19 Here we go. 518 00:33:19 --> 00:33:24 Here's a blackboard for another matrix, OK. 519 00:33:24 --> 00:33:29 So I'll call the matrix A again, but let me make it -- 520 00:33:29 --> 00:33:34 yeah, how big shall we make it this time? 521 00:33:34 --> 00:33:37 Why don't I do this? 522 00:33:37 --> 00:33:40 Just for the heck of it. 523 00:33:40 --> 00:33:49 Let me take the transpose of this A and see what happens to 524 00:33:49 --> 00:33:49 that. 525 00:33:49 --> 00:33:55 Two four six eight and three six eight ten. 526 00:33:55 --> 00:34:03 Before we do the calculations, tell me what's coming? 527 00:34:03 --> 00:34:10 How many pivot variables do you expect here? 528 00:34:10 --> 00:34:14 How many columns are going to have pivots? 529 00:34:14 --> 00:34:18 How many -- we have three columns in that matrix, 530 00:34:18 --> 00:34:22 but are we going to, are we going to have three 531 00:34:22 --> 00:34:23 pivots? 532 00:34:23 --> 00:34:28 No, because this third columns is the sum of the first two 533 00:34:28 --> 00:34:28 columns. 534 00:34:28 --> 00:34:33 I'm totally expecting, totally expecting that these 535 00:34:33 --> 00:34:38 will be pivot columns -- because they're independent, 536 00:34:38 --> 00:34:42.57 but that this third guy, the third column, 537 00:34:42.57 --> 00:34:47 which is dependent on the first two, is going to be a free 538 00:34:47 --> 00:34:48 column. 539 00:34:48 --> 00:34:51 Elimination better discover that. 540 00:34:51 --> 00:34:55 And elimination will also straighten out the rows, 541 00:34:55 --> 00:35:00 dependent rows and some independent rows. 542 00:35:00 --> 00:35:05 What's the, what's the row reduced echelon form for this? 543 00:35:05 --> 00:35:07 Let's just do it, OK. 544 00:35:07 --> 00:35:10 So, so that's the first pivot. 545 00:35:10 --> 00:35:15 Two times that away from that gives me a row of zeros. 546 00:35:15 --> 00:35:21 Two times that away from that gives me a zero two two. 547 00:35:21 --> 00:35:27 And two times that away from that gives me a zero four four. 548 00:35:27 --> 00:35:30 OK, first column is straight. 549 00:35:30 --> 00:35:34 First variable is a pivot variable. 550 00:35:34 --> 00:35:35 No problem. 551 00:35:35 --> 00:35:37 On to the second column. 552 00:35:37 --> 00:35:41 I look at the second pivot, it's a zero. 553 00:35:41 --> 00:35:44 I look below it. 554 00:35:44 --> 00:35:45 There's a two. 555 00:35:45 --> 00:35:48 OK, I do a row exchange. 556 00:35:48 --> 00:35:50 So this zero is now there. 557 00:35:50 --> 00:35:55 I now have a perfectly good pivot, and I use it. 558 00:35:55 --> 00:36:00 OK, and I subtract two of that row away from this row. 559 00:36:00 --> 00:36:04 All right if I do it like that? 560 00:36:04 --> 00:36:06 I've got to the form U now. 561 00:36:06 --> 00:36:09 This was my A. 562 00:36:09 --> 00:36:10 Now there's my U. 563 00:36:10 --> 00:36:13 I can see now -- I have to stop, right? 564 00:36:13 --> 00:36:16 I would go on to the third column. 565 00:36:16 --> 00:36:18 I should have tried. 566 00:36:18 --> 00:36:20 I quit, but without trying. 567 00:36:20 --> 00:36:22 I shouldn't have done that. 568 00:36:22 --> 00:36:26.79 On to the third column, look at the pivot position. 569 00:36:26.79 --> 00:36:29 It's got a zero in it. 570 00:36:29 --> 00:36:31 Look below, all zeros. 571 00:36:31 --> 00:36:34 Now I'm entitled to stop, OK. 572 00:36:34 --> 00:36:36 So the rank is two again. 573 00:36:36 --> 00:36:39 What about the null space? 574 00:36:39 --> 00:36:44.16 How many special solutions are there this time? 575 00:36:44.16 --> 00:36:48 How many special solutions for this matrix? 576 00:36:48 --> 00:36:54 I've got -- and which are the free variables and which are the 577 00:36:54 --> 00:36:58 pivot variables and so on? 578 00:36:58 --> 00:37:02 Pivot columns, I've got two pivot columns, 579 00:37:02 --> 00:37:04 and that's no accident. 580 00:37:04 --> 00:37:10 The number of pivot columns for a matrix A, that's a great fact, 581 00:37:10 --> 00:37:16 that the number of pivot columns for A and A transpose 582 00:37:16 --> 00:37:18 are the same. 583 00:37:18 --> 00:37:22 And then I have a free column. 584 00:37:22 --> 00:37:24 There's a free column. 585 00:37:24 --> 00:37:30 One free column, because the count is three 586 00:37:30 --> 00:37:31 minus two. 587 00:37:31 --> 00:37:36 Three minus two gives me one free column. 588 00:37:36 --> 00:37:42 OK, so now let me solve, what's in the null space. 589 00:37:42 --> 00:37:46 OK, so how do I -- let's see. 590 00:37:46 --> 00:37:52 These vectors have length three. 591 00:37:52 --> 00:37:55 They only have three components. 592 00:37:55 --> 00:38:00 I'm making too much space for the, to write x. 593 00:38:00 --> 00:38:05 x has just got three components, and what are they? 594 00:38:05 --> 00:38:08 I'm looking for the null space. 595 00:38:08 --> 00:38:10 OK, so how do I start? 596 00:38:10 --> 00:38:16 I give the free variable some convenient value. 597 00:38:16 --> 00:38:18.25 And what's that? 598 00:38:18.25 --> 00:38:19.82 I set it to one. 599 00:38:19.82 --> 00:38:22 I set the free variable to one. 600 00:38:22 --> 00:38:28 If I set the free variable to zero and solve for the pivot 601 00:38:28 --> 00:38:32 variables, I'll get all zeros: no progress. 602 00:38:32 --> 00:38:37 But by setting the free variable to one -- you see w- my 603 00:38:37 --> 00:38:44 two equations now are -- my equations are x1+ 2x2+ 3 x3 604 00:38:44 --> 00:38:48 is zero, that's my first equation. 605 00:38:48 --> 00:38:54 And my second equation is now 2x2+2x3 equals zero. 606 00:38:54 --> 00:38:55 And, OK. 607 00:38:55 --> 00:39:00.31 So if x3 is one, then x2 is minus one. 608 00:39:00.31 --> 00:39:06 And if x3 is one and x2 is minus one, then maybe x1 is 609 00:39:06 --> 00:39:07 minus one. 610 00:39:07 --> 00:39:13.74 And actually I go back to check now. 611 00:39:13.74 --> 00:39:17 I don't, like -- I did a quick calculation mentally. 612 00:39:17 --> 00:39:20 Can I mentally do a quick check? 613 00:39:20 --> 00:39:23 That matrix, that solution x says that minus 614 00:39:23 --> 00:39:27 this column minus this column plus this one is the zero 615 00:39:27 --> 00:39:28 column. 616 00:39:28 --> 00:39:29 And it is. 617 00:39:29 --> 00:39:33 Minus that minus that plus that is zero. 618 00:39:33 --> 00:39:35 So that's in the null space. 619 00:39:35 --> 00:39:39 And now you can tell me what else is in the null space. 620 00:39:39 --> 00:39:42 What's, what's the whole null space now? 621 00:39:42 --> 00:39:44 I multiply by C, right. 622 00:39:44 --> 00:39:46 The whole null space is a line. 623 00:39:46 --> 00:39:48 So that's the description. 624 00:39:48 --> 00:39:53 You know, if I ask you on a homework or a quiz or the final 625 00:39:53 --> 00:39:56 what -- give me, describe, 626 00:39:56 --> 00:40:00 tell me the null space, find the null space of this 627 00:40:00 --> 00:40:02 matrix, you can take those steps. 628 00:40:02 --> 00:40:05 And that's the answer I'm looking for. 629 00:40:05 --> 00:40:10 And I'm looking for that C too, because that's telling me that 630 00:40:10 --> 00:40:15.42 you're remembering that it's a whole space and not just one 631 00:40:15.42 --> 00:40:16 vector. 632 00:40:16 --> 00:40:21 Later I will ask you for a basis for the null space. 633 00:40:21 --> 00:40:23 Then I just want this vector. 634 00:40:23 --> 00:40:28 But if I ask for the whole null space, then there's the whole 635 00:40:28 --> 00:40:30 line through that vector. 636 00:40:30 --> 00:40:34 OK, now one more natural thing to do with this example, 637 00:40:34 --> 00:40:38 right, is keep going to the reduced matrix, 638 00:40:38 --> 00:40:39 R. 639 00:40:39 --> 00:40:42 So can I push onwards to R? 640 00:40:42 --> 00:40:46 That should be quick, but let's just practice. 641 00:40:46 --> 00:40:48.71 Let me keep going to R. 642 00:40:48.71 --> 00:40:51 OK, so what do I do here? 643 00:40:51 --> 00:40:57.01 I subtract -- I clear out above the pivot, so I subtract that 644 00:40:57.01 --> 00:41:02 from that, that's one zero one is left, right? 645 00:41:02 --> 00:41:07 When I subtracted this row from this it produced a zero above 646 00:41:07 --> 00:41:08 this pivot. 647 00:41:08 --> 00:41:12 And now I want that pivot to be a one. 648 00:41:12 --> 00:41:16 So for the R matrix, I'll divide this equation by 649 00:41:16 --> 00:41:21 two, and of course these zero, zeros are great, 650 00:41:21 --> 00:41:22 they don't change. 651 00:41:22 --> 00:41:23 There's R. 652 00:41:23 --> 00:41:25 That's R. 653 00:41:25 --> 00:41:27 You see what R is? 654 00:41:27 --> 00:41:32 You see the identity matrix sitting up here? 655 00:41:32 --> 00:41:37 You see the free part F, the F part here? 656 00:41:37 --> 00:41:40 And you see the zeros below. 657 00:41:40 --> 00:41:43 This is I F zero zero. 658 00:41:43 --> 00:41:45 And what's the x? 659 00:41:45 --> 00:41:52 The x has the identity -- well, it's only a single number 660 00:41:52 --> 00:41:55 one, but it's the identity matrix in the free, 661 00:41:55 --> 00:41:57 in the free part. 662 00:41:57 --> 00:42:01 And what does it have in the pivot variables? 663 00:42:01 --> 00:42:03.72 What did back substitution give? 664 00:42:03.72 --> 00:42:05 It gave minus these guys. 665 00:42:05 --> 00:42:10 You see that what this is is any multiple of -- this is the 666 00:42:10 --> 00:42:15 identity there, and this is minus F here. 667 00:42:15 --> 00:42:18 This is our null space matrix N for this. 668 00:42:18 --> 00:42:24 Our, our null space matrix is the guy whose columns are the 669 00:42:24 --> 00:42:25 special solutions. 670 00:42:25 --> 00:42:31 So their free variables have the special values one and, 671 00:42:31 --> 00:42:33 pivot variables have minus F. 672 00:42:33 --> 00:42:37 So do you see, though, how the minus F just 673 00:42:37 --> 00:42:43 automatically shows up in the special solutions. 674 00:42:43 --> 00:42:45 That's it really. 675 00:42:45 --> 00:42:51 I don't think there's anything more I can say about A x equals 676 00:42:51 --> 00:42:52 zero. 677 00:42:52 --> 00:42:58 There's more I can say about A x equal b, but that'll be on 678 00:42:58 --> 00:42:59 Friday. 679 00:42:59 --> 00:43:04 OK, so that's, that's the null space. 680 00:43:04 --> 00:43:07 Thanks.