1 00:00:12.79 --> 00:00:15 OK. cameras are rolling. 2 00:00:15 --> 00:00:20 This is lecture fourteen, starting a new chapter. 3 00:00:20 --> 00:00:24 Chapter about orthogonality. 4 00:00:24 --> 00:00:29.01 What it means for vectors to be orthogonal. 5 00:00:29.01 --> 00:00:34 What it means for subspaces to be orthogonal. 6 00:00:34 --> 00:00:39 What it means for bases to be orthogonal. 7 00:00:39 --> 00:00:45 So ninety degrees, this is a ninety-degree 8 00:00:45 --> 00:00:46 chapter. 9 00:00:46 --> 00:00:52 So what does it mean -- let me jump to subspaces. 10 00:00:52 --> 00:00:58 Because I've drawn here the big picture. 11 00:00:58 --> 00:01:02.32 This is the 18.06 picture here. 12 00:01:02.32 --> 00:01:06 And hold it down, guys. 13 00:01:06 --> 00:01:13 So this is the picture and we know a lot about that picture 14 00:01:13 --> 00:01:14 already. 15 00:01:14 --> 00:01:19 We know the dimension of every subspace. 16 00:01:19 --> 00:01:24 We know that these dimensions are r and n-r. 17 00:01:24 --> 00:01:30 We know that these dimensions are r and m-r. 18 00:01:30 --> 00:01:36 What I want to show now is what this figure is saying, 19 00:01:36 --> 00:01:42 that the angle -- the figure is just my attempt 20 00:01:42 --> 00:01:47 to draw what I'm now going to say, that the angle between 21 00:01:47 --> 00:01:50 these subspaces is ninety degrees. 22 00:01:50 --> 00:01:54 And the angle between these subspaces is ninety degrees. 23 00:01:54 --> 00:01:58 Now I have to say what does that mean? 24 00:01:58 --> 00:02:03 What does it mean for subspaces to be orthogonal? 25 00:02:03 --> 00:02:07 But I hope you appreciate the beauty of this picture, 26 00:02:07 --> 00:02:12 that that those subspaces are going to come out to be 27 00:02:12 --> 00:02:13 orthogonal. 28 00:02:13 --> 00:02:16 Those two and also those two. 29 00:02:16 --> 00:02:21 So that's like one point, one important point to step 30 00:02:21 --> 00:02:24 forward in understanding those subspaces. 31 00:02:24 --> 00:02:30 We knew what each subspace was like, we could compute bases for 32 00:02:30 --> 00:02:31.98 them. 33 00:02:31.98 --> 00:02:34.17 Now we know more. 34 00:02:34.17 --> 00:02:37 Or we will in a few minutes. 35 00:02:37 --> 00:02:38 OK. 36 00:02:38 --> 00:02:46 I have to say first of all what does it mean for two vectors to 37 00:02:46 --> 00:02:47 be orthogonal? 38 00:02:47 --> 00:02:51 So let me start with that. 39 00:02:51 --> 00:02:53 Orthogonal vectors. 40 00:02:53 --> 00:03:00 The word orthogonal is -- is just another word for 41 00:03:00 --> 00:03:03 perpendicular. 42 00:03:03 --> 00:03:10 It means that in n-dimensional space the angle between those 43 00:03:10 --> 00:03:12.97 vectors is ninety degrees. 44 00:03:12.97 --> 00:03:17 It means that they form a right triangle. 45 00:03:17 --> 00:03:24 It even means that the going way back to the Greeks that this 46 00:03:24 --> 00:03:29 angle that this triangle a vector x, a vector x, 47 00:03:29 --> 00:03:35 and a vector x+y -- of course that'll be the 48 00:03:35 --> 00:03:42 hypotenuse, so what was it the Greeks figured out and it's 49 00:03:42 --> 00:03:43 neat. 50 00:03:43 --> 00:03:48 It's the fact that the -- so these are orthogonal, 51 00:03:48 --> 00:03:54 this is a right angle, if -- so let me put the great 52 00:03:54 --> 00:04:00.08 name down, Pythagoras, I'm looking for -- what I 53 00:04:00.08 --> 00:04:02 looking for? 54 00:04:02 --> 00:04:07 I'm looking for the condition if you give me two vectors, 55 00:04:07 --> 00:04:10 how do I know if they're orthogonal? 56 00:04:10 --> 00:04:13 How can I tell two perpendicular vectors? 57 00:04:13 --> 00:04:17 And actually you probably know the answer. 58 00:04:17 --> 00:04:19 Let me write the answer down. 59 00:04:19 --> 00:04:23 Orthogonal vectors, what's the test for 60 00:04:23 --> 00:04:25 orthogonality? 61 00:04:25 --> 00:04:32 I take the dot product which I tend to write as x transpose y, 62 00:04:32 --> 00:04:38 because that's a row times a column, and that matrix 63 00:04:38 --> 00:04:45 multiplication just gives me the right thing, that x1y1+x2y2 and 64 00:04:45 --> 00:04:51 so on, so these vectors are orthogonal if this result x 65 00:04:51 --> 00:04:54 transpose y is zero. 66 00:04:54 --> 00:04:55 That's the test. 67 00:04:55 --> 00:04:56 OK. 68 00:04:56 --> 00:05:01 Can I connect that to other things? 69 00:05:01 --> 00:05:06 I mean -- it's just beautiful that here we have we're in n 70 00:05:06 --> 00:05:11 dimensions, we've got a couple of vectors, we want to know the 71 00:05:11 --> 00:05:15.43 angle between them, and the right thing to look at 72 00:05:15.43 --> 00:05:19.05 is the simplest thing that you could imagine, 73 00:05:19.05 --> 00:05:20 the dot product. 74 00:05:20 --> 00:05:20 OK. 75 00:05:20 --> 00:05:21 Now why? 76 00:05:21 --> 00:05:25.55 So I'm answering the question now why -- 77 00:05:25.55 --> 00:05:31 let's add some justification to this fact, that's the test. 78 00:05:31 --> 00:05:36 OK, so Pythagoras would say we've got a right triangle, 79 00:05:36 --> 00:05:42 if that length squared plus that length squared equals that 80 00:05:42 --> 00:05:44 length squared. 81 00:05:44 --> 00:05:50 OK, can I write it as x squared plus y squared equals x plus y 82 00:05:50 --> 00:05:51 squared? 83 00:05:51 --> 00:05:56 Now don't, please don't think that this is always true. 84 00:05:56 --> 00:06:01 This is only going to be true in this -- it's going to be 85 00:06:01 --> 00:06:04 equivalent to orthogonality. 86 00:06:04 --> 00:06:08 For other triangles of course, it's not true. 87 00:06:08 --> 00:06:11 For other triangles it's not. 88 00:06:11 --> 00:06:16 But for a right triangle somehow that fact should connect 89 00:06:16 --> 00:06:18 to that fact. 90 00:06:18 --> 00:06:21 Can we just make that connection? 91 00:06:21 --> 00:06:26 What's the connection between this test for orthogonality and 92 00:06:26 --> 00:06:29 this statement of orthogonality? 93 00:06:29 --> 00:06:34 Well, I guess I have to say what is the length squared? 94 00:06:34 --> 00:06:39 So let's continue on the board underneath with that equation. 95 00:06:39 --> 00:06:46 Give me another way to express the length squared of a vector. 96 00:06:46 --> 00:06:49 And let me just give you a vector. 97 00:06:49 --> 00:06:51 The vector one, two, three. 98 00:06:51 --> 00:06:53.81 That's in three dimensions. 99 00:06:53.81 --> 00:06:58 What is the length squared of the vector x equal one, 100 00:06:58 --> 00:06:59 two, three? 101 00:06:59 --> 00:07:02 So how do you find the length squared? 102 00:07:02 --> 00:07:07 Well, really you just, you want the length of that 103 00:07:07 --> 00:07:14 vector that goes along one -- up two, and out three -- and 104 00:07:14 --> 00:07:19 we'll come back to that right triangle stuff. 105 00:07:19 --> 00:07:24 The length squared is exactly x transpose x. 106 00:07:24 --> 00:07:30 Whenever I see x transpose x, I know I've got a number that's 107 00:07:30 --> 00:07:31.83 positive. 108 00:07:31.83 --> 00:07:36 It's a length squared unless it -- 109 00:07:36 --> 00:07:41 unless x happens to be the zero vector, that's the one case 110 00:07:41 --> 00:07:44.07 where the length is zero. 111 00:07:44.07 --> 00:07:49 So right -- this is just x1 squared plus x2 squared plus so 112 00:07:49 --> 00:07:51 on, plus xn squared. 113 00:07:51 --> 00:07:56.52 So one -- in the example I gave you what was the length squared 114 00:07:56.52 --> 00:08:00 of that vector one, two, three? 115 00:08:00 --> 00:08:02.89 So you square those, you get one, 116 00:08:02.89 --> 00:08:06 four and nine, you add, you get fourteen. 117 00:08:06 --> 00:08:10 So the vector one, two, three has length fourteen. 118 00:08:10 --> 00:08:13 So let me just put down a vector here. 119 00:08:13 --> 00:08:17 Let x be the vector one, two, three. 120 00:08:17 --> 00:08:22 Let me cook up a vector that's orthogonal to it. 121 00:08:22 --> 00:08:27 So what's the vector that's orthogonal? 122 00:08:27 --> 00:08:35.45 So right down here, x squared is one plus four plus 123 00:08:35.45 --> 00:08:42 nine, fourteen, let me cook up a vector that's 124 00:08:42 --> 00:08:50 orthogonal to it, we'll get right that that -- 125 00:08:50 --> 00:08:56 those two vectors are orthogonal, the length of y 126 00:08:56 --> 00:09:01 squared is five, and x plus y is one and two 127 00:09:01 --> 00:09:06 making three, two and minus one making one, 128 00:09:06 --> 00:09:13 three and zero making three, and the length of this squared 129 00:09:13 --> 00:09:18 is nine plus one plus nine, nineteen. 130 00:09:18 --> 00:09:24 And sure enough, I haven't proved anything. 131 00:09:24 --> 00:09:29 I've just like checked to see that my test x transpose y 132 00:09:29 --> 00:09:32 equals zero, which is true, right? 133 00:09:32 --> 00:09:37 Everybody sees that x transpose y is zero here? 134 00:09:37 --> 00:09:39 That's maybe the main point. 135 00:09:39 --> 00:09:44 That you should get really quick at doing x transpose y, 136 00:09:44 --> 00:09:50 so it's just this plus this plus this and that's zero. 137 00:09:50 --> 00:09:54 And sure enough, that clicks with fourteen plus 138 00:09:54 --> 00:09:57 five agreeing with nineteen. 139 00:09:57 --> 00:10:00.5 Now let me just do that in letters. 140 00:10:00.5 --> 00:10:02 So that's y transpose y. 141 00:10:02 --> 00:10:06 And this is x plus y transpose x plus y. 142 00:10:06 --> 00:10:06 OK. 143 00:10:06 --> 00:10:10 So I'm looking, again, this isn't always true. 144 00:10:10 --> 00:10:11 I repeat. 145 00:10:11 --> 00:10:17 This is going to be true when we have a right angle. 146 00:10:17 --> 00:10:22 And let's just -- well, of course, I'm just going to 147 00:10:22 --> 00:10:24 simplify this stuff here. 148 00:10:24 --> 00:10:27 There's an x transpose x there. 149 00:10:27 --> 00:10:30 And there's a y transpose y there. 150 00:10:30 --> 00:10:32 And there's an x transpose y. 151 00:10:32 --> 00:10:35.49 And there's a y transpose x. 152 00:10:35.49 --> 00:10:40.31 I knew I could do that simplification because I'm just 153 00:10:40.31 --> 00:10:46 doing matrix multiplication and I've just followed the rules. 154 00:10:46 --> 00:10:47 OK. 155 00:10:47 --> 00:10:50.2 So x transpose x-s cancel. 156 00:10:50.2 --> 00:10:52 Y transpose y-s cancel. 157 00:10:52 --> 00:10:55 And what about these guys? 158 00:10:55 --> 00:11:02.65 What can you tell me about the inner product of x with y and 159 00:11:02.65 --> 00:11:06 the inner product of y with x? 160 00:11:06 --> 00:11:08 Is there a difference? 161 00:11:08 --> 00:11:14 I think if we -- while we're doing real vectors, 162 00:11:14 --> 00:11:18 which is all we're doing now, there isn't a difference, 163 00:11:18 --> 00:11:20 there's no difference. 164 00:11:20 --> 00:11:24.42 If I take x transpose y, that'll give me zero, 165 00:11:24.42 --> 00:11:29.68 if I took y transpose x I would have the same x1y1 and x2y2 and 166 00:11:29.68 --> 00:11:33 x3y3, it would be the same, so this is -- 167 00:11:33 --> 00:11:39 this is the same as that, this is really I'll knock that 168 00:11:39 --> 00:11:42 guy out and say two of these. 169 00:11:42 --> 00:11:48 So actually that's the -- this equation boiled down to this 170 00:11:48 --> 00:11:50 thing being zero. 171 00:11:50 --> 00:11:50 Right? 172 00:11:50 --> 00:11:56.67 Everything else canceled and this equation boiled down to 173 00:11:56.67 --> 00:11:58 that one. 174 00:11:58 --> 00:12:01 So that's really all I wanted. 175 00:12:01 --> 00:12:07 I just wanted to check that Pythagoras for a right triangle 176 00:12:07 --> 00:12:11.85 led me to this -- of course I cancel the two now. 177 00:12:11.85 --> 00:12:12.92 No problem. 178 00:12:12.92 --> 00:12:16 To x transpose y equals zero as the test. 179 00:12:16 --> 00:12:17 Fair enough. 180 00:12:17 --> 00:12:18 OK. 181 00:12:18 --> 00:12:20 You knew it was coming. 182 00:12:20 --> 00:12:26 The dot product of orthogonal vectors is zero. 183 00:12:26 --> 00:12:30 It's just -- I just want to say that's really neat. 184 00:12:30 --> 00:12:33.43 That it comes out so well. 185 00:12:33.43 --> 00:12:34 All right. 186 00:12:34 --> 00:12:40 Now what about -- so now I know if two -- when two vectors are 187 00:12:40 --> 00:12:41.29 orthogonal. 188 00:12:41.29 --> 00:12:46 By the way, what about if one of these guys is the zero 189 00:12:46 --> 00:12:48 vector? 190 00:12:48 --> 00:12:52 Suppose x is the zero vector, and y is whatever. 191 00:12:52 --> 00:12:53 Are they orthogonal? 192 00:12:53 --> 00:12:54.19 Sure. 193 00:12:54.19 --> 00:12:58 In math the one thing about math is you're supposed to 194 00:12:58 --> 00:13:00 follow the rules. 195 00:13:00 --> 00:13:04 So you're supposed to -- if x is the zero vector, 196 00:13:04 --> 00:13:09 you're supposed to take the zero vector dotted with y and of 197 00:13:09 --> 00:13:12 course you always get zero. 198 00:13:12 --> 00:13:19 So just so we're all sure, the zero vector is orthogonal 199 00:13:19 --> 00:13:21 to everybody. 200 00:13:21 --> 00:13:28 But what I want to -- what I now want to think about is 201 00:13:28 --> 00:13:29 subspaces. 202 00:13:29 --> 00:13:37 What does it mean for me to say that some subspace is orthogonal 203 00:13:37 --> 00:13:41 to some other subspace? 204 00:13:41 --> 00:13:42 So OK. 205 00:13:42 --> 00:13:45.64 Now I've got to write this down. 206 00:13:45.64 --> 00:13:51 So because we're defining definition of subspace S is 207 00:13:51 --> 00:13:57 orthogonal so to subspace let's say T, so I've got a couple of 208 00:13:57 --> 00:13:58 subspaces. 209 00:13:58 --> 00:14:04 And what should it mean for those guys to be orthogonal? 210 00:14:04 --> 00:14:10 It's just sort of what's the natural extension from 211 00:14:10 --> 00:14:16 orthogonal vectors to orthogonal subspaces? 212 00:14:16 --> 00:14:25 Well, and in particular, let's think of some orthogonal 213 00:14:25 --> 00:14:30 subspaces, like this wall. 214 00:14:30 --> 00:14:35 Let's say in three dimensions. 215 00:14:35 --> 00:14:43 So the blackboard extended to infinity, right, 216 00:14:43 --> 00:14:47 is a -- is a subspace, 217 00:14:47 --> 00:14:52 a plane, a two-dimensional subspace. 218 00:14:52 --> 00:14:58 It's a little bumpy but anyway, it's a -- think of it as a 219 00:14:58 --> 00:15:05 subspace, let me take the floor as another subspace. 220 00:15:05 --> 00:15:11 Again, it's not a great subspace, MIT only built it like 221 00:15:11 --> 00:15:16.42 so-so, but I'll put the origin right here. 222 00:15:16.42 --> 00:15:22 So the origin of the world is right there. 223 00:15:22 --> 00:15:23 OK. 224 00:15:23 --> 00:15:29 Thereby giving linear algebra its proper importance in this. 225 00:15:29 --> 00:15:29 OK. 226 00:15:29 --> 00:15:34 So there is one subspace, there's another one. 227 00:15:34 --> 00:15:35 The floor. 228 00:15:35 --> 00:15:38 And are they orthogonal? 229 00:15:38 --> 00:15:44 What does it mean for two subspaces to be orthogonal and 230 00:15:44 --> 00:15:49 in that special case are they orthogonal? 231 00:15:49 --> 00:15:51 All right. 232 00:15:51 --> 00:15:54 Let's finish this sentence. 233 00:15:54 --> 00:16:01 What does it mean means we have to know what we're talking about 234 00:16:01 --> 00:16:01 here. 235 00:16:01 --> 00:16:06 So what would be a reasonable idea of orthogonal? 236 00:16:06 --> 00:16:10 Well, let me put the right thing up. 237 00:16:10 --> 00:16:16 It means that every vector in S, every vector in S, 238 00:16:16 --> 00:16:22 is orthogonal to -- what I going to say? 239 00:16:22 --> 00:16:25 Every vector in T. 240 00:16:25 --> 00:16:32 That's a reasonable and it's a good and it's the right 241 00:16:32 --> 00:16:39 definition for two subspaces to be orthogonal. 242 00:16:39 --> 00:16:47 But I just want you to see, hey, what does that mean? 243 00:16:47 --> 00:16:55 So answer the question about the -- the blackboard and the 244 00:16:55 --> 00:16:57 floor. 245 00:16:57 --> 00:17:03 Are those two subspaces, they're two-dimensional, 246 00:17:03 --> 00:17:06 right, and we're in R^3. 247 00:17:06 --> 00:17:12 It's like a xz plane or something and a xy plane. 248 00:17:12 --> 00:17:14 Are they orthogonal? 249 00:17:14 --> 00:17:21 Is every vector in the blackboard orthogonal to every 250 00:17:21 --> 00:17:27 vector in the floor, starting from the origin right 251 00:17:27 --> 00:17:29 there? 252 00:17:29 --> 00:17:30 Yes or no? 253 00:17:30 --> 00:17:32 I could take a vote. 254 00:17:32 --> 00:17:37 Well we get some yeses and some noes. 255 00:17:37 --> 00:17:39 No is the answer. 256 00:17:39 --> 00:17:40 They're not. 257 00:17:40 --> 00:17:47 You can tell me a vector in the blackboard and a vector in the 258 00:17:47 --> 00:17:51.57 floor that are not orthogonal. 259 00:17:51.57 --> 00:17:57 Well you can tell me quite a few, I guess. 260 00:17:57 --> 00:18:02 Maybe like I could take some forty-five-degree guy in the 261 00:18:02 --> 00:18:08 blackboard, and something in the floor, they're not at ninety 262 00:18:08 --> 00:18:09 degrees, right? 263 00:18:09 --> 00:18:13.49 In fact, even more, you could tell me a vector 264 00:18:13.49 --> 00:18:18 that's in both the blackboard plane and the floor plane, 265 00:18:18 --> 00:18:23 so it's certainly not orthogonal to itself. 266 00:18:23 --> 00:18:26 So for sure, those two planes aren't 267 00:18:26 --> 00:18:27 orthogonal. 268 00:18:27 --> 00:18:29 What would that be? 269 00:18:29 --> 00:18:34 So what's a vector that's -- in both of those planes? 270 00:18:34 --> 00:18:39 It's this guy running along the crack there, in the 271 00:18:39 --> 00:18:42 intersection, the intersection. 272 00:18:42 --> 00:18:47 A vector, you know -- if two subspaces meet at some 273 00:18:47 --> 00:18:51 vector, well then for sure they're not orthogonal, 274 00:18:51 --> 00:18:54 because that vector is in one and it's in the other, 275 00:18:54 --> 00:18:58 and it's not orthogonal to itself unless it's zero. 276 00:18:58 --> 00:19:02 So the only I mean so orthogonal is for me to say 277 00:19:02 --> 00:19:05 these two subspaces are orthogonal first of all I'm 278 00:19:05 --> 00:19:11 certainly saying that they don't intersect in any nonzero vector. 279 00:19:11 --> 00:19:17 But also I mean more than that just not intersecting isn't good 280 00:19:17 --> 00:19:17 enough. 281 00:19:17 --> 00:19:22 So give me an example, oh, let's say in the plane, 282 00:19:22 --> 00:19:26 oh well, when do we have orthogonal subspaces in the 283 00:19:26 --> 00:19:27.37 plane? 284 00:19:27.37 --> 00:19:32 Yeah, tell me in the plane, so we don't -- there aren't 285 00:19:32 --> 00:19:37 that many different subspaces in the plane. 286 00:19:37 --> 00:19:42 What what have we got in the plane as possible subspaces? 287 00:19:42 --> 00:19:45 The zero vector, real small. 288 00:19:45 --> 00:19:47 A line through the origin. 289 00:19:47 --> 00:19:49 Or the whole plane. 290 00:19:49 --> 00:19:49 OK. 291 00:19:49 --> 00:19:55 Now so when is a line through the origin orthogonal to the 292 00:19:55 --> 00:19:56 whole plane? 293 00:19:56 --> 00:19:58 Never, right, never. 294 00:19:58 --> 00:20:03 When is a line through the origin orthogonal to the zero 295 00:20:03 --> 00:20:05 subspace? 296 00:20:05 --> 00:20:06 Always. 297 00:20:06 --> 00:20:06 Right. 298 00:20:06 --> 00:20:12 When is a line through the origin orthogonal to a different 299 00:20:12 --> 00:20:14.65 line through the origin? 300 00:20:14.65 --> 00:20:20 Well, that's the case that we all have a clear picture of, 301 00:20:20 --> 00:20:25 they -- the two lines have to meet at ninety degrees. 302 00:20:25 --> 00:20:30 They have only the -- so that's like this simple case I'm 303 00:20:30 --> 00:20:31 talking about. 304 00:20:31 --> 00:20:35.55 There's one subspace, there's the other subspace. 305 00:20:35.55 --> 00:20:37 They only meet at zero. 306 00:20:37 --> 00:20:39 And they're orthogonal. 307 00:20:39 --> 00:20:39 OK. 308 00:20:39 --> 00:20:39 Now. 309 00:20:39 --> 00:20:44 So we now know what it means for two subspaces to be 310 00:20:44 --> 00:20:45 orthogonal. 311 00:20:45 --> 00:20:53 And now I want to say that this is true for the row space and 312 00:20:53 --> 00:20:55 the null space. 313 00:20:55 --> 00:20:55 OK. 314 00:20:55 --> 00:20:58.55 So that's the neat fact. 315 00:20:58.55 --> 00:21:04 So row space is orthogonal to the null space. 316 00:21:04 --> 00:21:07 Now how did I come up with that? 317 00:21:07 --> 00:21:16 But you see the rank it's great, that means that these -- 318 00:21:16 --> 00:21:22 that these subspaces are just the right things, 319 00:21:22 --> 00:21:30 they're just cutting the whole space up into two perpendicular 320 00:21:30 --> 00:21:31 subspaces. 321 00:21:31 --> 00:21:32 OK. 322 00:21:32 --> 00:21:32 So why? 323 00:21:32 --> 00:21:37 Well, what have I got to work with? 324 00:21:37 --> 00:21:41.3 All I know is the null space. 325 00:21:41.3 --> 00:21:49 The null space has vectors that solve Ax equals zero. 326 00:21:49 --> 00:21:53 So this is a guy x. x is in the null space. 327 00:21:53 --> 00:21:55 Then Ax is zero. 328 00:21:55 --> 00:21:59 So why is it orthogonal to the rows of A? 329 00:21:59 --> 00:22:04 If I write down Ax equals zero, which is all I know about the 330 00:22:04 --> 00:22:10 null space, then I guess I want you to see that that's telling 331 00:22:10 --> 00:22:16 me, just that equation right there is telling me that the 332 00:22:16 --> 00:22:20 rows of A, let me write it out. 333 00:22:20 --> 00:22:23 There's row one of A. 334 00:22:23 --> 00:22:24 Row two. 335 00:22:24 --> 00:22:27 Row m of A. that's A. 336 00:22:27 --> 00:22:30 And it's multiplying X. 337 00:22:30 --> 00:22:34 And it's producing zero. 338 00:22:34 --> 00:22:34 OK. 339 00:22:34 --> 00:22:39 Written out that way you'll see it. 340 00:22:39 --> 00:22:48 So I'm saying that a vector in the row space is perpendicular 341 00:22:48 --> 00:22:52 to this guy X in the null space. 342 00:22:52 --> 00:22:57 And you see why? 343 00:22:57 --> 00:23:02 Because this equation is telling you that row one of A 344 00:23:02 --> 00:23:06 multiplying that's a dot product, right? 345 00:23:06 --> 00:23:12 Row one of A dot product with this x is producing this zero. 346 00:23:12 --> 00:23:15 So x is orthogonal to the first row. 347 00:23:15 --> 00:23:18 And to the second row. 348 00:23:18 --> 00:23:22 Row two of A, x is giving that zero. 349 00:23:22 --> 00:23:26 Row m of A times x is giving that zero. 350 00:23:26 --> 00:23:31 So x is -- the equation is telling me that x is orthogonal 351 00:23:31 --> 00:23:32 to all the rows. 352 00:23:32 --> 00:23:35 Right, it's just sitting there. 353 00:23:35 --> 00:23:41 That's all we -- it had to be sitting there because we didn't 354 00:23:41 --> 00:23:46 know anything more about the null space than this. 355 00:23:46 --> 00:23:50 And now I guess to be totally complete here I'd now check that 356 00:23:50 --> 00:23:53 x is orthogonal to each separate row. 357 00:23:53 --> 00:23:56 But what else strictly speaking do I have to do? 358 00:23:56 --> 00:23:59 To show that those subspaces are orthogonal, 359 00:23:59 --> 00:24:03 I have to take this x in the null space and show that it's 360 00:24:03 --> 00:24:07 orthogonal to every vector in the row space, 361 00:24:07 --> 00:24:10 every vector in the row space, so what -- 362 00:24:10 --> 00:24:13 what else is in the row space? 363 00:24:13 --> 00:24:19 This row is in the row space, that row is in the row space, 364 00:24:19 --> 00:24:23 they're all there, but it's not the whole row 365 00:24:23 --> 00:24:27.15 space, right, we just have to like remember, 366 00:24:27.15 --> 00:24:31 what does it mean, what does that word space 367 00:24:31 --> 00:24:33 telling us? 368 00:24:33 --> 00:24:35 And what else is in the row space? 369 00:24:35 --> 00:24:37 Besides the rows? 370 00:24:37 --> 00:24:38 All their combinations. 371 00:24:38 --> 00:24:42 So I really have to check that sure enough if x is 372 00:24:42 --> 00:24:46 perpendicular to row one, row two, all the different 373 00:24:46 --> 00:24:50 separate rows, then also x is perpendicular to 374 00:24:50 --> 00:24:53 a combination of the rows. 375 00:24:53 --> 00:24:57 And that's just matrix multiplication again. 376 00:24:57 --> 00:25:02 You know, I have row one transpose x is zero, 377 00:25:02 --> 00:25:08 so on, row two transpose x is zero, so I'm entitled to 378 00:25:08 --> 00:25:12 multiply that by some c1, this by some c2, 379 00:25:12 --> 00:25:16 I still have zeroes, I'm entitled to add, 380 00:25:16 --> 00:25:24 so I have c1 row one so -- so all this when I put that 381 00:25:24 --> 00:25:31 together that's big parentheses c1 row one plus c2 row two and 382 00:25:31 --> 00:25:32 so on. 383 00:25:32 --> 00:25:34 Transpose x is zero. 384 00:25:34 --> 00:25:35 Right? 385 00:25:35 --> 00:25:42 I just added the zeroes and got zero, and I just added these 386 00:25:42 --> 00:25:44 following the rule. 387 00:25:44 --> 00:25:46 No big deal. 388 00:25:46 --> 00:25:52 The whole point was right sitting in that. 389 00:25:52 --> 00:25:53 OK. 390 00:25:53 --> 00:26:00 So if I come back to this figure now I'm like a happier 391 00:26:00 --> 00:26:01 person. 392 00:26:01 --> 00:26:08 Because I have this -- I now see how those subspaces are 393 00:26:08 --> 00:26:09 oriented. 394 00:26:09 --> 00:26:14 And these subspaces are also oriented. 395 00:26:14 --> 00:26:21 Well, actually why is that orthogonality? 396 00:26:21 --> 00:26:27 Well, it's the same statement for A transpose that that one 397 00:26:27 --> 00:26:28.27 was for A. 398 00:26:28.27 --> 00:26:34 So I won't take time to prove it again because we've checked 399 00:26:34 --> 00:26:39 it for every matrix and A transpose is just as good a 400 00:26:39 --> 00:26:41 matrix as A. 401 00:26:41 --> 00:26:44 So we're orthogonal over there. 402 00:26:44 --> 00:26:48 So we really have carved up this -- 403 00:26:48 --> 00:26:53 this was like carving up m-dimensional space into two 404 00:26:53 --> 00:26:59.26 subspaces and this one was carving up n-dimensional space 405 00:26:59.26 --> 00:27:01 into two subspaces. 406 00:27:01 --> 00:27:03 And well, one more thing here. 407 00:27:03 --> 00:27:06 One more important thing. 408 00:27:06 --> 00:27:09 Let me move into three dimensions. 409 00:27:09 --> 00:27:15 Tell me a couple of orthogonal subspaces in three dimensions 410 00:27:15 --> 00:27:21 that somehow don't carve up the whole space, there's stuff left 411 00:27:21 --> 00:27:23.08 there. 412 00:27:23.08 --> 00:27:28 I'm thinking of a couple of orthogonal lines. 413 00:27:28 --> 00:27:33 If I -- suppose I'm in three dimensions, R^3. 414 00:27:33 --> 00:27:39 And I have one line, one one-dimensional subspace, 415 00:27:39 --> 00:27:42 and a perpendicular one. 416 00:27:42 --> 00:27:49 Could those be the row space and the null space? 417 00:27:49 --> 00:27:55 Could those be the row space and the null space? 418 00:27:55 --> 00:28:02 Could I be in three dimensions and have a row space that's a 419 00:28:02 --> 00:28:06 line and a null space that's a line? 420 00:28:06 --> 00:28:07 No. 421 00:28:07 --> 00:28:08 Why not? 422 00:28:08 --> 00:28:12 Because the dimensions aren't right. 423 00:28:12 --> 00:28:12 Right? 424 00:28:12 --> 00:28:17 The dimensions are no good. 425 00:28:17 --> 00:28:21 The dimensions here, r and n-r, they add up to 426 00:28:21 --> 00:28:23 three, they add up to n. 427 00:28:23 --> 00:28:28 If I take -- just follow that example -- if the row space is 428 00:28:28 --> 00:28:32 one-dimensional, suppose A is what's a good in 429 00:28:32 --> 00:28:37 R^3, I want a one-dimensional row space, let me take one, 430 00:28:37 --> 00:28:40 two, five, two, four, ten. 431 00:28:40 --> 00:28:45 What's the dimension of that row space? 432 00:28:45 --> 00:28:45.97 One. 433 00:28:45.97 --> 00:28:50 What's the dimension of the null space? 434 00:28:50 --> 00:28:57 Tell what's the null space look like in that case? 435 00:28:57 --> 00:29:00 The row space is a line, right? 436 00:29:00 --> 00:29:06.51 One-dimensional, it's just a line through one, 437 00:29:06.51 --> 00:29:09 two, five. 438 00:29:09 --> 00:29:14 Geometrically what's the row space look like? 439 00:29:14 --> 00:29:17 What's its dimension? 440 00:29:17 --> 00:29:22 So here r here n is three, the rank is one, 441 00:29:22 --> 00:29:29 so the dimension of the null space, so I'm looking at this x, 442 00:29:29 --> 00:29:30 x1, x2, x3. 443 00:29:30 --> 00:29:32 To give zero. 444 00:29:32 --> 00:29:40 So the dimension of the null space is we all know is two. 445 00:29:40 --> 00:29:41 Right. 446 00:29:41 --> 00:29:42 It's a plane. 447 00:29:42 --> 00:29:47 And now actually we know, we see better, 448 00:29:47 --> 00:29:49.07 what plane is it? 449 00:29:49.07 --> 00:29:51 What plane is it? 450 00:29:51 --> 00:29:55 It's the plane that's perpendicular to one, 451 00:29:55 --> 00:29:56 two, five. 452 00:29:56 --> 00:29:57 Right? 453 00:29:57 --> 00:29:58 We now see. 454 00:29:58 --> 00:30:04.08 In fact the two, four, ten didn't actually have 455 00:30:04.08 --> 00:30:07 any effect at all. 456 00:30:07 --> 00:30:11 I could have just ignored that. 457 00:30:11 --> 00:30:17 That didn't change the row space or the null space. 458 00:30:17 --> 00:30:21 I'll just make that one equation. 459 00:30:21 --> 00:30:21 Yeah. 460 00:30:21 --> 00:30:22 OK. 461 00:30:22 --> 00:30:22 Sure. 462 00:30:22 --> 00:30:26 That's the easiest to deal with. 463 00:30:26 --> 00:30:28 One equation. 464 00:30:28 --> 00:30:30 Three unknowns. 465 00:30:30 --> 00:30:36 And I want to ask -- what would the equation give me 466 00:30:36 --> 00:30:40 the null space, and you would have said back in 467 00:30:40 --> 00:30:45 September you would have said it gives you a plane, 468 00:30:45 --> 00:30:47 and we're completely right. 469 00:30:47 --> 00:30:51 And the plane it gives you, the normal vector, 470 00:30:51 --> 00:30:55.68 you remember in calculus, there was this dumb normal 471 00:30:55.68 --> 00:30:57 vector called N. 472 00:30:57 --> 00:30:59 Well there it is. 473 00:30:59 --> 00:31:02 One, two, five. 474 00:31:02 --> 00:31:02 OK. 475 00:31:02 --> 00:31:10 What is the what's the point I want to make here? 476 00:31:10 --> 00:31:20 I want to make -- I want to emphasize that not only are the 477 00:31:20 --> 00:31:24 -- let me write it in words. 478 00:31:24 --> 00:31:33 So I want to write the null space and the row space are 479 00:31:33 --> 00:31:43 orthogonal, that's this neat fact, which we've -- 480 00:31:43 --> 00:31:47 we've just checked from Ax equals zero, but now I want to 481 00:31:47 --> 00:31:51 say more because there's a little more that's true. 482 00:31:51 --> 00:31:55 Their dimensions add to the whole space. 483 00:31:55 --> 00:31:58 So that's like a little extra information. 484 00:31:58 --> 00:32:03 That it's not like I could have -- I couldn't have a line and a 485 00:32:03 --> 00:32:06 line in three dimensions. 486 00:32:06 --> 00:32:11 Those don't add up one and one don't add to three. 487 00:32:11 --> 00:32:16 So I used the word orthogonal complements in R^n. 488 00:32:16 --> 00:32:21 And the idea of this word complement is that the 489 00:32:21 --> 00:32:27 orthogonal complement of a row space contains not just some 490 00:32:27 --> 00:32:31 vectors that are orthogonal to it, but all. 491 00:32:31 --> 00:32:35.53 So what does that mean? 492 00:32:35.53 --> 00:32:42 That means that the null space contains all, 493 00:32:42 --> 00:32:50 not just some but all, vectors that are perpendicular 494 00:32:50 --> 00:32:52 to the row space. 495 00:32:52 --> 00:32:53.11 OK. 496 00:32:53.11 --> 00:33:01.67 Really what I've done in this half of the lecture is just 497 00:33:01.67 --> 00:33:09 notice some of the nice geometry that -- 498 00:33:09 --> 00:33:13 that we didn't pick up before because we didn't discuss 499 00:33:13 --> 00:33:16 perpendicular vectors before. 500 00:33:16 --> 00:33:19 But it was all sitting there. 501 00:33:19 --> 00:33:21 And now we picked it up. 502 00:33:21 --> 00:33:25 That these vectors are orthogonal complements. 503 00:33:25 --> 00:33:30 And I guess I even call this part one of the fundamental 504 00:33:30 --> 00:33:33 theorem of linear algebra. 505 00:33:33 --> 00:33:38 The fundamental theorem of linear algebra is about these 506 00:33:38 --> 00:33:42.41 four subspaces, so part one is about their 507 00:33:42.41 --> 00:33:46 dimension, maybe I should call it part two now. 508 00:33:46 --> 00:33:48 Their dimensions we got. 509 00:33:48 --> 00:33:52 Now we're getting their orthogonality, 510 00:33:52 --> 00:33:53 that's part two. 511 00:33:53 --> 00:33:57 And part three will be about bases for them. 512 00:33:57 --> 00:33:59 Orthogonal bases. 513 00:33:59 --> 00:34:02 So that's coming up. 514 00:34:02 --> 00:34:03 OK. 515 00:34:03 --> 00:34:09 So I'm happy with that geometry right now. 516 00:34:09 --> 00:34:09 OK. 517 00:34:09 --> 00:34:10 OK. 518 00:34:10 --> 00:34:16 Now what's my next goal in this chapter? 519 00:34:16 --> 00:34:21 Here's the main problem of the chapter. 520 00:34:21 --> 00:34:30 The main problem of the chapter is -- so this is coming. 521 00:34:30 --> 00:34:35 It's coming attraction. 522 00:34:35 --> 00:34:42 This is the very last chapter that's about Ax=b. 523 00:34:42 --> 00:34:50.03 I would like to solve that system of equations when there 524 00:34:50.03 --> 00:34:52 is no solution. 525 00:34:52 --> 00:34:58 You may say what a ridiculous thing to do. 526 00:34:58 --> 00:35:03 But I have to say it's done all the time. 527 00:35:03 --> 00:35:07 In fact it has to be done. 528 00:35:07 --> 00:35:14 You get -- so the problem is solve -- 529 00:35:14 --> 00:35:20 the best possible solve I'll put quote Ax=b when there is no 530 00:35:20 --> 00:35:21.65 solution. 531 00:35:21.65 --> 00:35:25 And of course what does that mean? 532 00:35:25 --> 00:35:28 b isn't in the column space. 533 00:35:28 --> 00:35:34 And it's quite typical if this matrix A is rectangular if I -- 534 00:35:34 --> 00:35:41 maybe I have m equations and that's bigger than the number of 535 00:35:41 --> 00:35:47 unknowns, then for sure the rank is not m, 536 00:35:47 --> 00:35:52 the rank couldn't be m now, so there'll be a lot of 537 00:35:52 --> 00:35:57 right-hand sides with no solution, but here's an example. 538 00:35:57 --> 00:36:01 Some satellite is buzzing along. 539 00:36:01 --> 00:36:03 You measure its position. 540 00:36:03 --> 00:36:06.98 You make a thousand measurements. 541 00:36:06.98 --> 00:36:12 So that gives you a thousand equations for the -- for the 542 00:36:12 --> 00:36:18 parameters that -- that give the position. 543 00:36:18 --> 00:36:23 But there aren't a thousand parameters, there's just maybe 544 00:36:23 --> 00:36:25 six or something. 545 00:36:25 --> 00:36:30 Or you're measuring the -- you're doing questionnaires. 546 00:36:30 --> 00:36:33 You're measuring resistances. 547 00:36:33 --> 00:36:35 You're taking pulses. 548 00:36:35 --> 00:36:40.69 You're measuring somebody's pulse rate. 549 00:36:40.69 --> 00:36:40 OK. 550 00:36:40 --> 00:36:42 There's just one unknown. 551 00:36:42 --> 00:36:43 The pulse rate. 552 00:36:43 --> 00:36:47 So you measure it once, OK, fine, but if you really 553 00:36:47 --> 00:36:50 want to know it, you measure it multiple times, 554 00:36:50 --> 00:36:53 but then the measurements have noise in them, 555 00:36:53 --> 00:36:57 so there's -- the problem is that in many many problems we've 556 00:36:57 --> 00:37:00 got too many equations and they've got noise in the 557 00:37:00 --> 00:37:02 right-hand side. 558 00:37:02 --> 00:37:08 So Ax=b I can't expect to solve it exactly right, 559 00:37:08 --> 00:37:15 because I don't even know what -- there's a measurement mistake 560 00:37:15 --> 00:37:16 in b. 561 00:37:16 --> 00:37:19.27 But there's information too. 562 00:37:19.27 --> 00:37:25 There's a lot of information about x in there. 563 00:37:25 --> 00:37:30 And what I want to do is like separate the noise, 564 00:37:30 --> 00:37:33 the junk, from the information. 565 00:37:33 --> 00:37:37 And so this is a straightforward linear algebra 566 00:37:37 --> 00:37:38 problem. 567 00:37:38 --> 00:37:42 How do I solve, what's the best solution? 568 00:37:42 --> 00:37:42.57 OK. 569 00:37:42.57 --> 00:37:42.96 Now. 570 00:37:42.96 --> 00:37:48 I want to say so that's like describes the problem in an 571 00:37:48 --> 00:37:50 algebraic way. 572 00:37:50 --> 00:37:56 I got some equations, I'm looking for the best 573 00:37:56 --> 00:37:57 solution. 574 00:37:57 --> 00:38:03 Well, one way to find it is -- one way to start, 575 00:38:03 --> 00:38:10 one way to find a solution is throw away equations until 576 00:38:10 --> 00:38:15.99 you've got a nice, square invertible system and 577 00:38:15.99 --> 00:38:17 solve that. 578 00:38:17 --> 00:38:21 That's not satisfactory. 579 00:38:21 --> 00:38:25 There's no reason in these measurements to say these 580 00:38:25 --> 00:38:30 measurements are perfect and these measurements are useless. 581 00:38:30 --> 00:38:34 We want to use all the measurements to get the best 582 00:38:34 --> 00:38:37 information, to get the maximum information. 583 00:38:37 --> 00:38:38 But how? 584 00:38:38 --> 00:38:38 OK. 585 00:38:38 --> 00:38:42 Let me anticipate a matrix that's going to show up. 586 00:38:42 --> 00:38:45 This A is typically rectangular. 587 00:38:45 --> 00:38:50 But a matrix that shows up whenever you have -- and we 588 00:38:50 --> 00:38:53 chapter three was all about rectangular matrices. 589 00:38:53 --> 00:38:57 And we know when this is solvable, you could do 590 00:38:57 --> 00:38:59 elimination on it, right? 591 00:38:59 --> 00:39:03 But I'm thinking hey, you do elimination and you get 592 00:39:03 --> 00:39:06 equation zero equal other non-zeroes. 593 00:39:06 --> 00:39:11 I'm thinking we really -- elimination is going to fail. 594 00:39:11 --> 00:39:14 So that's our question. 595 00:39:14 --> 00:39:19 Elimination will get us down to -- will tell us if there is a 596 00:39:19 --> 00:39:21 solution or not. 597 00:39:21 --> 00:39:23.35 But I'm now thinking not. 598 00:39:23.35 --> 00:39:23 OK. 599 00:39:23 --> 00:39:26.1 So what are we going to do? 600 00:39:26.1 --> 00:39:27 All right. 601 00:39:27 --> 00:39:32 I want to tell you to jump ahead to the matrix that will 602 00:39:32 --> 00:39:34 play a key role. 603 00:39:34 --> 00:39:41 So this is the matrix that you want to understand for this 604 00:39:41 --> 00:39:42.54 chapter four. 605 00:39:42.54 --> 00:39:46 And it's the matrix A transpose A. 606 00:39:46 --> 00:39:51 What's -- tell me some things about that matrix. 607 00:39:51 --> 00:39:55 So A is this m by n matrix, rectangular, 608 00:39:55 --> 00:40:02 but now I'm saying that the good matrix that shows up in the 609 00:40:02 --> 00:40:06 end is A transpose A. 610 00:40:06 --> 00:40:09 So tell me something about that. 611 00:40:09 --> 00:40:16 Is it -- what's the first thing you know about A transpose A. 612 00:40:16 --> 00:40:17 It's square. 613 00:40:17 --> 00:40:18 Right? 614 00:40:18 --> 00:40:23.49 Square because this is m by n and this is n by m. 615 00:40:23.49 --> 00:40:26 So this is the result is n by n. 616 00:40:26 --> 00:40:27.41 Good. 617 00:40:27.41 --> 00:40:28 Square. 618 00:40:28 --> 00:40:30 What else? 619 00:40:30 --> 00:40:32 It's symmetric. 620 00:40:32 --> 00:40:32 Good. 621 00:40:32 --> 00:40:34 It's symmetric. 622 00:40:34 --> 00:40:37 Because you remember how to do that. 623 00:40:37 --> 00:40:42.61 If we transpose that matrix let's transpose it, 624 00:40:42.61 --> 00:40:45 A transpose A, if I transpose it, 625 00:40:45 --> 00:40:51.15 then that comes first transposed, this comes second, 626 00:40:51.15 --> 00:40:56 transposed, and then transposing twice is leaves it 627 00:40:56 --> 00:41:01 -- brings it back to the same so 628 00:41:01 --> 00:41:03 it's symmetric. 629 00:41:03 --> 00:41:04 Good. 630 00:41:04 --> 00:41:10 Now we now know how to ask more about a matrix. 631 00:41:10 --> 00:41:14 I'm interested in is it invertible? 632 00:41:14 --> 00:41:18 If not, what's its null space? 633 00:41:18 --> 00:41:25.39 So I want to know about -- because you're going to see, 634 00:41:25.39 --> 00:41:30.18 well, let me -- let me even, 635 00:41:30.18 --> 00:41:34 well I shouldn't do this, but I will. 636 00:41:34 --> 00:41:41 Let me tell you what equation to solve when you can't solve 637 00:41:41 --> 00:41:42 that one. 638 00:41:42 --> 00:41:49 The good equation comes from multiplying both sides by A 639 00:41:49 --> 00:41:57 transpose, so the good equation that you get to is this one. 640 00:41:57 --> 00:42:00 A transpose Ax equals A transpose b. 641 00:42:00 --> 00:42:05 That will be the central equation in the chapter. 642 00:42:05 --> 00:42:08 So I think why not tell it to you. 643 00:42:08 --> 00:42:10 Why not admit it right away. 644 00:42:10 --> 00:42:11 OK. 645 00:42:11 --> 00:42:14 I have to -- I should really give x. 646 00:42:14 --> 00:42:20 I want to sort of indicate that this x isn't I mean this x was 647 00:42:20 --> 00:42:27 the solution to that equation if it existed, but probably didn't. 648 00:42:27 --> 00:42:32 Now let me give this a different name, 649 00:42:32 --> 00:42:33 x hat. 650 00:42:33 --> 00:42:39 Because I'm hoping this one will have a solution. 651 00:42:39 --> 00:42:45.3 And I'm saying that it's my best solution. 652 00:42:45.3 --> 00:42:50 I'll have to say what does best mean. 653 00:42:50 --> 00:42:55 But that's going to be my -- my plan. 654 00:42:55 --> 00:43:01 I'm going to say that the best solution solves this equation. 655 00:43:01 --> 00:43:07 So you see right away why I'm so interested in this matrix A 656 00:43:07 --> 00:43:08 transpose A. 657 00:43:08 --> 00:43:11 And in its invertibility. 658 00:43:11 --> 00:43:11 OK. 659 00:43:11 --> 00:43:14 Now, when is it invertible? 660 00:43:14 --> 00:43:14 OK. 661 00:43:14 --> 00:43:19 Let me take a case, let me just do an example and 662 00:43:19 --> 00:43:24 then -- I'll just pick a matrix here. 663 00:43:24 --> 00:43:29 Just so we see what A transpose A looks like. 664 00:43:29 --> 00:43:34 So let me take a matrix A one, one, one, one, 665 00:43:34 --> 00:43:35 two, five. 666 00:43:35 --> 00:43:38 Just to invent a matrix. 667 00:43:38 --> 00:43:40 So there's a matrix A. 668 00:43:40 --> 00:43:46.44 Notice that it has M equal three rows and N equal two 669 00:43:46.44 --> 00:43:48.59 columns. 670 00:43:48.59 --> 00:43:55 Its rank is -- the rank of that matrix is two. 671 00:43:55 --> 00:44:01 Right, yeah, the columns are independent. 672 00:44:01 --> 00:44:03 Does Ax equal b? 673 00:44:03 --> 00:44:08.94 If I look at Ax=b, so x is just x1 x2, 674 00:44:08.94 --> 00:44:11 and b is b1 b2 b3. 675 00:44:11 --> 00:44:15 Do I expect to solve Ax=b? 676 00:44:15 --> 00:44:20 What's -- no way, right? 677 00:44:20 --> 00:44:26 I mean linear algebra's great, but solving three equations 678 00:44:26 --> 00:44:30 with only two unknowns usually we can't do it. 679 00:44:30 --> 00:44:35 We can only solve it if this vector is b is what? 680 00:44:35 --> 00:44:40 I can solve that equation if that vector b1 b2 b3 is in the 681 00:44:40 --> 00:44:42 column space. 682 00:44:42 --> 00:44:47 If it's a combination of those columns then fine. 683 00:44:47 --> 00:44:50 But usually it won't be. 684 00:44:50 --> 00:44:55.96 The combinations just fill up a plane and most vectors aren't on 685 00:44:55.96 --> 00:44:56 that plane. 686 00:44:56 --> 00:45:02 So what I'm saying is that I'm going to work with the matrix A 687 00:45:02 --> 00:45:03 transpose A. 688 00:45:03 --> 00:45:09 And I just want to figure out in this example what A transpose 689 00:45:09 --> 00:45:09.93 A is. 690 00:45:09.93 --> 00:45:12 So it's two by two. 691 00:45:12 --> 00:45:18 The first entry is a three, the next entry is an eight, 692 00:45:18 --> 00:45:21.96 this entry is -- what's that entry? 693 00:45:21.96 --> 00:45:23 Eight, for sure. 694 00:45:23 --> 00:45:27 We knew it had to be, and this entry is, 695 00:45:27 --> 00:45:31 what's that now, getting out my trusty 696 00:45:31 --> 00:45:35.08 calculator, thirty, is that right? 697 00:45:35.08 --> 00:45:35 Thirty. 698 00:45:35 --> 00:45:40 And is that matrix invertible? 699 00:45:40 --> 00:45:42 There's an A transpose A. 700 00:45:42 --> 00:45:44 And it is invertible, right? 701 00:45:44 --> 00:45:49 Three, eight is not a multiple of eight, thirty -- and it's 702 00:45:49 --> 00:45:50 invertible. 703 00:45:50 --> 00:45:54 And that's the normal, that's what I expect. 704 00:45:54 --> 00:45:56 So this is I want to show. 705 00:45:56 --> 00:46:00 So here's the final -- here's the key point. 706 00:46:00 --> 00:46:06.64 The null space of A transpose A -- it's not going to be always 707 00:46:06.64 --> 00:46:07 invertible. 708 00:46:07 --> 00:46:13 Tell me a matrix -- I have to say that I can't say A transpose 709 00:46:13 --> 00:46:15 A is always invertible. 710 00:46:15 --> 00:46:18.31 Because that's asking too much. 711 00:46:18.31 --> 00:46:22 I mean what could the matrix A be, for example, 712 00:46:22 --> 00:46:27 so that A transpose A was not invertible? 713 00:46:27 --> 00:46:33 Well, it even could be the zero matrix. 714 00:46:33 --> 00:46:37 I mean that's like extreme case. 715 00:46:37 --> 00:46:45 Suppose I make this rank -- suppose I change to that A. 716 00:46:45 --> 00:46:54 Now I figure out A transpose A again and I get -- what do I 717 00:46:54 --> 00:46:55 get? 718 00:46:55 --> 00:47:03 I get nine, I get nine of course and here I get what's 719 00:47:03 --> 00:47:06 that entry? 720 00:47:06 --> 00:47:07 Twenty-seven. 721 00:47:07 --> 00:47:10.76 And is that matrix invertible? 722 00:47:10.76 --> 00:47:11 No. 723 00:47:11 --> 00:47:16 And why do I -- I knew it wouldn't be invertible anyway. 724 00:47:16 --> 00:47:20 Because this matrix only has rank one. 725 00:47:20 --> 00:47:24 And if I have a product of matrices of rank one, 726 00:47:24 --> 00:47:31 the product is not going to have a rank bigger than one. 727 00:47:31 --> 00:47:37 So I'm not surprised that the answer only has rank one. 728 00:47:37 --> 00:47:43 And that's what I -- always happens, that the rank of A 729 00:47:43 --> 00:47:48 transpose A comes out to equal the rank of A. 730 00:47:48 --> 00:47:54 So, yes, so the null space of A transpose A equals the null 731 00:47:54 --> 00:48:01 space of A, the rank of A transpose A equals the rank of 732 00:48:01 --> 00:48:02 A. 733 00:48:02 --> 00:48:10.13 So let's -- as soon as I can why that's true. 734 00:48:10.13 --> 00:48:18 But let's draw from that what the fact that I want. 735 00:48:18 --> 00:48:28.95 This tells me that this square symmetric matrix is invertible 736 00:48:28.95 --> 00:48:34 if -- so here's my conclusion. 737 00:48:34 --> 00:48:43 A transpose A is invertible if exactly when -- 738 00:48:43 --> 00:48:51 exactly if this null space is only got the zero vector. 739 00:48:51 --> 00:48:58 Which means the columns of A are independent. 740 00:48:58 --> 00:49:07 So A transpose A is invertible exactly if A has independent 741 00:49:07 --> 00:49:08 columns. 742 00:49:08 --> 00:49:16 That's the fact that I need about A transpose A. 743 00:49:16 --> 00:49:20 And then you'll see next time how A transpose A enters 744 00:49:20 --> 00:49:21 everything. 745 00:49:21 --> 00:49:24 Next lecture is actually a crucial one. 746 00:49:24 --> 00:49:28 Here I'm preparing for it by getting us thinking about A 747 00:49:28 --> 00:49:29 transpose A. 748 00:49:29 --> 00:49:33 And its rank is the same as the rank of A, and we can decide 749 00:49:33 --> 00:49:35 when it's invertible. 750 00:49:35 --> 00:49:36 OK. 751 00:49:36 --> 00:49:38 So I'll see you Friday. 752 00:49:38 --> 00:49:41 Thanks.