1 00:00:01 --> 00:00:12 OK, here's the last lecture in the chapter on orthogonality. 2 00:00:12 --> 00:00:20 So we met orthogonal vectors, two vectors, 3 00:00:20 --> 00:00:31 we met orthogonal subspaces, like the row space and null 4 00:00:31 --> 00:00:32 space. 5 00:00:32 --> 00:00:44 Now today we meet an orthogonal basis, and an orthogonal matrix. 6 00:00:44 --> 00:00:51 So we really -- this chapter cleans up 7 00:00:51 --> 00:00:52 orthogonality. 8 00:00:52 --> 00:00:57 And really I want -- I should use the word orthonormal. 9 00:00:57 --> 00:01:03 Orthogonal is -- so my vectors are q1,q2 up to qn -- I use the 10 00:01:03 --> 00:01:06 letter "q", here, to remind me, 11 00:01:06 --> 00:01:12 I'm talking about orthogonal things, not just any vectors, 12 00:01:12 --> 00:01:15 but orthogonal ones. 13 00:01:15 --> 00:01:17 So what does that mean? 14 00:01:17 --> 00:01:23 That means that every q is orthogonal to every other q. 15 00:01:23 --> 00:01:28 It's a natural idea, to have a basis that's headed 16 00:01:28 --> 00:01:33 off at ninety-degree angles, the inner products are all 17 00:01:33 --> 00:01:34 zero. 18 00:01:34 --> 00:01:41 Of course if q is -- certainly qi is not orthogonal to itself. 19 00:01:41 --> 00:01:44 But there we'll make the best choice again, 20 00:01:44 --> 00:01:46 make it a unit vector. 21 00:01:46 --> 00:01:51 Then qi transpose qi is one, for a unit vector. 22 00:01:51 --> 00:01:53 The length squared is one. 23 00:01:53 --> 00:01:57 And that's what I would use the word normal. 24 00:01:57 --> 00:02:00 So for this part, normalized, 25 00:02:00 --> 00:02:03 unit length for this part. 26 00:02:03 --> 00:02:03 OK. 27 00:02:03 --> 00:02:09 So first part of the lecture is how does having an orthonormal 28 00:02:09 --> 00:02:11 basis make things nice? 29 00:02:11 --> 00:02:12 It certainly does. 30 00:02:12 --> 00:02:18 It makes all the calculations better, a whole lot of numerical 31 00:02:18 --> 00:02:23 linear algebra is built around working with orthonormal 32 00:02:23 --> 00:02:27 vectors, because they never get out of 33 00:02:27 --> 00:02:30 hand, they never overflow or underflow. 34 00:02:30 --> 00:02:35 And I'll put them into a matrix Q, and then the second part of 35 00:02:35 --> 00:02:40 the lecture will be suppose my basis, my columns of A are not 36 00:02:40 --> 00:02:41 orthonormal. 37 00:02:41 --> 00:02:43 How do I make them so? 38 00:02:43 --> 00:02:48.41 And the two names associated with that simple idea are Graham 39 00:02:48.41 --> 00:02:50.36 and Schmidt. 40 00:02:50.36 --> 00:02:56 So the first part is we've got a basis like this. 41 00:02:56 --> 00:03:02 Let's put those into the columns of a matrix. 42 00:03:02 --> 00:03:10 So a matrix Q that has -- I'll put these orthonormal vectors, 43 00:03:10 --> 00:03:17 q1 will be the first column, qn will be the n-th column. 44 00:03:17 --> 00:03:24 And I want to say, I want to write this property, 45 00:03:24 --> 00:03:30 qi transpose qj being zero, I want to put that in a matrix 46 00:03:30 --> 00:03:31 form. 47 00:03:31 --> 00:03:36 And just the right thing is to look at Q transpose Q. 48 00:03:36 --> 00:03:42 So this chapter has been looking at A transpose A. 49 00:03:42 --> 00:03:47 So it's natural to look at Q transpose Q. 50 00:03:47 --> 00:03:51 And the beauty is it comes out perfectly. 51 00:03:51 --> 00:03:55 Because Q transpose has these vectors in its rows, 52 00:03:55 --> 00:04:01 the first row is q1 transpose, the nth row is qn transpose. 53 00:04:01 --> 00:04:03 So that's Q transpose. 54 00:04:03 --> 00:04:06 And now I want to multiply by Q. 55 00:04:06 --> 00:04:10 That has q1 along to qn in the columns. 56 00:04:10 --> 00:04:12 That's Q. 57 00:04:12 --> 00:04:14 And what do I get? 58 00:04:14 --> 00:04:19 You really -- this is the first simplest most basic fact, 59 00:04:19 --> 00:04:25 that how do orthonormal vectors, orthonormal columns in 60 00:04:25 --> 00:04:30 a matrix, what happens if I compute Q transpose Q? 61 00:04:30 --> 00:04:32 Do you see it? 62 00:04:32 --> 00:04:37 If I take the first row times the first column, 63 00:04:37 --> 00:04:39 what do I get? 64 00:04:39 --> 00:04:40.41 A one. 65 00:04:40.41 --> 00:04:44 If I take the first row times the second column, 66 00:04:44 --> 00:04:45 what do I get? 67 00:04:45 --> 00:04:46 Zero. 68 00:04:46 --> 00:04:48 That's the orthogonality. 69 00:04:48 --> 00:04:52 The first row times the last column is zero. 70 00:04:52 --> 00:04:57 And so I'm getting ones on the diagonal and I'm getting zeroes 71 00:04:57 --> 00:04:58 everywhere else. 72 00:04:58 --> 00:05:01 I'm getting the identity matrix. 73 00:05:01 --> 00:05:06.54 You see how that's -- it's just like the right 74 00:05:06.54 --> 00:05:08 calculation to do. 75 00:05:08 --> 00:05:13 If you have orthonormal columns, and the matrix doesn't 76 00:05:13 --> 00:05:15.08 have to be square here. 77 00:05:15.08 --> 00:05:17 We might have just two columns. 78 00:05:17 --> 00:05:21 And they might have four, lots of components. 79 00:05:21 --> 00:05:26 So but they're orthonormal, and when we do Q transpose 80 00:05:26 --> 00:05:30 times Q, that Q transpose times Q or A 81 00:05:30 --> 00:05:35 transpose A just asks for all those dot products. 82 00:05:35 --> 00:05:36 Rows times columns. 83 00:05:36 --> 00:05:41 And in this orthonormal case, we get the best possible 84 00:05:41 --> 00:05:43 answer, the identity. 85 00:05:43 --> 00:05:48 OK, so this is -- so I mean now we have a new bunch of important 86 00:05:48 --> 00:05:49 matrices. 87 00:05:49 --> 00:05:53 What have we seen previously? 88 00:05:53 --> 00:05:59 We've seen in the distant past we had triangular matrices, 89 00:05:59 --> 00:06:04 diagonal matrices, permutation matrices, 90 00:06:04 --> 00:06:10 that was early chapters, then we had row echelon forms, 91 00:06:10 --> 00:06:15 then in this chapter we've already seen projection 92 00:06:15 --> 00:06:20 matrices, and now we're seeing this new 93 00:06:20 --> 00:06:23.99 class of matrices with orthonormal columns. 94 00:06:23.99 --> 00:06:26 That's a very long expression. 95 00:06:26 --> 00:06:31 I sorry that I can't just call them orthogonal matrices. 96 00:06:31 --> 00:06:36 But that word orthogonal matrices -- or maybe I should be 97 00:06:36 --> 00:06:41.06 able to call it orthonormal matrices, why don't we call it 98 00:06:41.06 --> 00:06:45 orthonormal -- I mean that would be an 99 00:06:45 --> 00:06:47 absolutely perfect name. 100 00:06:47 --> 00:06:51 For Q, call it an orthonormal matrix because its columns are 101 00:06:51 --> 00:06:52 orthonormal. 102 00:06:52 --> 00:06:58.06 OK, but the convention is that we only use that name orthogonal 103 00:06:58.06 --> 00:07:02 matrix, we only use this -- this word orthogonal, 104 00:07:02 --> 00:07:07 we don't even say orthonormal for some unknown reason, 105 00:07:07 --> 00:07:10 matrix when it's square. 106 00:07:10 --> 00:07:16 So in the case when this is a square matrix, 107 00:07:16 --> 00:07:23 that's the case we call it an orthogonal matrix. 108 00:07:23 --> 00:07:30 And what's special about the case when it's square? 109 00:07:30 --> 00:07:36 When it's a square matrix, we've got its inverse, 110 00:07:36 --> 00:07:43 so -- so in the case if Q is square, 111 00:07:43 --> 00:07:51 then Q transpose Q equals I tells us -- let me write that 112 00:07:51 --> 00:07:58 underneath -- tells us that Q transpose is Q inverse. 113 00:07:58 --> 00:08:06.45 There we have the easy to remember property for a square 114 00:08:06.45 --> 00:08:11 matrix with orthonormal columns. 115 00:08:11 --> 00:08:17 That -- I need to write some examples down. 116 00:08:17 --> 00:08:20 Let's see. 117 00:08:20 --> 00:08:24 Some examples like if I take any -- so examples, 118 00:08:24 --> 00:08:27 let's do some examples. 119 00:08:27 --> 00:08:32 Any permutation matrix, let me take just some random 120 00:08:32 --> 00:08:34 permutation matrix. 121 00:08:34 --> 00:08:39 Permutation Q equals let's say oh, make it three by three, 122 00:08:39 --> 00:08:42 say zero, zero, one, one, zero, 123 00:08:42 --> 00:08:46 zero, zero, one, zero. 124 00:08:46 --> 00:08:46 OK. 125 00:08:46 --> 00:08:51 That certainly has unit vectors in its columns. 126 00:08:51 --> 00:08:57.67 Those vectors are certainly perpendicular to each other. 127 00:08:57.67 --> 00:09:00 And if I -- and so that's it. 128 00:09:00 --> 00:09:02 That makes it a Q. 129 00:09:02 --> 00:09:09 And -- if I took its transpose, if I multiplied by Q transpose, 130 00:09:09 --> 00:09:15 shall I do that -- and let me stick in Q transpose 131 00:09:15 --> 00:09:15 here. 132 00:09:15 --> 00:09:20 Just to do that multiplication once more, transpose it'll put 133 00:09:20 --> 00:09:24 the -- make that into a column, make that into a column, 134 00:09:24 --> 00:09:26 make that into a column. 135 00:09:26 --> 00:09:29 And the transpose is also -- another Q. 136 00:09:29 --> 00:09:31 Another orthonormal matrix. 137 00:09:31 --> 00:09:35 And when I multiply that product I get I. 138 00:09:35 --> 00:09:37 OK, so there's an example. 139 00:09:37 --> 00:09:41 And actually there's a second example. 140 00:09:41 --> 00:09:45 But those are real easy examples, right, 141 00:09:45 --> 00:09:50.39 I mean to get orthogonal columns by just putting ones in 142 00:09:50.39 --> 00:09:53 different places is like too easy. 143 00:09:53 --> 00:09:56 So let me keep going with examples. 144 00:09:56 --> 00:10:01 So here's another simple example. 145 00:10:01 --> 00:10:05 Cos theta sine theta, there's a unit vector, 146 00:10:05 --> 00:10:08 oh, let me even take it, well, yeah. 147 00:10:08 --> 00:10:14 Cos theta sine theta and now the other way I want sine theta 148 00:10:14 --> 00:10:15 cos theta. 149 00:10:15 --> 00:10:19 But I want the inner product to be zero. 150 00:10:19 --> 00:10:23 And if I put a minus there, it'll do it. 151 00:10:23 --> 00:10:27 So that's -- unit vector, 152 00:10:27 --> 00:10:29 that's a unit vector. 153 00:10:29 --> 00:10:34 And if I take the dot product, I get minus plus zero. 154 00:10:34 --> 00:10:34 OK. 155 00:10:34 --> 00:10:39 For example Q equals say one, one, one, minus one, 156 00:10:39 --> 00:10:42 is that an orthogonal matrix? 157 00:10:42 --> 00:10:48 I've got orthogonal columns there, but it's not quite an 158 00:10:48 --> 00:10:51.01 orthogonal matrix. 159 00:10:51.01 --> 00:10:54 How shall I fix it to be an orthogonal matrix? 160 00:10:54 --> 00:10:59 Well, what's the length of those column vectors, 161 00:10:59 --> 00:11:04 the dot product with themselves is -- right now it's two, 162 00:11:04 --> 00:11:06 right, the -- the length squared. 163 00:11:06 --> 00:11:11 The length squared would be one plus one would be two, 164 00:11:11 --> 00:11:15 the length would be square root of two, 165 00:11:15 --> 00:11:19 so I better divide by square root of two. 166 00:11:19 --> 00:11:19 OK. 167 00:11:19 --> 00:11:23 So there's a -- there now I have got an orthogonal matrix, 168 00:11:23 --> 00:11:27 in fact, it's this one -- when theta is pi over four. 169 00:11:27 --> 00:11:32 The cosines and well almost, I guess the minus sine is down 170 00:11:32 --> 00:11:34 there, so maybe, I don't know, 171 00:11:34 --> 00:11:38.76 maybe minus pi over four or something. 172 00:11:38.76 --> 00:11:39 OK. 173 00:11:39 --> 00:11:45 Let me do one final example, just to show that you can get 174 00:11:45 --> 00:11:46 bigger ones. 175 00:11:46 --> 00:11:53 Q equals let me take that matrix up in the corner and I'll 176 00:11:53 --> 00:11:58 sort of repeat that pattern, repeat it again, 177 00:11:58 --> 00:12:01 and then minus it down here. 178 00:12:01 --> 00:12:09 That's one of the world's favorite orthogonal matrices. 179 00:12:09 --> 00:12:12 I hope I got it right, is -- can you see whether -- if 180 00:12:12 --> 00:12:16 I take the inner product of one column with another one, 181 00:12:16 --> 00:12:21 let's see, if I take the inner product of that column with that 182 00:12:21 --> 00:12:24 I have two minuses and two pluses, that's good. 183 00:12:24 --> 00:12:29 When I take the inner product of that with that I have a plus 184 00:12:29 --> 00:12:32 and a minus, a minus and a plus. 185 00:12:32 --> 00:12:32 Good. 186 00:12:32 --> 00:12:35 I think it all works out. 187 00:12:35 --> 00:12:39.03 And what do I have to divide by now? 188 00:12:39.03 --> 00:12:42 To make those into unit vectors. 189 00:12:42 --> 00:12:47 Right now the vector one, one, one, one has length two. 190 00:12:47 --> 00:12:49 Square root of four. 191 00:12:49 --> 00:12:55 So I have to divide by two to make it unit vector, 192 00:12:55 --> 00:12:58 so there's another. 193 00:12:58 --> 00:13:03 That's my entire array of simple examples. 194 00:13:03 --> 00:13:10 This construction is named after a guy called Adhemar and 195 00:13:10 --> 00:13:16 we know how to do it for two, four, sixteen, 196 00:13:16 --> 00:13:22 sixty-four and so on, but we -- nobody knows exactly 197 00:13:22 --> 00:13:30 which size matrices have -- which size -- which sizes allow 198 00:13:30 --> 00:13:34 orthogonal matrices of ones and minus ones. 199 00:13:34 --> 00:13:38 So Adhemar matrix is an orthogonal matrix that's got 200 00:13:38 --> 00:13:42 ones and minus ones, and a lot of ones -- some we 201 00:13:42 --> 00:13:47 know, some other sizes, there couldn't be a five by 202 00:13:47 --> 00:13:48 five I think. 203 00:13:48 --> 00:13:54 But there are some sizes that nobody yet knows whether 204 00:13:54 --> 00:13:58 there could be or can't be a matrix like that. 205 00:13:58 --> 00:13:59.13 OK. 206 00:13:59.13 --> 00:14:02 You see those orthogonal matrices. 207 00:14:02 --> 00:14:07 Now let me ask what -- why is it good to have orthogonal 208 00:14:07 --> 00:14:08 matrices? 209 00:14:08 --> 00:14:11 What calculation is made easy? 210 00:14:11 --> 00:14:14 If I have an orthogonal matrix. 211 00:14:14 --> 00:14:19 And -- let me remember that the matrix could be 212 00:14:19 --> 00:14:21 rectangular. 213 00:14:21 --> 00:14:26 Shall I put down -- I better put a rectangular example down. 214 00:14:26 --> 00:14:29 So the -- these were all square examples. 215 00:14:29 --> 00:14:34.62 Can I put down just -- a rectangular one just to be sure 216 00:14:34.62 --> 00:14:37 that we realize that this is possible. 217 00:14:37 --> 00:14:39 let's help me out. 218 00:14:39 --> 00:14:43.55 Let's see, if I put like a one, 219 00:14:43.55 --> 00:14:47 two, two and a minus two, minus one, two. 220 00:14:47 --> 00:14:54 That's -- a matrix -- oh its columns aren't normalized yet. 221 00:14:54 --> 00:14:57 I always have to remember to do that. 222 00:14:57 --> 00:15:02 I always do that last because it's easy to do. 223 00:15:02 --> 00:15:06 What's the length of those columns? 224 00:15:06 --> 00:15:12 So if I wanted them -- if I wanted them to be length 225 00:15:12 --> 00:15:16 one, I should divide by their length, which is -- so I'd look 226 00:15:16 --> 00:15:20 at one squared plus two squared plus two squared, 227 00:15:20 --> 00:15:24 that's one and four and four is nine, so I take the square root 228 00:15:24 --> 00:15:27 and I need to divide by three. 229 00:15:27 --> 00:15:27 OK. 230 00:15:27 --> 00:15:29.75 So there is -- well, without that, 231 00:15:29.75 --> 00:15:33 I've got one orthonormal vector. 232 00:15:33 --> 00:15:35 I mean just one unit vector. 233 00:15:35 --> 00:15:37 Now put that guy in. 234 00:15:37 --> 00:15:42 Now I have a basis for the column space for a 235 00:15:42 --> 00:15:46 two-dimensional space, an orthonormal basis, 236 00:15:46 --> 00:15:47 right? 237 00:15:47 --> 00:15:52.27 These two columns are orthonormal, they would be an 238 00:15:52.27 --> 00:15:58 orthonormal basis for this two-dimensional space that they 239 00:15:58 --> 00:15:59 span. 240 00:15:59 --> 00:16:04 Orthonormal vectors by the way have got to be independent. 241 00:16:04 --> 00:16:09 It's easy to show that orthonormal vectors since 242 00:16:09 --> 00:16:14 they're headed off all at ninety degrees there's no combination 243 00:16:14 --> 00:16:16 that gives zero. 244 00:16:16 --> 00:16:21 Now if I wanted to create now a third one, I could either just 245 00:16:21 --> 00:16:26 put in some third vector that was independent 246 00:16:26 --> 00:16:32 and go to this Graham-Schmidt calculation that I'm going to 247 00:16:32 --> 00:16:36.9 explain, or I could be inspired and say look, 248 00:16:36.9 --> 00:16:42 that -- with that pattern, why not put a one in there, 249 00:16:42 --> 00:16:45 and a two in there, and a two in there, 250 00:16:45 --> 00:16:50 and try to fix up the signs so that they worked. 251 00:16:50 --> 00:16:51 Hmm. 252 00:16:51 --> 00:16:56 I don't know if I've done this too brilliantly. 253 00:16:56 --> 00:16:59 Let's see, what signs, that's minus, 254 00:16:59 --> 00:17:04 maybe I'd make a minus sign there, how would that be? 255 00:17:04 --> 00:17:06 Yeah, maybe that works. 256 00:17:06 --> 00:17:11 I think that those three columns are orthonormal and they 257 00:17:11 --> 00:17:17 -- the beauty of this -- this is the last example I'll probably 258 00:17:17 --> 00:17:22 find where there's no square root, the -- the punishing thing 259 00:17:22 --> 00:17:26 in Graham-Schmidt, maybe we better know that in 260 00:17:26 --> 00:17:30 advance, is that because I want these vectors to be unit 261 00:17:30 --> 00:17:34 vectors, I'm always running into square roots. 262 00:17:34 --> 00:17:37 I'm always dividing by lengths. 263 00:17:37 --> 00:17:40 And those lengths are square roots. 264 00:17:40 --> 00:17:46 So you'll see as soon as I do a Graham-Schmidt example, 265 00:17:46 --> 00:17:50 square roots are going to show up. 266 00:17:50 --> 00:17:56 But here are some examples where we did it without any 267 00:17:56 --> 00:17:57 square root. 268 00:17:57 --> 00:17:57 OK. 269 00:17:57 --> 00:17:58 OK. 270 00:17:58 --> 00:17:59 So -- so great. 271 00:17:59 --> 00:18:06 Now next question is what's the good of having a Q? 272 00:18:06 --> 00:18:09 What formulas become easier? 273 00:18:09 --> 00:18:15.59 Suppose I want to project, so suppose Q -- suppose Q has 274 00:18:15.59 --> 00:18:17 orthonormal columns. 275 00:18:17 --> 00:18:24 I'm using the letter Q to mean this, I'll write it this one 276 00:18:24 --> 00:18:29 more time, but I always mean when I write a Q, 277 00:18:29 --> 00:18:35 I always mean that it has orthonormal columns. 278 00:18:35 --> 00:18:41 So suppose I want to project onto its column space. 279 00:18:41 --> 00:18:45 So what's the projection matrix? 280 00:18:45 --> 00:18:52 What's the projection matrix is I project onto a column space? 281 00:18:52 --> 00:18:59 OK, that gives me a chance to review the projection section, 282 00:18:59 --> 00:19:07 including that big formula, which used to be -- those four 283 00:19:07 --> 00:19:13 As in a row, but now it's got Qs, because I'm projecting onto 284 00:19:13 --> 00:19:18 the column space of Q, so do you remember what it was? 285 00:19:18 --> 00:19:22 It's Q Q transpose Q inverse Q transpose. 286 00:19:22 --> 00:19:24 That's my four Qs in a row. 287 00:19:24 --> 00:19:26 But what's good here? 288 00:19:26 --> 00:19:32 What -- what makes this formula nice if 289 00:19:32 --> 00:19:38 I'm projecting onto a column space when I have orthonormal 290 00:19:38 --> 00:19:41 basis for that space? 291 00:19:41 --> 00:19:46 What makes it nice is this is the identity. 292 00:19:46 --> 00:19:49 I don't have to do any inversion. 293 00:19:49 --> 00:19:52 I just get Q Q transpose. 294 00:19:52 --> 00:19:57 So Q Q transpose is a projection matrix. 295 00:19:57 --> 00:20:02 Oh, I can't help -- I can't resist just checking 296 00:20:02 --> 00:20:06 the properties, what are the properties of a 297 00:20:06 --> 00:20:07 projection matrix? 298 00:20:07 --> 00:20:12 There are two properties to know for any projection matrix. 299 00:20:12 --> 00:20:16 And I'm saying that this is the right projection matrix when 300 00:20:16 --> 00:20:21 we've got this orthonormal basis in the columns. 301 00:20:21 --> 00:20:21 OK. 302 00:20:21 --> 00:20:24 So there's the projection matrix. 303 00:20:24 --> 00:20:27 Suppose the matrix is square. 304 00:20:27 --> 00:20:30 First just tell me first this extreme case. 305 00:20:30 --> 00:20:36 If my matrix is square and it's got these orthonormal columns, 306 00:20:36 --> 00:20:38 then what's the column space? 307 00:20:38 --> 00:20:43 If I have a square matrix and I have independent columns, 308 00:20:43 --> 00:20:49 and even orthonormal columns, then the column space is the 309 00:20:49 --> 00:20:51 whole space, right? 310 00:20:51 --> 00:20:56 And what's the projection matrix onto the whole space? 311 00:20:56 --> 00:20:58 The identity matrix. 312 00:20:58 --> 00:21:04 If I'm projecting in the whole space, every vector B is right 313 00:21:04 --> 00:21:09 where it's supposed to be and I don't have to move it by 314 00:21:09 --> 00:21:11 projection. 315 00:21:11 --> 00:21:18 So this would be -- I'll put in parentheses this is I if Q is 316 00:21:18 --> 00:21:19 square. 317 00:21:19 --> 00:21:22 Well that we said that already. 318 00:21:22 --> 00:21:27 If Q is square, that's the case where Q 319 00:21:27 --> 00:21:33 transpose is Q inverse, we can put it on the right, 320 00:21:33 --> 00:21:38 we can put it on the left, we always get the identity 321 00:21:38 --> 00:21:40 matrix, if it's square. 322 00:21:40 --> 00:21:46 But if it's not a square matrix then it's not -- we don't get 323 00:21:46 --> 00:21:48 the identity matrix. 324 00:21:48 --> 00:21:53 We have Q Q transpose, and just again what are those 325 00:21:53 --> 00:21:57.41 two properties of a projection matrix? 326 00:21:57.41 --> 00:21:59.63 First of all, it's symmetric. 327 00:21:59.63 --> 00:22:03.04 OK, no problem, that's certainly a symmetric 328 00:22:03.04 --> 00:22:03 matrix. 329 00:22:03 --> 00:22:07 So what's that second property of a projection? 330 00:22:07 --> 00:22:12 That if you project and project again you don't move the second 331 00:22:12 --> 00:22:12 time. 332 00:22:12 --> 00:22:17 So the other property of a projection matrix should be that 333 00:22:17 --> 00:22:21 Q Q transpose twice should be the same 334 00:22:21 --> 00:22:23 as Q Q transpose once. 335 00:22:23 --> 00:22:25 That's projection matrices. 336 00:22:25 --> 00:22:31 And that property better fall out right away because from the 337 00:22:31 --> 00:22:36 fact we know about orthonormal matrices, Q transpose Q is I. 338 00:22:36 --> 00:22:37 OK, you see it. 339 00:22:37 --> 00:22:42.52 In the middle here is sitting Q Q t- Q transpose Q, 340 00:22:42.52 --> 00:22:46 sorry, that's what I meant to say, 341 00:22:46 --> 00:22:48 Q transpose Q is I. 342 00:22:48 --> 00:22:53 So that's sitting right in the middle, that cancels out, 343 00:22:53 --> 00:22:57.13 to give the identity, we're left with one Q Q 344 00:22:57.13 --> 00:22:59 transpose, and we're all set. 345 00:22:59 --> 00:23:00 OK. 346 00:23:00 --> 00:23:04 So this is the projection matrix -- all the equation -- 347 00:23:04 --> 00:23:11 all the messy equations of this chapter become trivial when our 348 00:23:11 --> 00:23:16 matrix -- when we have this orthonormal basis. 349 00:23:16 --> 00:23:20 I mean what do I mean by all the equations? 350 00:23:20 --> 00:23:24 Well, the most important equation was the normal 351 00:23:24 --> 00:23:30 equation, do you remember old A transpose A x hat equals A 352 00:23:30 --> 00:23:31 transpose b? 353 00:23:31 --> 00:23:33 But now -- now A is Q. 354 00:23:33 --> 00:23:41 Now I'm thinking I have Q transpose Q X hat equals 355 00:23:41 --> 00:23:42 Q transpose b. 356 00:23:42 --> 00:23:46 And what's good about that? 357 00:23:46 --> 00:23:54 What's good is that matrix on the left side is the identity. 358 00:23:54 --> 00:24:01 The matrix on the left is the identity, Q transpose Q, 359 00:24:01 --> 00:24:07 normally it isn't, normally it's that matrix of 360 00:24:07 --> 00:24:12 inner products and you've to compute all those dopey inner 361 00:24:12 --> 00:24:15 products and -- and -- and solve the system. 362 00:24:15 --> 00:24:19 Here the inner products are all one or zero. 363 00:24:19 --> 00:24:21 This is the identity matrix. 364 00:24:21 --> 00:24:22 It's gone. 365 00:24:22 --> 00:24:24 And there's the answer. 366 00:24:24 --> 00:24:27 There's no inversion involved. 367 00:24:27 --> 00:24:29 Each component of x is a Q times b. 368 00:24:29 --> 00:24:36 What that equation is saying is that the i-th 369 00:24:36 --> 00:24:41 component is the i-th basis vector times b. 370 00:24:41 --> 00:24:49 That's -- probably the most important formula in some major 371 00:24:49 --> 00:24:55 parts of mathematics, that if we have orthonormal 372 00:24:55 --> 00:25:02 basis, then the component in the -- in the i-th, 373 00:25:02 --> 00:25:07 along the i-th -- the projection on the i-th basis 374 00:25:07 --> 00:25:11 vector is just qi transpose b. 375 00:25:11 --> 00:25:16 That number x that we look for is just a dot product. 376 00:25:16 --> 00:25:17 OK. 377 00:25:17 --> 00:25:23 OK, so I'm ready now for the sort of like second half of the 378 00:25:23 --> 00:25:25 lecture. 379 00:25:25 --> 00:25:30 Where we don't start with an orthogonal matrix, 380 00:25:30 --> 00:25:32 orthonormal vectors. 381 00:25:32 --> 00:25:39 We just start with independent vectors and we want to make them 382 00:25:39 --> 00:25:40 orthonormal. 383 00:25:40 --> 00:25:44 So I'm going to -- can I do that now? 384 00:25:44 --> 00:25:48 Now here comes Graham-Schmidt. 385 00:25:48 --> 00:25:50 So -- Graham-Schmidt. 386 00:25:50 --> 00:25:54.11 So this is a calculation, 387 00:25:54.11 --> 00:26:00 I won't say -- I can't quite say it's like elimination, 388 00:26:00 --> 00:26:05 because it's different, our goal isn't triangular 389 00:26:05 --> 00:26:06 anymore. 390 00:26:06 --> 00:26:12 With elimination our goal was make the matrix triangular. 391 00:26:12 --> 00:26:18 Now our goal is make the matrix orthogonal. 392 00:26:18 --> 00:26:22.2 Make those columns orthonormal. 393 00:26:22.2 --> 00:26:25.92 So let me start with two columns. 394 00:26:25.92 --> 00:26:29 So I start with vectors a and b. 395 00:26:29 --> 00:26:34 And they're just like -- here, let me draw them. 396 00:26:34 --> 00:26:36 Here's a. 397 00:26:36 --> 00:26:37 Here's b. 398 00:26:37 --> 00:26:38 For example. 399 00:26:38 --> 00:26:44 A isn't specially horizontal, wasn't meant to be, 400 00:26:44 --> 00:26:48 just a is one vector, b is another. 401 00:26:48 --> 00:26:53.7 I want to produce those two vectors, they might be in 402 00:26:53.7 --> 00:26:57 twelve-dimensional space, or they might be in 403 00:26:57 --> 00:26:59 two-dimensional space. 404 00:26:59 --> 00:27:02 They're independent, anyway. 405 00:27:02 --> 00:27:04 So I better be sure I say that. 406 00:27:04 --> 00:27:07 I start with independent vectors. 407 00:27:07 --> 00:27:12 And I want to produce out of that q 408 00:27:12 --> 00:27:17.4 1 and q2, I want to produce orthonormal vectors. 409 00:27:17.4 --> 00:27:21 And Graham and Schmidt tell me how. 410 00:27:21 --> 00:27:21 OK. 411 00:27:21 --> 00:27:27 Well, actually you could tell me how, we don't need -- 412 00:27:27 --> 00:27:33 frankly, I don't know -- there's only one idea here, 413 00:27:33 --> 00:27:39 if Graham had the idea, I don't know what Schmidt did. 414 00:27:39 --> 00:27:41 But OK. 415 00:27:41 --> 00:27:43 So you'll see it. 416 00:27:43 --> 00:27:47 We don't need either of them, actually. 417 00:27:47 --> 00:27:50 OK, so what I going to do. 418 00:27:50 --> 00:27:54 I'll take that -- this first guy. 419 00:27:54 --> 00:27:54.8 OK. 420 00:27:54.8 --> 00:27:56.65 Well, he's fine. 421 00:27:56.65 --> 00:28:02 That direction is fine except -- yeah, I'll say OK, 422 00:28:02 --> 00:28:06 I'll settle for that direction. 423 00:28:06 --> 00:28:10 So I'm going to -- I'm going to get, 424 00:28:10 --> 00:28:15 so what I going to -- my goal is I'm going to get orthogonal 425 00:28:15 --> 00:28:19 vectors and I'll call those capital A and B. 426 00:28:19 --> 00:28:24 So that's the key step is to get from any two vectors to two 427 00:28:24 --> 00:28:26 orthogonal vectors. 428 00:28:26 --> 00:28:29 And then at the end, no problem, I'll get 429 00:28:29 --> 00:28:33 orthonormal vectors, how will -- 430 00:28:33 --> 00:28:39 what will those will be my qs, q1 and q2, and what will they 431 00:28:39 --> 00:28:39 be? 432 00:28:39 --> 00:28:43 Once I've got A and B orthogonal, well, 433 00:28:43 --> 00:28:49 look, it's no big deal -- maybe that's what Schmidt did, 434 00:28:49 --> 00:28:54 he, brilliant Schmidt, thought OK, divide by the 435 00:28:54 --> 00:28:56 length, all right. 436 00:28:56 --> 00:28:59 That's Schmidt's contribution. 437 00:28:59 --> 00:28:59 OK. 438 00:28:59 --> 00:29:05 But Graham had a little more thinking to do, 439 00:29:05 --> 00:29:06 right? 440 00:29:06 --> 00:29:09 We haven't done Graham's part. 441 00:29:09 --> 00:29:13 This part except OK, I'm happy with A, 442 00:29:13 --> 00:29:14.65 A can be A. 443 00:29:14.65 --> 00:29:17 That first direction is fine. 444 00:29:17 --> 00:29:21 Why should -- no complaint about that. 445 00:29:21 --> 00:29:27 The trouble is the second direction is not fine. 446 00:29:27 --> 00:29:31 Because it's not orthogonal to the first. 447 00:29:31 --> 00:29:38 I'm looking for a vector that's -- starts with B, 448 00:29:38 --> 00:29:41 but makes it orthogonal to A. 449 00:29:41 --> 00:29:43.32 What's the vector? 450 00:29:43.32 --> 00:29:45 How do I do that? 451 00:29:45 --> 00:29:51 How do I produce from this vector a piece that's orthogonal 452 00:29:51 --> 00:29:52 to this one? 453 00:29:52 --> 00:29:58 And the -- remember these vectors might be in two 454 00:29:58 --> 00:30:03 dimensions or they might be in twelve 455 00:30:03 --> 00:30:04 dimensions. 456 00:30:04 --> 00:30:06 I'm just looking for the idea. 457 00:30:06 --> 00:30:07 So what's the idea? 458 00:30:07 --> 00:30:12 Where did we have orthogonal -- a vector showing up that was 459 00:30:12 --> 00:30:14 orthogonal to this guy? 460 00:30:14 --> 00:30:18 Well, that was the first basic calculation of the whole 461 00:30:18 --> 00:30:19 chapter. 462 00:30:19 --> 00:30:23 We -- we did a projection and the projection gave us this 463 00:30:23 --> 00:30:27 part, which was the part in the A 464 00:30:27 --> 00:30:28 direction. 465 00:30:28 --> 00:30:33 Now, the part we want is the other part, the e part. 466 00:30:33 --> 00:30:34 This part. 467 00:30:34 --> 00:30:39 This is going to be our -- that guy is that guy. 468 00:30:39 --> 00:30:41 This is our vector B. 469 00:30:41 --> 00:30:46 That gives us that ninety-degree angle. 470 00:30:46 --> 00:30:53 So B is you could say -- B is really what we previously called 471 00:30:53 --> 00:30:53 e. 472 00:30:53 --> 00:30:55 The error vector. 473 00:30:55 --> 00:30:57 And what is it? 474 00:30:57 --> 00:31:01 I mean what do I -- what is B here? 475 00:31:01 --> 00:31:03 A is A, no problem. 476 00:31:03 --> 00:31:09 B is -- OK, what's this error piece? 477 00:31:09 --> 00:31:10.72 Do you remember? 478 00:31:10.72 --> 00:31:16 It's I start with the original B and I take away what? 479 00:31:16 --> 00:31:18 Its projection, this P. 480 00:31:18 --> 00:31:22 This -- the vector B, this error vector, 481 00:31:22 --> 00:31:27 is the original vector removing the projection. 482 00:31:27 --> 00:31:31 So instead of wanting the projection, 483 00:31:31 --> 00:31:34 now that's what I want to throw away. 484 00:31:34 --> 00:31:38 I want to get the part that's perpendicular. 485 00:31:38 --> 00:31:41 And there will be a perpendicular part, 486 00:31:41 --> 00:31:42.82 it won't be zero. 487 00:31:42.82 --> 00:31:47 Because these vectors were independent, so B -- if B was 488 00:31:47 --> 00:31:52 along the direction of A, then if the original B and A 489 00:31:52 --> 00:31:56 were in the same direction, then I'm -- I've only got one 490 00:31:56 --> 00:31:57.08 direction. 491 00:31:57.08 --> 00:32:00 But here they're in two independent directions and all 492 00:32:00 --> 00:32:03 I'm doing is getting that guy. 493 00:32:03 --> 00:32:04 So what's its formula? 494 00:32:04 --> 00:32:09 What's the formula for that if -- I want to subtract 495 00:32:09 --> 00:32:14 the projection, so do you remember the 496 00:32:14 --> 00:32:15 projection? 497 00:32:15 --> 00:32:21 It's some multiple of A and what's that multiple? 498 00:32:21 --> 00:32:29 It's -- it's that thing we called x in the very very first 499 00:32:29 --> 00:32:32 lecture on this chapter. 500 00:32:32 --> 00:32:39 There's an A transpose A in the bottom and there's an A 501 00:32:39 --> 00:32:44 transpose B, isn't that it? 502 00:32:44 --> 00:32:46 I think that's Graham's formula. 503 00:32:46 --> 00:32:47 Or Graham-Schmidt. 504 00:32:47 --> 00:32:49 No, that's Graham. 505 00:32:49 --> 00:32:53 Schmidt has got to divide the whole thing by the length, 506 00:32:53 --> 00:32:57 so he -- his formula makes a mess which I'm not willing to 507 00:32:57 --> 00:32:58 write down. 508 00:32:58 --> 00:33:01 So let's just see that what I saying here? 509 00:33:01 --> 00:33:06 I'm saying that this vector is perpendicular to A. 510 00:33:06 --> 00:33:08 That these are orthogonal. 511 00:33:08 --> 00:33:10 A is perpendicular to B. 512 00:33:10 --> 00:33:11 Can you check that? 513 00:33:11 --> 00:33:16 How do you see that yes, of course, we -- our picture is 514 00:33:16 --> 00:33:19 telling us, yes, we did it right. 515 00:33:19 --> 00:33:23 How would I check that this matrix is perpendicular to A? 516 00:33:23 --> 00:33:29 I would multiply by A transpose and I better get zero, 517 00:33:29 --> 00:33:30 right? 518 00:33:30 --> 00:33:32 I should check that. 519 00:33:32 --> 00:33:35 A transpose B should come out zero. 520 00:33:35 --> 00:33:41.71 So this is A transpose times -- now what did we say B was? 521 00:33:41.71 --> 00:33:46 We start with the original B, and we take away this 522 00:33:46 --> 00:33:51 projection, and that should come out zero. 523 00:33:51 --> 00:33:56 Well, here we get an A transpose B minus -- and here is 524 00:33:56 --> 00:34:03 another A transpose B, and the -- and it's an A 525 00:34:03 --> 00:34:09 transpose A over A transpose A, a one, those cancel, 526 00:34:09 --> 00:34:12 and we do get zero. 527 00:34:12 --> 00:34:13 Right. 528 00:34:13 --> 00:34:17 Now I guess I can do numbers in there. 529 00:34:17 --> 00:34:24 But I have to take a third vector to be sure we've got this 530 00:34:24 --> 00:34:26.44 system down. 531 00:34:26.44 --> 00:34:34 So now I have to say if I have independent vectors A, 532 00:34:34 --> 00:34:39 B and C, I'm looking for orthogonal vectors A, 533 00:34:39 --> 00:34:44 B and capital C, and then of course the third 534 00:34:44 --> 00:34:50 guy will just be C over its length, the unit vector. 535 00:34:50 --> 00:34:54 So this is now the problem. 536 00:34:54 --> 00:34:55 I got B here. 537 00:34:55 --> 00:34:58 I got A very easily. 538 00:34:58 --> 00:35:05 And now -- if you see the idea, we could figure out a formula 539 00:35:05 --> 00:35:07 for C. 540 00:35:07 --> 00:35:13 So now that -- so this is like a typical homework quiz problem. 541 00:35:13 --> 00:35:16.81 I give you two vectors, you do this, 542 00:35:16.81 --> 00:35:21 I give you three vectors, and you have to make them 543 00:35:21 --> 00:35:22 orthonormal. 544 00:35:22 --> 00:35:27 So you do this again, the first vector's fine, 545 00:35:27 --> 00:35:31 the second vector is perpendicular 546 00:35:31 --> 00:35:35 to the first, and now I need a third vector 547 00:35:35 --> 00:35:39 that's perpendicular to the first one and the second one. 548 00:35:39 --> 00:35:40 Right? 549 00:35:40 --> 00:35:44 Tthis is the end of a -- the lecture is to find this guy. 550 00:35:44 --> 00:35:49 Find this vector -- this vector C, that's perpendicular we n- at 551 00:35:49 --> 00:35:52 this point we know A and B. 552 00:35:52 --> 00:35:58 But C, the little c that we're given, is off in some -- it's 553 00:35:58 --> 00:36:02.82 got to come out of the blackboard to be independent, 554 00:36:02.82 --> 00:36:08 so -- so can I sort of draw off -- off comes a c somewhere. 555 00:36:08 --> 00:36:12 I don't know, where I going to put the darn 556 00:36:12 --> 00:36:12 thing? 557 00:36:12 --> 00:36:17 Maybe I'll put it off, oh, I don't know, 558 00:36:17 --> 00:36:21 like that somehow, C, little c. 559 00:36:21 --> 00:36:26 And I already know that perpendicular direction, 560 00:36:26 --> 00:36:29.17 that one and that one. 561 00:36:29.17 --> 00:36:31 So now what's the idea? 562 00:36:31 --> 00:36:36 Give me the Graham-Schmidt formula for C. 563 00:36:36 --> 00:36:38 What is this C here? 564 00:36:38 --> 00:36:40 Equals what? 565 00:36:40 --> 00:36:43 What I going to do? 566 00:36:43 --> 00:36:46 I'll start with the given one. 567 00:36:46 --> 00:36:47 As before. 568 00:36:47 --> 00:36:47 Right? 569 00:36:47 --> 00:36:50 I start with the vector I'm given. 570 00:36:50 --> 00:36:52 And what do I do with it? 571 00:36:52 --> 00:36:56 I want to remove out of it, I want to subtract off, 572 00:36:56 --> 00:37:01 so I'll put a minus sign in, I want to subtract off its 573 00:37:01 --> 00:37:04 components in the A, capital A and capital B 574 00:37:04 --> 00:37:06 directions. 575 00:37:06 --> 00:37:11 I just want to get those out of there. 576 00:37:11 --> 00:37:14 Well, I know how to do that. 577 00:37:14 --> 00:37:16 I did it with B. 578 00:37:16 --> 00:37:23 So I'll just -- so let me take away -- what if I do this? 579 00:37:23 --> 00:37:24 What have I done? 580 00:37:24 --> 00:37:31 I've got little c and what have I subtracted from it? 581 00:37:31 --> 00:37:36 Its component, its projection if 582 00:37:36 --> 00:37:39 you like, in the A direction. 583 00:37:39 --> 00:37:46 And now I've got to subtract off its component B transpose C 584 00:37:46 --> 00:37:50 over B transpose B, that multiple of B, 585 00:37:50 --> 00:37:55 is its component in the B direction. 586 00:37:55 --> 00:38:02 And that gives me the vector capital C that if anything is -- 587 00:38:02 --> 00:38:09.56 if there's any justice, this C should be perpendicular 588 00:38:09.56 --> 00:38:14 to A and it should be perpendicular to B. 589 00:38:14 --> 00:38:19 And the only thing it's -- hasn't got is unit vector, 590 00:38:19 --> 00:38:24.68 so we divide by its length to get that too. 591 00:38:24.68 --> 00:38:25 OK. 592 00:38:25 --> 00:38:27 Let me do an example. 593 00:38:27 --> 00:38:33 Can I -- I'll make my life easy, I'll just take two 594 00:38:33 --> 00:38:35 vectors. 595 00:38:35 --> 00:38:38 So let me do a numerical example. 596 00:38:38 --> 00:38:43 If I'll give you two vectors, you give me back the 597 00:38:43 --> 00:38:48 Graham-Schmidt orthonormal basis, and we'll see how to 598 00:38:48 --> 00:38:51 express that in matrix form. 599 00:38:51 --> 00:38:51 OK. 600 00:38:51 --> 00:38:55 So let me give you the two vectors. 601 00:38:55 --> 00:39:00 So I'll take the vector A equals let's say 602 00:39:00 --> 00:39:03 one, one, one, why not? 603 00:39:03 --> 00:39:08 And B equals let's say one, zero, two, OK? 604 00:39:08 --> 00:39:15 I didn't want to cheat and make them orthogonal in the first 605 00:39:15 --> 00:39:20 place because then Graham-Schmidt wouldn't be 606 00:39:20 --> 00:39:21 needed. 607 00:39:21 --> 00:39:22 OK. 608 00:39:22 --> 00:39:25 So those are not orthogonal. 609 00:39:25 --> 00:39:29 So what is capital A? 610 00:39:29 --> 00:39:32 Well that's the same as big A. 611 00:39:32 --> 00:39:33 That was fine. 612 00:39:33 --> 00:39:34 What's B? 613 00:39:34 --> 00:39:40 So B is this b -- is the original B, and then I subtract 614 00:39:40 --> 00:39:42 off some multiple of the A. 615 00:39:42 --> 00:39:44.99 And what's the multiple? 616 00:39:44.99 --> 00:39:46 What goes in here? 617 00:39:46 --> 00:39:52 B -- here's the A -- this is the -- this is the little b, 618 00:39:52 --> 00:39:56 this is the big A, also the little a, 619 00:39:56 --> 00:40:03 and I want to multiply it by that right -- that right ratio, 620 00:40:03 --> 00:40:07 which has A transpose A, here's my ratio. 621 00:40:07 --> 00:40:09 I'm just doing this. 622 00:40:09 --> 00:40:14 So it's A transpose B, what is A transpose B, 623 00:40:14 --> 00:40:16.97 it looks like three. 624 00:40:16.97 --> 00:40:23 And what is A -- oh, my -- what's A transpose A? 625 00:40:23 --> 00:40:23.8 Three. 626 00:40:23.8 --> 00:40:24 I'm sorry. 627 00:40:24 --> 00:40:28 I didn't know that was going to happen. 628 00:40:28 --> 00:40:28 OK. 629 00:40:28 --> 00:40:30 But it happened. 630 00:40:30 --> 00:40:32 Why should we knock it? 631 00:40:32 --> 00:40:32 OK. 632 00:40:32 --> 00:40:35 So do you see it all right? 633 00:40:35 --> 00:40:39 That's A transpose B, there's A transpose A, 634 00:40:39 --> 00:40:44 that's the fraction, so I take this away, 635 00:40:44 --> 00:40:50 and I get one take away one is a zero, zero minus this one is a 636 00:40:50 --> 00:40:53 minus one, and two minus the one is a one. 637 00:40:53 --> 00:40:54 OK. 638 00:40:54 --> 00:40:58 And what's this vector that we finally found? 639 00:40:58 --> 00:40:59 This is B. 640 00:40:59 --> 00:41:01 And how do I know it's right? 641 00:41:01 --> 00:41:05.03 How do I know I've got a vector I want? 642 00:41:05.03 --> 00:41:09 I check that B is perpendicular to -- that A and B are 643 00:41:09 --> 00:41:12 perpendicular. 644 00:41:12 --> 00:41:15 That A is perpendicular to B. 645 00:41:15 --> 00:41:17 Just look at that. 646 00:41:17 --> 00:41:22 That one -- the dot product of that with that is zero. 647 00:41:22 --> 00:41:22 OK. 648 00:41:22 --> 00:41:25 So now what is my q1 and q2? 649 00:41:25 --> 00:41:28 Why don't I put them in a matrix? 650 00:41:28 --> 00:41:29 Of course. 651 00:41:29 --> 00:41:36 Since I'm always putting these -- so the Q, I'll put the q1 and 652 00:41:36 --> 00:41:39.14 the q2 in a matrix. 653 00:41:39.14 --> 00:41:41 And what are they? 654 00:41:41 --> 00:41:46 Now when I'm writing q-s I'm supposed to make things 655 00:41:46 --> 00:41:47 normalized. 656 00:41:47 --> 00:41:51 I'm supposed to make things unit vectors. 657 00:41:51 --> 00:41:58 So I'm going to take that A but I'm going to divide it by square 658 00:41:58 --> 00:41:59 root of three. 659 00:41:59 --> 00:42:04 And I'm going to take this B but I'm 660 00:42:04 --> 00:42:10 going to divide it by square root of two to make it a unit 661 00:42:10 --> 00:42:13 vector, and there is my matrix. 662 00:42:13 --> 00:42:18.93 That's my matrix with orthonormal columns coming from 663 00:42:18.93 --> 00:42:24 Graham-Schmidt and it sort of it -- it came from the original 664 00:42:24 --> 00:42:27 one, one, one, one, zero, two, 665 00:42:27 --> 00:42:28 right? 666 00:42:28 --> 00:42:31 That was my original guys. 667 00:42:31 --> 00:42:35.71 These were the two I started with. 668 00:42:35.71 --> 00:42:39 These are the two that I'm happy to end with. 669 00:42:39 --> 00:42:42 Because those are orthonormal. 670 00:42:42 --> 00:42:45 So that's what Graham-Schmidt did. 671 00:42:45 --> 00:42:51 It -- well, tell me about the column spaces of these matrices. 672 00:42:51 --> 00:42:57 How is the column space of Q related to the column space of 673 00:42:57 --> 00:42:57 A? 674 00:42:57 --> 00:43:03 So I'm always asking you things like this, and that makes you 675 00:43:03 --> 00:43:07 think, OK, the column space is all 676 00:43:07 --> 00:43:10 combinations of the columns, it's that plane, 677 00:43:10 --> 00:43:10 right? 678 00:43:10 --> 00:43:14.74 I've got two vectors in three-dimensional space, 679 00:43:14.74 --> 00:43:19 their column space is a plane, the column space of this matrix 680 00:43:19 --> 00:43:23 is a plane, what's the relation between the planes? 681 00:43:23 --> 00:43:25.97 Between the two column spaces? 682 00:43:25.97 --> 00:43:29 They're one and the same, right? 683 00:43:29 --> 00:43:32 It's the same column space. 684 00:43:32 --> 00:43:37 All I'm taking is here this B thing that I computed, 685 00:43:37 --> 00:43:43.26 this B thing that I computed is a combination of B and A, 686 00:43:43.26 --> 00:43:48 and A was little A, so I'm always working here with 687 00:43:48 --> 00:43:50.77 this in the same space. 688 00:43:50.77 --> 00:43:56 I'm just like getting ninety-degree angles in there. 689 00:43:56 --> 00:44:02.53 Where my original column space had a perfectly good basis, 690 00:44:02.53 --> 00:44:07 but it wasn't as good as this basis, because it wasn't 691 00:44:07 --> 00:44:08 orthonormal. 692 00:44:08 --> 00:44:13 Now this one is orthonormal, and I have a basis then that -- 693 00:44:13 --> 00:44:17.07 so now projections, all the calculations I would 694 00:44:17.07 --> 00:44:21 ever want to do are -- are a cinch with this orthonormal 695 00:44:21 --> 00:44:22 basis. 696 00:44:22 --> 00:44:23 One final point. 697 00:44:23 --> 00:44:27.77 One final point in this chapter. 698 00:44:27.77 --> 00:44:31 And it's -- just like elimination. 699 00:44:31 --> 00:44:36 We learned how to do elimination, we know all the 700 00:44:36 --> 00:44:38.96 steps, we can do it. 701 00:44:38.96 --> 00:44:45 But then I came back to it and said look at it as a matrix in 702 00:44:45 --> 00:44:52 matrix language and elimination gave me -- what was elimination 703 00:44:52 --> 00:44:55 in matrix language? 704 00:44:55 --> 00:44:57 I'll just put it up there. 705 00:44:57 --> 00:44:58 A was LU. 706 00:44:58 --> 00:45:01 That was matrix, that was elimination. 707 00:45:01 --> 00:45:05 Now, I want to do the same for Graham-Schmidt. 708 00:45:05 --> 00:45:11.13 Everybody who works in linear algebra isn't going to write out 709 00:45:11.13 --> 00:45:14 the columns are orthogonal, or orthonormal. 710 00:45:14 --> 00:45:19 And isn't going to write out these formulas. 711 00:45:19 --> 00:45:24.88 They're going to write out the connection between the matrix A 712 00:45:24.88 --> 00:45:26 and the matrix Q. 713 00:45:26 --> 00:45:30 And the two matrices have the same column space, 714 00:45:30 --> 00:45:35.58 but there's some -- some matrix is taking the -- and I'm going 715 00:45:35.58 --> 00:45:39.86 to call it R, so A equals QR is the magic 716 00:45:39.86 --> 00:45:41 formula here. 717 00:45:41 --> 00:45:46 It's the expression of Graham-Schmidt. 718 00:45:46 --> 00:45:51 And I'll -- let me just capture that. 719 00:45:51 --> 00:45:57 So that's the -- my final step then is A equal QR. 720 00:45:57 --> 00:46:02.09 Maybe I can squeeze it in here. 721 00:46:02.09 --> 00:46:08 So A has columns, let's say a1 and a2. 722 00:46:08 --> 00:46:12 Let me suppose n is two, just two vectors. 723 00:46:12 --> 00:46:13 OK. 724 00:46:13 --> 00:46:17 So that's some combination of q1 and q2. 725 00:46:17 --> 00:46:19 And times some matrix R. 726 00:46:19 --> 00:46:23.14 They have the same column space. 727 00:46:23.14 --> 00:46:29 This is just -- this matrix just includes in it whatever 728 00:46:29 --> 00:46:35.33 these numbers like three over three and one over square root 729 00:46:35.33 --> 00:46:40 of three and one over square root of two, 730 00:46:40 --> 00:46:43 probably that's what it's got. 731 00:46:43 --> 00:46:49 One over square root of three, one over square root of two, 732 00:46:49 --> 00:46:53 something there, but actually it's got a zero 733 00:46:53 --> 00:46:54 there. 734 00:46:54 --> 00:47:00 So the main point about this A equal QR is this R turns out to 735 00:47:00 --> 00:47:03 be upper triangular. 736 00:47:03 --> 00:47:08 It turns out that this zero is upper triangular. 737 00:47:08 --> 00:47:10 We could see why. 738 00:47:10 --> 00:47:16.05 Let me see, I can put in general formulas for what these 739 00:47:16.05 --> 00:47:16 are. 740 00:47:16 --> 00:47:23 This I think in here should be the inner product of a1 with q1. 741 00:47:23 --> 00:47:30 And this one should be the -- the inner product of a1 with 742 00:47:30 --> 00:47:31 q2. 743 00:47:31 --> 00:47:35 And that's what I believe is zero. 744 00:47:35 --> 00:47:42 This will be something here, and this will be something here 745 00:47:42 --> 00:47:49 with inner -- a1 transpose q2, sorry a2 transpose q1 and a2 746 00:47:49 --> 00:47:50 transpose q2. 747 00:47:50 --> 00:47:53 But why is that guy zero? 748 00:47:53 --> 00:47:57 Why is a1 q2 zero? 749 00:47:57 --> 00:48:03.39 That's the key to this being -- this R here being upper 750 00:48:03.39 --> 00:48:04 triangular. 751 00:48:04 --> 00:48:10 You know why a1q2 is zero, because a1 -- that was my -- 752 00:48:10 --> 00:48:13 this was really a and b here. 753 00:48:13 --> 00:48:16 This was really a and b. 754 00:48:16 --> 00:48:19 So this is a transpose q2. 755 00:48:19 --> 00:48:25 And the whole point of Graham-Schmidt was that we 756 00:48:25 --> 00:48:31 constructed these later q-s to be perpendicular to the earlier 757 00:48:31 --> 00:48:36 vectors, to the earlier -- all the earlier vectors. 758 00:48:36 --> 00:48:40.21 So that's why we get a triangular matrix. 759 00:48:40.21 --> 00:48:44 The -- result is extremely satisfactory. 760 00:48:44 --> 00:48:49 That if I have a matrix with independent columns, 761 00:48:49 --> 00:48:55 the Graham-Schmidt produces a matrix with orthonormal columns, 762 00:48:55 --> 00:49:01 and the connection between those is a triangular matrix. 763 00:49:01 --> 00:49:05 That last point, that the connection is a 764 00:49:05 --> 00:49:09 triangular matrix, please look in the book, 765 00:49:09 --> 00:49:12 you have to see that one more time. 766 00:49:12 --> 00:49:13 OK. 767 00:49:13 --> 00:49:16 Thanks, that's great.