WEBVTT 00:00:07.270 --> 00:00:12.540 OK, here's the last lecture in the chapter on orthogonality. 00:00:12.540 --> 00:00:16.900 So we met orthogonal vectors, two vectors, 00:00:16.900 --> 00:00:22.530 we met orthogonal subspaces, like the row space and null 00:00:22.530 --> 00:00:23.590 space. 00:00:23.590 --> 00:00:28.720 Now today we meet an orthogonal basis, 00:00:28.720 --> 00:00:31.370 and an orthogonal matrix. 00:00:31.370 --> 00:00:32.920 So we really -- 00:00:32.920 --> 00:00:36.170 this chapter cleans up orthogonality. 00:00:36.170 --> 00:00:38.930 And really I want -- 00:00:38.930 --> 00:00:43.880 I should use the word orthonormal. 00:00:43.880 --> 00:00:51.690 Orthogonal is -- so my vectors are q1,q2 up to qn -- 00:00:51.690 --> 00:00:56.180 I use the letter "q", here, to remind me, 00:00:56.180 --> 00:01:01.270 I'm talking about orthogonal things, not just any vectors, 00:01:01.270 --> 00:01:02.690 but orthogonal ones. 00:01:02.690 --> 00:01:04.230 So what does that mean? 00:01:04.230 --> 00:01:08.600 That means that every q is orthogonal to every other q. 00:01:08.600 --> 00:01:13.880 It's a natural idea, to have a basis that's 00:01:13.880 --> 00:01:17.060 headed off at ninety-degree angles, 00:01:17.060 --> 00:01:19.120 the inner products are all zero. 00:01:19.120 --> 00:01:26.260 Of course if q is -- certainly qi is not orthogonal to itself. 00:01:26.260 --> 00:01:29.200 But there we'll make the best choice again, 00:01:29.200 --> 00:01:31.050 make it a unit vector. 00:01:31.050 --> 00:01:35.610 Then qi transpose qi is one, for a unit vector. 00:01:35.610 --> 00:01:37.590 The length squared is one. 00:01:37.590 --> 00:01:41.430 And that's what I would use the word normal. 00:01:41.430 --> 00:01:48.479 So for this part, normalized, unit length for this part. 00:01:48.479 --> 00:01:48.979 OK. 00:01:48.979 --> 00:01:55.120 So first part of the lecture is how 00:01:55.120 --> 00:01:59.320 does having an orthonormal basis make things nice? 00:01:59.320 --> 00:02:00.330 It certainly does. 00:02:00.330 --> 00:02:02.490 It makes all the calculations better, 00:02:02.490 --> 00:02:05.150 a whole lot of numerical linear algebra 00:02:05.150 --> 00:02:10.520 is built around working with orthonormal vectors, 00:02:10.520 --> 00:02:12.230 because they never get out of hand, 00:02:12.230 --> 00:02:16.220 they never overflow or underflow. 00:02:16.220 --> 00:02:20.390 And I'll put them into a matrix Q, 00:02:20.390 --> 00:02:22.430 and then the second part of the lecture 00:02:22.430 --> 00:02:26.630 will be suppose my basis, my columns of A 00:02:26.630 --> 00:02:28.710 are not orthonormal. 00:02:28.710 --> 00:02:30.660 How do I make them so? 00:02:30.660 --> 00:02:35.110 And the two names associated with that simple idea 00:02:35.110 --> 00:02:36.840 are Graham and Schmidt. 00:02:36.840 --> 00:02:43.410 So the first part is we've got a basis like this. 00:02:43.410 --> 00:02:46.040 Let's put those into the columns of a matrix. 00:02:48.620 --> 00:02:53.990 So a matrix Q that has -- 00:02:53.990 --> 00:02:56.680 I'll put these orthonormal vectors, 00:02:56.680 --> 00:03:01.650 q1 will be the first column, qn will be the n-th column. 00:03:04.350 --> 00:03:09.610 And I want to say, I want to write this property, 00:03:09.610 --> 00:03:13.890 qi transpose qj being zero, I want 00:03:13.890 --> 00:03:17.640 to put that in a matrix form. 00:03:17.640 --> 00:03:23.390 And just the right thing is to look at Q transpose Q. 00:03:23.390 --> 00:03:27.520 So this chapter has been looking at A transpose A. 00:03:27.520 --> 00:03:30.470 So it's natural to look at Q transpose Q. 00:03:30.470 --> 00:03:33.490 And the beauty is it comes out perfectly. 00:03:33.490 --> 00:03:38.160 Because Q transpose has these vectors in its rows, 00:03:38.160 --> 00:03:45.510 the first row is q1 transpose, the nth row is qn transpose. 00:03:45.510 --> 00:03:49.070 So that's Q transpose. 00:03:49.070 --> 00:03:51.610 And now I want to multiply by Q. 00:03:51.610 --> 00:03:56.810 That has q1 along to qn in the columns. 00:03:56.810 --> 00:03:58.430 That's Q. 00:03:58.430 --> 00:03:59.410 And what do I get? 00:04:01.940 --> 00:04:06.240 You really -- this is the first simplest most basic fact, 00:04:06.240 --> 00:04:10.930 that how do orthonormal vectors, orthonormal columns 00:04:10.930 --> 00:04:17.660 in a matrix, what happens if I compute Q transpose Q? 00:04:17.660 --> 00:04:18.850 Do you see it? 00:04:18.850 --> 00:04:23.060 If I take the first row times the first column, 00:04:23.060 --> 00:04:25.170 what do I get? 00:04:25.170 --> 00:04:26.680 A one. 00:04:26.680 --> 00:04:29.570 If I take the first row times the second column, 00:04:29.570 --> 00:04:31.330 what do I get? 00:04:31.330 --> 00:04:32.280 Zero. 00:04:32.280 --> 00:04:34.170 That's the orthogonality. 00:04:34.170 --> 00:04:37.460 The first row times the last column is zero. 00:04:37.460 --> 00:04:40.630 And so I'm getting ones on the diagonal 00:04:40.630 --> 00:04:43.620 and I'm getting zeroes everywhere else. 00:04:43.620 --> 00:04:45.080 I'm getting the identity matrix. 00:04:47.880 --> 00:04:50.920 You see how that's -- it's just like the right calculation 00:04:50.920 --> 00:04:51.740 to do. 00:04:51.740 --> 00:04:55.540 If you have orthonormal columns, and the matrix 00:04:55.540 --> 00:04:59.420 doesn't have to be square here. 00:04:59.420 --> 00:05:02.290 We might have just two columns. 00:05:02.290 --> 00:05:05.670 And they might have four, lots of components. 00:05:05.670 --> 00:05:13.370 So but they're orthonormal, and when we do Q transpose times Q, 00:05:13.370 --> 00:05:17.110 that Q transpose times Q or A transpose A 00:05:17.110 --> 00:05:22.680 just asks for all those dot products. 00:05:22.680 --> 00:05:24.530 Rows times columns. 00:05:24.530 --> 00:05:29.390 And in this orthonormal case, we get the best possible answer, 00:05:29.390 --> 00:05:30.850 the identity. 00:05:30.850 --> 00:05:33.330 OK, so this is -- 00:05:33.330 --> 00:05:39.690 so I mean now we have a new bunch of important matrices. 00:05:39.690 --> 00:05:41.070 What have we seen previously? 00:05:41.070 --> 00:05:43.440 We've seen in the distant past we 00:05:43.440 --> 00:05:48.180 had triangular matrices, diagonal matrices, permutation 00:05:48.180 --> 00:05:51.760 matrices, that was early chapters, 00:05:51.760 --> 00:05:59.190 then we had row echelon forms, then in this chapter 00:05:59.190 --> 00:06:02.900 we've already seen projection matrices, 00:06:02.900 --> 00:06:08.830 and now we're seeing this new class of matrices 00:06:08.830 --> 00:06:11.490 with orthonormal columns. 00:06:11.490 --> 00:06:13.510 That's a very long expression. 00:06:13.510 --> 00:06:19.020 I sorry that I can't just call them orthogonal matrices. 00:06:19.020 --> 00:06:22.090 But that word orthogonal matrices -- 00:06:22.090 --> 00:06:25.440 or maybe I should be able to call it orthonormal matrices, 00:06:25.440 --> 00:06:28.090 why don't we call it orthonormal -- 00:06:28.090 --> 00:06:31.290 I mean that would be an absolutely perfect name. 00:06:31.290 --> 00:06:33.860 For Q, call it an orthonormal matrix 00:06:33.860 --> 00:06:36.370 because its columns are orthonormal. 00:06:36.370 --> 00:06:42.270 OK, but the convention is that we only use that name 00:06:42.270 --> 00:06:46.930 orthogonal matrix, we only use this -- 00:06:46.930 --> 00:06:49.670 this word orthogonal, we don't even 00:06:49.670 --> 00:06:51.950 say orthonormal for some unknown reason, 00:06:51.950 --> 00:06:54.530 matrix when it's square. 00:06:59.130 --> 00:07:04.740 So in the case when this is a square matrix, that's the case 00:07:04.740 --> 00:07:07.830 we call it an orthogonal matrix. 00:07:07.830 --> 00:07:12.290 And what's special about the case when it's square? 00:07:12.290 --> 00:07:19.390 When it's a square matrix, we've got its inverse, so -- 00:07:19.390 --> 00:07:33.850 so in the case if Q is square, then Q transpose Q equals I 00:07:33.850 --> 00:07:35.730 tells us -- 00:07:35.730 --> 00:07:37.820 let me write that underneath -- 00:07:37.820 --> 00:07:47.670 tells us that Q transpose is Q inverse. 00:07:47.670 --> 00:07:51.000 There we have the easy to remember 00:07:51.000 --> 00:07:57.440 property for a square matrix with orthonormal columns. 00:07:57.440 --> 00:08:01.960 That -- I need to write some examples down. 00:08:01.960 --> 00:08:04.180 Let's see. 00:08:04.180 --> 00:08:07.750 Some examples like if I take any -- so examples, 00:08:07.750 --> 00:08:08.720 let's do some examples. 00:08:12.100 --> 00:08:14.830 Any permutation matrix, let me take just 00:08:14.830 --> 00:08:17.360 some random permutation matrix. 00:08:17.360 --> 00:08:23.910 Permutation Q equals let's say oh, make it three by three, 00:08:23.910 --> 00:08:30.050 say zero, zero, one, one, zero, zero, zero, one, zero. 00:08:30.050 --> 00:08:30.550 OK. 00:08:36.470 --> 00:08:41.860 That certainly has unit vectors in its columns. 00:08:41.860 --> 00:08:45.940 Those vectors are certainly perpendicular to each other. 00:08:45.940 --> 00:08:48.700 And if I -- and so that's it. 00:08:48.700 --> 00:08:50.440 That makes it a Q. 00:08:50.440 --> 00:08:56.360 And -- if I took its transpose, if I multiplied by Q transpose, 00:08:56.360 --> 00:08:59.640 shall I do that -- and let me stick in Q transpose 00:08:59.640 --> 00:09:00.400 here. 00:09:00.400 --> 00:09:02.990 Just to do that multiplication once more, 00:09:02.990 --> 00:09:05.200 transpose it'll put the -- 00:09:05.200 --> 00:09:08.720 make that into a column, make that into a column, 00:09:08.720 --> 00:09:10.920 make that into a column. 00:09:10.920 --> 00:09:14.200 And the transpose is also -- 00:09:14.200 --> 00:09:15.690 another Q. 00:09:15.690 --> 00:09:17.070 Another orthonormal matrix. 00:09:17.070 --> 00:09:22.110 And when I multiply that product I get I. OK, 00:09:22.110 --> 00:09:23.890 so there's an example. 00:09:23.890 --> 00:09:26.860 And actually there's a second example. 00:09:26.860 --> 00:09:29.420 But those are real easy examples, right, 00:09:29.420 --> 00:09:34.780 I mean to get orthogonal columns by just 00:09:34.780 --> 00:09:40.630 putting ones in different places is like too easy. 00:09:40.630 --> 00:09:43.030 So let me keep going with examples. 00:09:43.030 --> 00:09:46.170 So here's another simple example. 00:09:46.170 --> 00:09:51.170 Cos theta sine theta, there's a unit vector, 00:09:51.170 --> 00:09:53.920 oh, let me even take it, well, yeah. 00:09:53.920 --> 00:09:58.500 Cos theta sine theta and now the other way 00:09:58.500 --> 00:10:01.790 I want sine theta cos theta. 00:10:01.790 --> 00:10:05.700 But I want the inner product to be zero. 00:10:05.700 --> 00:10:09.150 And if I put a minus there, it'll do it. 00:10:09.150 --> 00:10:12.760 So that's -- unit vector, that's a unit vector. 00:10:12.760 --> 00:10:16.920 And if I take the dot product, I get minus plus zero. 00:10:19.470 --> 00:10:19.970 OK. 00:10:19.970 --> 00:10:27.230 For example Q equals say one, one, one, minus one, 00:10:27.230 --> 00:10:30.710 is that an orthogonal matrix? 00:10:33.280 --> 00:10:35.600 I've got orthogonal columns there, 00:10:35.600 --> 00:10:37.920 but it's not quite an orthogonal matrix. 00:10:37.920 --> 00:10:43.000 How shall I fix it to be an orthogonal matrix? 00:10:43.000 --> 00:10:45.780 Well, what's the length of those column vectors, 00:10:45.780 --> 00:10:51.540 the dot product with themselves is -- right now it's two, 00:10:51.540 --> 00:10:52.760 right, the -- 00:10:52.760 --> 00:10:55.040 the length squared. 00:10:55.040 --> 00:10:57.710 The length squared would be one plus one would be two, 00:10:57.710 --> 00:10:59.340 the length would be square root of two, 00:10:59.340 --> 00:11:02.790 so I better divide by square root of two. 00:11:02.790 --> 00:11:03.750 OK. 00:11:03.750 --> 00:11:08.080 So there's a -- there now I have got an orthogonal matrix, 00:11:08.080 --> 00:11:13.350 in fact, it's this one -- when theta is pi over four. 00:11:13.350 --> 00:11:16.210 The cosines and well almost, I guess 00:11:16.210 --> 00:11:19.610 the minus sine is down there, so maybe, I 00:11:19.610 --> 00:11:23.650 don't know, maybe minus pi over four or something. 00:11:23.650 --> 00:11:24.150 OK. 00:11:26.760 --> 00:11:28.760 Let me do one final example, just 00:11:28.760 --> 00:11:32.000 to show that you can get bigger ones. 00:11:32.000 --> 00:11:38.270 Q equals let me take that matrix up in the corner 00:11:38.270 --> 00:11:41.830 and I'll sort of repeat that pattern, 00:11:41.830 --> 00:11:47.010 repeat it again, and then minus it down here. 00:11:51.110 --> 00:11:57.800 That's one of the world's favorite orthogonal matrices. 00:11:57.800 --> 00:12:00.090 I hope I got it right, is -- 00:12:00.090 --> 00:12:02.500 can you see whether -- 00:12:02.500 --> 00:12:05.890 if I take the inner product of one column with another one, 00:12:05.890 --> 00:12:07.990 let's see, if I take the inner product 00:12:07.990 --> 00:12:11.070 of that column with that I have two minuses and two pluses, 00:12:11.070 --> 00:12:11.960 that's good. 00:12:11.960 --> 00:12:14.280 When I take the inner product of that with that 00:12:14.280 --> 00:12:17.390 I have a plus and a minus, a minus and a plus. 00:12:17.390 --> 00:12:17.890 Good. 00:12:17.890 --> 00:12:19.530 I think it all works out. 00:12:19.530 --> 00:12:22.210 And what do I have to divide by now? 00:12:22.210 --> 00:12:24.500 To make those into unit vectors. 00:12:27.320 --> 00:12:34.520 Right now the vector one, one, one, one has length two. 00:12:34.520 --> 00:12:35.660 Square root of four. 00:12:35.660 --> 00:12:39.680 So I have to divide by two to make it unit vector, 00:12:39.680 --> 00:12:41.190 so there's another. 00:12:41.190 --> 00:12:45.550 That's my entire array of simple examples. 00:12:49.190 --> 00:12:56.610 This construction is named after a guy called Adhemar and we 00:12:56.610 --> 00:13:02.960 know how to do it for two, four, sixteen, 00:13:02.960 --> 00:13:10.670 sixty-four and so on, but we -- nobody knows exactly which size 00:13:10.670 --> 00:13:12.970 matrices have -- 00:13:12.970 --> 00:13:18.250 which size -- which sizes allow orthogonal matrices of ones 00:13:18.250 --> 00:13:19.300 and minus ones. 00:13:19.300 --> 00:13:24.450 So Adhemar matrix is an orthogonal matrix that's got 00:13:24.450 --> 00:13:30.070 ones and minus ones, and a lot of ones -- some we know, 00:13:30.070 --> 00:13:34.130 some other sizes, there couldn't be a five by five I think. 00:13:34.130 --> 00:13:35.970 But there are some sizes that nobody 00:13:35.970 --> 00:13:42.220 yet knows whether there could be or can't be a matrix like that. 00:13:42.220 --> 00:13:43.160 OK. 00:13:43.160 --> 00:13:47.860 You see those orthogonal matrices. 00:13:47.860 --> 00:13:54.910 Now let me ask what -- why is it good to have orthogonal 00:13:54.910 --> 00:13:56.050 matrices? 00:13:56.050 --> 00:13:59.830 What calculation is made easy? 00:13:59.830 --> 00:14:02.040 If I have an orthogonal matrix. 00:14:02.040 --> 00:14:06.660 And -- let me remember that the matrix could be rectangular. 00:14:06.660 --> 00:14:07.880 Shall I put down -- 00:14:07.880 --> 00:14:10.680 I better put a rectangular example down. 00:14:10.680 --> 00:14:13.270 So the -- these were all square examples. 00:14:13.270 --> 00:14:14.910 Can I put down just -- 00:14:14.910 --> 00:14:18.190 a rectangular one just to be sure 00:14:18.190 --> 00:14:22.080 that we realize that this is possible. 00:14:22.080 --> 00:14:23.930 let's help me out. 00:14:23.930 --> 00:14:33.870 Let's see, if I put like a one, two, two and a minus two, 00:14:33.870 --> 00:14:35.630 minus one, two. 00:14:40.530 --> 00:14:45.340 That's -- a matrix -- oh its columns aren't normalized yet. 00:14:45.340 --> 00:14:47.950 I always have to remember to do that. 00:14:47.950 --> 00:14:50.590 I always do that last because it's easy to do. 00:14:50.590 --> 00:14:53.460 What's the length of those columns? 00:14:53.460 --> 00:14:56.420 So if I wanted them -- if I wanted them to be length one, 00:14:56.420 --> 00:14:59.840 I should divide by their length, which is -- 00:14:59.840 --> 00:15:03.440 so I'd look at one squared plus two squared plus two squared, 00:15:03.440 --> 00:15:06.420 that's one and four and four is nine, 00:15:06.420 --> 00:15:11.420 so I take the square root and I need to divide by three. 00:15:11.420 --> 00:15:11.920 OK. 00:15:11.920 --> 00:15:14.780 So there is -- 00:15:14.780 --> 00:15:21.320 well, without that, I've got one orthonormal vector. 00:15:21.320 --> 00:15:24.060 I mean just one unit vector. 00:15:24.060 --> 00:15:26.130 Now put that guy in. 00:15:26.130 --> 00:15:29.230 Now I have a basis for the column 00:15:29.230 --> 00:15:34.980 space for a two-dimensional space, an orthonormal basis, 00:15:34.980 --> 00:15:35.500 right? 00:15:35.500 --> 00:15:38.140 These two columns are orthonormal, 00:15:38.140 --> 00:15:40.660 they would be an orthonormal basis 00:15:40.660 --> 00:15:45.190 for this two-dimensional space that they span. 00:15:45.190 --> 00:15:48.700 Orthonormal vectors by the way have got to be independent. 00:15:48.700 --> 00:15:52.910 It's easy to show that orthonormal vectors 00:15:52.910 --> 00:15:55.640 since they're headed off all at ninety degrees 00:15:55.640 --> 00:15:58.630 there's no combination that gives zero. 00:15:58.630 --> 00:16:06.970 Now if I wanted to create now a third one, 00:16:06.970 --> 00:16:13.220 I could either just put in some third vector that was 00:16:13.220 --> 00:16:18.800 independent and go to this Graham-Schmidt calculation that 00:16:18.800 --> 00:16:22.860 I'm going to explain, or I could be inspired and say look, 00:16:22.860 --> 00:16:26.990 that -- with that pattern, why not put a one in there, 00:16:26.990 --> 00:16:29.650 and a two in there, and a two in there, 00:16:29.650 --> 00:16:33.260 and try to fix up the signs so that they worked. 00:16:36.530 --> 00:16:37.070 Hmm. 00:16:37.070 --> 00:16:41.250 I don't know if I've done this too brilliantly. 00:16:41.250 --> 00:16:43.230 Let's see, what signs, that's minus, 00:16:43.230 --> 00:16:49.270 maybe I'd make a minus sign there, how would that be? 00:16:49.270 --> 00:16:52.990 Yeah, maybe that works. 00:16:52.990 --> 00:17:00.170 I think that those three columns are orthonormal and they -- 00:17:00.170 --> 00:17:03.720 the beauty of this -- this is the last example I'll probably 00:17:03.720 --> 00:17:08.089 find where there's no square root, the -- 00:17:08.089 --> 00:17:11.250 the punishing thing in Graham-Schmidt, 00:17:11.250 --> 00:17:14.720 maybe we better know that in advance, 00:17:14.720 --> 00:17:19.950 is that because I want these vectors to be unit vectors, 00:17:19.950 --> 00:17:21.900 I'm always running into square roots. 00:17:21.900 --> 00:17:24.270 I'm always dividing by lengths. 00:17:24.270 --> 00:17:26.109 And those lengths are square roots. 00:17:26.109 --> 00:17:29.900 So you'll see as soon as I do a Graham-Schmidt example, 00:17:29.900 --> 00:17:32.190 square roots are going to show up. 00:17:32.190 --> 00:17:34.960 But here are some examples where we did it 00:17:34.960 --> 00:17:36.850 without any square root. 00:17:36.850 --> 00:17:38.900 OK. 00:17:38.900 --> 00:17:42.110 So -- so great. 00:17:42.110 --> 00:17:50.760 Now next question is what's the good of having a Q? 00:17:50.760 --> 00:17:52.860 What formulas become easier? 00:17:52.860 --> 00:17:57.540 Suppose I want to project, so suppose Q -- 00:17:57.540 --> 00:18:03.010 suppose Q has orthonormal columns. 00:18:03.010 --> 00:18:05.150 I'm using the letter Q to mean this, 00:18:05.150 --> 00:18:07.130 I'll write it this one more time, 00:18:07.130 --> 00:18:12.120 but I always mean when I write a Q, 00:18:12.120 --> 00:18:14.910 I always mean that it has orthonormal columns. 00:18:14.910 --> 00:18:25.910 So suppose I want to project onto its column space. 00:18:31.590 --> 00:18:33.170 So what's the projection matrix? 00:18:36.480 --> 00:18:40.760 What's the projection matrix is I project onto a column space? 00:18:40.760 --> 00:18:46.310 OK, that gives me a chance to review the projection section, 00:18:46.310 --> 00:18:50.770 including that big formula, which used to be -- 00:18:50.770 --> 00:18:53.580 those four As in a row, but now it's 00:18:53.580 --> 00:18:57.350 got Qs, because I'm projecting onto the column space of Q, 00:18:57.350 --> 00:18:58.780 so do you remember what it was? 00:18:58.780 --> 00:19:05.700 It's Q Q transpose Q inverse Q transpose. 00:19:08.500 --> 00:19:12.390 That's my four Qs in a row. 00:19:12.390 --> 00:19:13.690 But what's good here? 00:19:16.490 --> 00:19:21.040 What -- what makes this formula nice if I'm projecting onto 00:19:21.040 --> 00:19:24.660 a column space when I have orthonormal basis for that 00:19:24.660 --> 00:19:25.440 space? 00:19:25.440 --> 00:19:29.110 What makes it nice is this is the identity. 00:19:29.110 --> 00:19:31.360 I don't have to do any inversion. 00:19:31.360 --> 00:19:33.190 I just get Q Q transpose. 00:19:40.170 --> 00:19:44.250 So Q Q transpose is a projection matrix. 00:19:44.250 --> 00:19:45.590 Oh, I can't help -- 00:19:45.590 --> 00:19:47.970 I can't resist just checking the properties, 00:19:47.970 --> 00:19:52.640 what are the properties of a projection matrix? 00:19:52.640 --> 00:19:57.280 There are two properties to know for any projection matrix. 00:19:57.280 --> 00:20:00.560 And I'm saying that this is the right projection 00:20:00.560 --> 00:20:04.500 matrix when we've got this orthonormal basis 00:20:04.500 --> 00:20:06.980 in the columns. 00:20:06.980 --> 00:20:07.540 OK. 00:20:07.540 --> 00:20:10.650 So there's the projection matrix. 00:20:10.650 --> 00:20:13.670 Suppose the matrix is square. 00:20:13.670 --> 00:20:17.090 First just tell me first this extreme case. 00:20:17.090 --> 00:20:22.190 If my matrix is square and it's got these orthonormal columns, 00:20:22.190 --> 00:20:26.170 then what's the column space? 00:20:26.170 --> 00:20:31.270 If I have a square matrix and I have independent columns, 00:20:31.270 --> 00:20:34.950 and even orthonormal columns, then the column space 00:20:34.950 --> 00:20:37.170 is the whole space, right? 00:20:37.170 --> 00:20:41.250 And what's the projection matrix onto the whole space? 00:20:41.250 --> 00:20:43.790 The identity matrix. 00:20:43.790 --> 00:20:45.360 If I'm projecting in the whole space, 00:20:45.360 --> 00:20:49.690 every vector B is right where it's supposed to be 00:20:49.690 --> 00:20:52.810 and I don't have to move it by projection. 00:20:52.810 --> 00:20:57.800 So this would be -- 00:20:57.800 --> 00:21:02.390 I'll put in parentheses this is I if Q is square. 00:21:07.890 --> 00:21:10.360 Well that we said that already. 00:21:10.360 --> 00:21:14.990 If Q is square, that's the case where Q transpose is Q inverse, 00:21:14.990 --> 00:21:18.140 we can put it on the right, we can put it on the left, 00:21:18.140 --> 00:21:23.000 we always get the identity matrix, if it's square. 00:21:23.000 --> 00:21:29.070 But if it's not a square matrix then it's not -- 00:21:29.070 --> 00:21:32.510 we don't get the identity matrix. 00:21:32.510 --> 00:21:37.710 We have Q Q transpose, and just again 00:21:37.710 --> 00:21:41.050 what are those two properties of a projection matrix? 00:21:41.050 --> 00:21:44.400 First of all, it's symmetric. 00:21:44.400 --> 00:21:48.870 OK, no problem, that's certainly a symmetric So what's 00:21:48.870 --> 00:21:50.860 that second property of a projection? 00:21:50.860 --> 00:21:51.360 matrix. 00:21:51.360 --> 00:21:55.010 That if you project and project again you don't move the second 00:21:55.010 --> 00:21:55.760 time. 00:21:55.760 --> 00:21:58.220 So the other property of a projection matrix 00:21:58.220 --> 00:22:04.070 should be that Q Q transpose twice 00:22:04.070 --> 00:22:09.320 should be the same as Q Q transpose once. 00:22:09.320 --> 00:22:11.550 That's projection matrices. 00:22:11.550 --> 00:22:14.170 And that property better fall out 00:22:14.170 --> 00:22:18.540 right away because from the fact we 00:22:18.540 --> 00:22:24.060 know about orthonormal matrices, Q transpose Q is I. OK, 00:22:24.060 --> 00:22:25.010 you see it. 00:22:25.010 --> 00:22:30.320 In the middle here is sitting Q Q t- Q transpose Q, sorry, 00:22:30.320 --> 00:22:34.030 that's what I meant to say, Q transpose Q is I. 00:22:34.030 --> 00:22:37.000 So that's sitting right in the middle, that cancels out, 00:22:37.000 --> 00:22:40.760 to give the identity, we're left with one Q Q transpose, 00:22:40.760 --> 00:22:43.480 and we're all set. 00:22:43.480 --> 00:22:44.180 OK. 00:22:44.180 --> 00:22:48.840 So this is the projection matrix -- 00:22:48.840 --> 00:22:54.670 all the equation -- all the messy equations of this chapter 00:22:54.670 --> 00:22:59.370 become trivial when our matrix -- 00:22:59.370 --> 00:23:02.540 when we have this orthonormal basis. 00:23:02.540 --> 00:23:04.630 I mean what do I mean by all the equations? 00:23:04.630 --> 00:23:06.280 Well, the most important equation 00:23:06.280 --> 00:23:10.680 was the normal equation, do you remember old A transpose 00:23:10.680 --> 00:23:15.620 A x hat equals A transpose b? 00:23:15.620 --> 00:23:22.200 But now -- now A is Q. 00:23:22.200 --> 00:23:28.140 Now I'm thinking I have Q transpose Q X hat 00:23:28.140 --> 00:23:32.000 equals Q transpose b. 00:23:32.000 --> 00:23:33.280 And what's good about that? 00:23:37.220 --> 00:23:42.950 What's good is that matrix on the left side is the identity. 00:23:42.950 --> 00:23:46.110 The matrix on the left is the identity, Q transpose Q, 00:23:46.110 --> 00:23:49.510 normally it isn't, normally it's that matrix of inner products 00:23:49.510 --> 00:23:53.870 and you've to compute all those dopey inner products and -- 00:23:53.870 --> 00:23:56.030 and -- and solve the system. 00:23:56.030 --> 00:23:59.950 Here the inner products are all one or zero. 00:23:59.950 --> 00:24:01.540 This is the identity matrix. 00:24:01.540 --> 00:24:03.280 It's gone. 00:24:03.280 --> 00:24:05.910 And there's the answer. 00:24:05.910 --> 00:24:09.140 There's no inversion involved. 00:24:09.140 --> 00:24:15.730 Each component of x is a Q times b. 00:24:15.730 --> 00:24:21.240 What that equation is saying is that the i-th component is 00:24:21.240 --> 00:24:26.420 the i-th basis vector times b. 00:24:26.420 --> 00:24:34.760 That's -- probably the most important formula in some major 00:24:34.760 --> 00:24:40.720 parts of mathematics, that if we have orthonormal basis, 00:24:40.720 --> 00:24:47.660 then the component in the -- in the i-th, along the i-th -- 00:24:47.660 --> 00:24:54.630 the projection on the i-th basis vector is just qi transpose b. 00:24:54.630 --> 00:25:00.580 That number x that we look for is just a dot product. 00:25:00.580 --> 00:25:01.080 OK. 00:25:04.050 --> 00:25:10.040 OK, so I'm ready now for the sort of like second half 00:25:10.040 --> 00:25:11.530 of the lecture. 00:25:11.530 --> 00:25:16.320 Where we don't start with an orthogonal matrix, 00:25:16.320 --> 00:25:18.910 orthonormal vectors. 00:25:18.910 --> 00:25:21.610 We just start with independent vectors 00:25:21.610 --> 00:25:25.370 and we want to make them orthonormal. 00:25:25.370 --> 00:25:27.860 So I'm going to -- can I do that now? 00:25:27.860 --> 00:25:29.940 Now here comes Graham-Schmidt. 00:25:29.940 --> 00:25:31.490 So -- Graham-Schmidt. 00:25:39.460 --> 00:25:43.990 So this is a calculation, I won't say -- 00:25:43.990 --> 00:25:53.190 I can't quite say it's like elimination, because it's 00:25:53.190 --> 00:25:56.630 different, our goal isn't triangular anymore. 00:25:56.630 --> 00:26:00.780 With elimination our goal was make the matrix triangular. 00:26:00.780 --> 00:26:04.200 Now our goal is make the matrix orthogonal. 00:26:04.200 --> 00:26:07.580 Make those columns orthonormal. 00:26:07.580 --> 00:26:10.240 So let me start with two columns. 00:26:10.240 --> 00:26:13.000 So I start with vectors a and b. 00:26:16.960 --> 00:26:20.430 And they're just like -- here, let me draw them. 00:26:20.430 --> 00:26:22.780 Here's a. 00:26:22.780 --> 00:26:23.400 Here's b. 00:26:26.370 --> 00:26:27.580 For example. 00:26:27.580 --> 00:26:29.700 A isn't specially horizontal, wasn't 00:26:29.700 --> 00:26:34.150 meant to be, just a is one vector, b is another. 00:26:34.150 --> 00:26:38.040 I want to produce those two vectors, 00:26:38.040 --> 00:26:40.760 they might be in twelve-dimensional space, 00:26:40.760 --> 00:26:43.940 or they might be in two-dimensional space. 00:26:43.940 --> 00:26:46.300 They're independent, anyway. 00:26:46.300 --> 00:26:49.500 So I better be sure I say that. 00:26:49.500 --> 00:26:51.330 I start with independent vectors. 00:26:54.860 --> 00:26:58.500 And I want to produce out of that q 1 and q2, 00:26:58.500 --> 00:27:00.970 I want to produce orthonormal vectors. 00:27:03.880 --> 00:27:09.730 And Graham and Schmidt tell me how. 00:27:09.730 --> 00:27:10.260 OK. 00:27:10.260 --> 00:27:14.330 Well, actually you could tell me how, we don't need -- frankly, 00:27:14.330 --> 00:27:17.800 I don't know -- there's only one idea here, 00:27:17.800 --> 00:27:24.850 if Graham had the idea, I don't know what Schmidt did. 00:27:24.850 --> 00:27:28.140 But OK. 00:27:28.140 --> 00:27:29.470 So you'll see it. 00:27:29.470 --> 00:27:31.900 We don't need either of them, actually. 00:27:31.900 --> 00:27:33.630 OK, so what I going to do. 00:27:33.630 --> 00:27:36.630 I'll take that -- this first guy. 00:27:36.630 --> 00:27:37.970 OK. 00:27:37.970 --> 00:27:39.790 Well, he's fine. 00:27:43.480 --> 00:27:46.620 That direction is fine except -- 00:27:46.620 --> 00:27:50.300 yeah, I'll say OK, I'll settle for that direction. 00:27:50.300 --> 00:27:51.410 So I'm going to -- 00:27:51.410 --> 00:27:53.630 I'm going to get, so what I going to -- 00:27:53.630 --> 00:27:59.370 my goal is I'm going to get orthogonal vectors 00:27:59.370 --> 00:28:02.780 and I'll call those capital A and B. 00:28:02.780 --> 00:28:07.590 So that's the key step is to get from any two vectors 00:28:07.590 --> 00:28:09.430 to two orthogonal vectors. 00:28:09.430 --> 00:28:14.530 And then at the end, no problem, I'll get orthonormal vectors, 00:28:14.530 --> 00:28:20.630 how will -- what will those will be my qs, q1 and q2, 00:28:20.630 --> 00:28:21.450 and what will they 00:28:21.450 --> 00:28:21.950 be? 00:28:25.980 --> 00:28:30.270 Once I've got A and B orthogonal, well, look, 00:28:30.270 --> 00:28:35.390 it's no big deal -- maybe that's what Schmidt did, he, 00:28:35.390 --> 00:28:38.840 brilliant Schmidt, thought OK, divide by the length, 00:28:38.840 --> 00:28:40.310 all right. 00:28:40.310 --> 00:28:42.820 That's Schmidt's contribution. 00:28:45.640 --> 00:28:46.140 OK. 00:28:51.250 --> 00:28:55.620 But Graham had a little more thinking to do, right? 00:28:55.620 --> 00:28:58.780 We haven't done Graham's part. 00:28:58.780 --> 00:29:03.050 This part except OK, I'm happy with A, 00:29:03.050 --> 00:29:07.170 A can be A. That first direction is fine. 00:29:07.170 --> 00:29:09.620 Why should -- no complaint about that. 00:29:09.620 --> 00:29:13.620 The trouble is the second direction is not fine. 00:29:13.620 --> 00:29:18.170 Because it's not orthogonal to the first. 00:29:18.170 --> 00:29:23.710 I'm looking for a vector that's -- starts with B, 00:29:23.710 --> 00:29:29.550 but makes it orthogonal to A. 00:29:29.550 --> 00:29:30.980 What's the vector? 00:29:30.980 --> 00:29:32.870 How do I do that? 00:29:32.870 --> 00:29:35.960 How do I produce from this vector 00:29:35.960 --> 00:29:41.820 a piece that's orthogonal to this one? 00:29:41.820 --> 00:29:44.900 And the -- remember these vectors might be in two 00:29:44.900 --> 00:29:48.570 dimensions or they might be in twelve dimensions. 00:29:48.570 --> 00:29:51.210 I'm just looking for the idea. 00:29:51.210 --> 00:29:53.840 So what's the idea? 00:29:53.840 --> 00:29:57.110 Where did we have orthogonal -- 00:29:57.110 --> 00:30:01.440 a vector showing up that was orthogonal to this guy? 00:30:01.440 --> 00:30:03.990 Well, that was the first basic calculation 00:30:03.990 --> 00:30:05.780 of the whole chapter. 00:30:05.780 --> 00:30:10.850 We -- we did a projection and the projection gave us this 00:30:10.850 --> 00:30:15.890 part, which was the part in the A direction. 00:30:15.890 --> 00:30:19.500 Now, the part we want is the other part, the e part. 00:30:19.500 --> 00:30:21.050 This part. 00:30:21.050 --> 00:30:23.790 This is going to be our -- 00:30:23.790 --> 00:30:25.450 that guy is that guy. 00:30:25.450 --> 00:30:31.110 This is our vector B. That gives us that ninety-degree angle. 00:30:31.110 --> 00:30:33.240 So B is you could say -- 00:30:33.240 --> 00:30:35.730 B is really what we previously called 00:30:35.730 --> 00:30:37.140 e. 00:30:37.140 --> 00:30:41.060 The error vector. 00:30:41.060 --> 00:30:42.950 And what is it? 00:30:42.950 --> 00:30:45.950 I mean what do I -- what is B here? 00:30:45.950 --> 00:30:47.690 A is A, no problem. 00:30:47.690 --> 00:30:52.410 B is -- 00:30:52.410 --> 00:30:54.120 OK, what's this error piece? 00:30:54.120 --> 00:30:56.530 Do you remember? 00:30:56.530 --> 00:31:04.150 It's I start with the original B and I take away what? 00:31:04.150 --> 00:31:08.670 Its projection, this P. This -- the vector B, 00:31:08.670 --> 00:31:12.160 this error vector, is the original vector removing 00:31:12.160 --> 00:31:12.890 the projection. 00:31:12.890 --> 00:31:15.590 So instead of wanting the projection, 00:31:15.590 --> 00:31:20.260 now that's what I want to throw away. 00:31:20.260 --> 00:31:22.780 I want to get the part that's perpendicular. 00:31:22.780 --> 00:31:24.720 And there will be a perpendicular part, 00:31:24.720 --> 00:31:25.450 it won't be zero. 00:31:28.890 --> 00:31:32.880 Because these vectors were independent, so B -- 00:31:32.880 --> 00:31:34.880 if B was along the direction of A, 00:31:34.880 --> 00:31:37.830 then if the original B and A were in the same direction, 00:31:37.830 --> 00:31:38.760 then I'm -- 00:31:38.760 --> 00:31:40.500 I've only got one direction. 00:31:40.500 --> 00:31:43.440 But here they're in two independent directions 00:31:43.440 --> 00:31:46.190 and all I'm doing is getting that guy. 00:31:46.190 --> 00:31:49.550 So what's its formula? 00:31:49.550 --> 00:31:53.510 What's the formula for that if -- 00:31:53.510 --> 00:31:55.540 I want to subtract the projection, 00:31:55.540 --> 00:31:57.450 so do you remember the projection? 00:31:57.450 --> 00:32:03.490 It's some multiple of A and what's that multiple? 00:32:03.490 --> 00:32:07.030 It's -- it's that thing we called x in the very very first 00:32:07.030 --> 00:32:10.150 lecture on this chapter. 00:32:10.150 --> 00:32:16.540 There's an A transpose A in the bottom 00:32:16.540 --> 00:32:23.990 and there's an A transpose B, isn't that it? 00:32:29.780 --> 00:32:32.240 I think that's Graham's formula. 00:32:32.240 --> 00:32:33.300 Or Graham-Schmidt. 00:32:33.300 --> 00:32:34.600 No, that's Graham. 00:32:34.600 --> 00:32:38.120 Schmidt has got to divide the whole thing by the length, 00:32:38.120 --> 00:32:39.660 so he -- 00:32:39.660 --> 00:32:43.280 his formula makes a mess which I'm not willing to write down. 00:32:43.280 --> 00:32:48.070 So let's just see that what I saying here? 00:32:48.070 --> 00:32:51.670 I'm saying that this vector is perpendicular to A. 00:32:51.670 --> 00:32:53.260 That these are orthogonal. 00:32:53.260 --> 00:32:56.710 A is perpendicular to B. 00:32:56.710 --> 00:32:58.490 Can you check that? 00:32:58.490 --> 00:33:01.430 How do you see that yes, of course, we -- 00:33:01.430 --> 00:33:04.160 our picture is telling us, yes, we did it right. 00:33:04.160 --> 00:33:08.720 How would I check that this matrix is perpendicular to A? 00:33:11.390 --> 00:33:16.740 I would multiply by A transpose and I better get zero, right? 00:33:16.740 --> 00:33:18.410 I should check that. 00:33:18.410 --> 00:33:22.930 A transpose B should come out zero. 00:33:22.930 --> 00:33:26.920 So this is A transpose times -- now what did we say B was? 00:33:26.920 --> 00:33:30.370 We start with the original B, and we take away 00:33:30.370 --> 00:33:38.090 this projection, and that should come out zero. 00:33:38.090 --> 00:33:42.950 Well, here we get an A transpose B minus -- 00:33:42.950 --> 00:33:46.050 and here is another A transpose B, and the -- 00:33:46.050 --> 00:33:50.120 and it's an A transpose A over A transpose A, a one, 00:33:50.120 --> 00:33:53.840 those cancel, and we do get zero. 00:33:53.840 --> 00:33:54.340 Right. 00:33:58.360 --> 00:34:05.940 Now I guess I can do numbers in there. 00:34:05.940 --> 00:34:09.420 But I have to take a third vector 00:34:09.420 --> 00:34:13.330 to be sure we've got this system down. 00:34:13.330 --> 00:34:20.250 So now I have to say if I have independent vectors A, B and C, 00:34:20.250 --> 00:34:25.820 I'm looking for orthogonal vectors A, B and capital C, 00:34:25.820 --> 00:34:29.170 and then of course the third guy will just 00:34:29.170 --> 00:34:32.969 be C over its length, the unit vector. 00:34:35.860 --> 00:34:39.980 So this is now the problem. 00:34:39.980 --> 00:34:42.929 I got B here. 00:34:42.929 --> 00:34:45.889 I got A very easily. 00:34:45.889 --> 00:34:52.610 And now -- if you see the idea, we could figure out a formula 00:34:52.610 --> 00:35:01.390 for C. So now that -- so this is like a typical homework quiz 00:35:01.390 --> 00:35:02.150 problem. 00:35:02.150 --> 00:35:07.260 I give you two vectors, you do this, I give you three vectors, 00:35:07.260 --> 00:35:10.460 and you have to make them orthonormal. 00:35:10.460 --> 00:35:13.520 So you do this again, the first vector's fine, 00:35:13.520 --> 00:35:17.160 the second vector is perpendicular to the first, 00:35:17.160 --> 00:35:19.720 and now I need a third vector that's 00:35:19.720 --> 00:35:23.240 perpendicular to the first one and the second one. 00:35:23.240 --> 00:35:25.560 Right? 00:35:25.560 --> 00:35:29.660 Tthis is the end of a -- the lecture is to find this guy. 00:35:29.660 --> 00:35:33.980 Find this vector -- this vector C, that's perpendicular we n- 00:35:33.980 --> 00:35:39.410 at this point we know A and B. 00:35:39.410 --> 00:35:44.540 But C, the little c that we're given, is off in some -- 00:35:44.540 --> 00:35:48.030 it's got to come out of the blackboard to be independent, 00:35:48.030 --> 00:35:52.440 so -- so can I sort of draw off -- off comes a c somewhere. 00:35:52.440 --> 00:35:54.240 I don't know, where I going to put the darn 00:35:54.240 --> 00:35:55.050 thing? 00:35:55.050 --> 00:35:59.450 Maybe I'll put it off, oh, I don't know, 00:35:59.450 --> 00:36:02.160 like that somehow, C, little c. 00:36:05.610 --> 00:36:08.930 And I already know that perpendicular direction, 00:36:08.930 --> 00:36:10.750 that one and that one. 00:36:10.750 --> 00:36:13.800 So now what's the idea? 00:36:13.800 --> 00:36:17.080 Give me the Graham-Schmidt formula for C. 00:36:17.080 --> 00:36:20.700 What is this C here? 00:36:20.700 --> 00:36:21.520 Equals what? 00:36:27.340 --> 00:36:28.160 What I going to do? 00:36:28.160 --> 00:36:31.300 I'll start with the given one. 00:36:31.300 --> 00:36:32.730 As before. 00:36:32.730 --> 00:36:33.280 Right? 00:36:33.280 --> 00:36:36.410 I start with the vector I'm given. 00:36:36.410 --> 00:36:38.400 And what do I do with it? 00:36:38.400 --> 00:36:42.060 I want to remove out of it, I want to subtract off, 00:36:42.060 --> 00:36:46.510 so I'll put a minus sign in, I want to subtract off 00:36:46.510 --> 00:36:53.010 its components in the A, capital A and capital B directions. 00:36:53.010 --> 00:36:55.850 I just want to get those out of there. 00:36:55.850 --> 00:36:57.290 Well, I know how to do that. 00:36:57.290 --> 00:36:58.710 I did it with B. 00:36:58.710 --> 00:37:02.410 So I'll just -- so let me take away -- 00:37:02.410 --> 00:37:03.440 what if I do this? 00:37:07.960 --> 00:37:08.850 What have I done? 00:37:12.060 --> 00:37:16.520 I've got little c and what have I subtracted from it? 00:37:16.520 --> 00:37:22.280 Its component, its projection if you like, in the A direction. 00:37:25.010 --> 00:37:30.820 And now I've got to subtract off its component B transpose 00:37:30.820 --> 00:37:35.200 C over B transpose B, that multiple of B, 00:37:35.200 --> 00:37:37.570 is its component in the B direction. 00:37:40.100 --> 00:37:47.480 And that gives me the vector capital C that if anything is 00:37:47.480 --> 00:37:48.160 -- 00:37:48.160 --> 00:37:54.700 if there's any justice, this C should be perpendicular to A 00:37:54.700 --> 00:37:58.750 and it should be perpendicular to B. 00:37:58.750 --> 00:38:02.660 And the only thing it's -- hasn't got is unit vector, 00:38:02.660 --> 00:38:05.830 so we divide by its length to get that too. 00:38:08.660 --> 00:38:10.390 OK. 00:38:10.390 --> 00:38:14.940 Let me do an example. 00:38:14.940 --> 00:38:16.410 Can I -- 00:38:16.410 --> 00:38:20.990 I'll make my life easy, I'll just take two vectors. 00:38:20.990 --> 00:38:23.880 So let me do a numerical example. 00:38:23.880 --> 00:38:26.480 If I'll give you two vectors, you 00:38:26.480 --> 00:38:31.220 give me back the Graham-Schmidt orthonormal basis, 00:38:31.220 --> 00:38:35.990 and we'll see how to express that in matrix form. 00:38:35.990 --> 00:38:36.530 OK. 00:38:36.530 --> 00:38:41.080 So let me give you the two vectors. 00:38:41.080 --> 00:38:46.420 So I'll take the vector A equals let's say one, one, one, 00:38:46.420 --> 00:38:47.910 why not? 00:38:47.910 --> 00:38:55.810 And B equals let's say one, zero, two, OK? 00:39:02.400 --> 00:39:05.300 I didn't want to cheat and make them orthogonal 00:39:05.300 --> 00:39:07.330 in the first place because then Graham-Schmidt 00:39:07.330 --> 00:39:08.430 wouldn't be needed. 00:39:08.430 --> 00:39:08.930 OK. 00:39:08.930 --> 00:39:10.760 So those are not orthogonal. 00:39:10.760 --> 00:39:12.330 So what is capital A? 00:39:12.330 --> 00:39:14.280 Well that's the same as big A. 00:39:14.280 --> 00:39:15.350 That was fine. 00:39:15.350 --> 00:39:18.590 What's B? 00:39:18.590 --> 00:39:21.600 So B is this b -- is the original B, 00:39:21.600 --> 00:39:29.680 and then I subtract off some multiple of the A. 00:39:29.680 --> 00:39:30.900 And what's the multiple? 00:39:33.810 --> 00:39:36.520 What goes in here? 00:39:36.520 --> 00:39:41.240 B -- here's the A -- this is the -- this is the little b, 00:39:41.240 --> 00:39:45.430 this is the big A, also the little a, and I want 00:39:45.430 --> 00:39:48.680 to multiply it by that right -- that right ratio, 00:39:48.680 --> 00:39:53.480 which has A transpose A, here's my ratio. 00:39:53.480 --> 00:39:57.100 I'm just doing this. 00:39:57.100 --> 00:40:00.610 So it's A transpose B, what is A transpose B, 00:40:00.610 --> 00:40:02.790 it looks like three. 00:40:02.790 --> 00:40:06.780 And what is A -- oh, my -- 00:40:06.780 --> 00:40:08.470 what's A transpose A? 00:40:08.470 --> 00:40:08.970 Three. 00:40:11.540 --> 00:40:12.690 I'm sorry. 00:40:12.690 --> 00:40:14.990 I didn't know that was going to happen. 00:40:14.990 --> 00:40:15.490 OK. 00:40:15.490 --> 00:40:16.440 But it happened. 00:40:16.440 --> 00:40:18.750 Why should we knock it? 00:40:18.750 --> 00:40:19.610 OK. 00:40:19.610 --> 00:40:21.430 So do you see it all right? 00:40:21.430 --> 00:40:25.040 That's A transpose B, there's A transpose A, that's 00:40:25.040 --> 00:40:28.640 the fraction, so I take this away, 00:40:28.640 --> 00:40:33.640 and I get one take away one is a zero, zero minus this one 00:40:33.640 --> 00:40:39.180 is a minus one, and two minus the one is a one. 00:40:39.180 --> 00:40:39.930 OK. 00:40:39.930 --> 00:40:42.330 And what's this vector that we finally found? 00:40:42.330 --> 00:40:47.020 This is B. 00:40:47.020 --> 00:40:48.580 And how do I know it's right? 00:40:51.930 --> 00:40:54.690 How do I know I've got a vector I want? 00:40:54.690 --> 00:40:57.700 I check that B is perpendicular to -- 00:40:57.700 --> 00:40:59.980 that A and B are perpendicular. 00:40:59.980 --> 00:41:02.280 That A is perpendicular to B. 00:41:02.280 --> 00:41:03.190 Just look at that. 00:41:03.190 --> 00:41:06.550 That one -- the dot product of that with that is zero. 00:41:06.550 --> 00:41:07.180 OK. 00:41:07.180 --> 00:41:10.850 So now what is my q1 and q2? 00:41:14.720 --> 00:41:17.700 Why don't I put them in a matrix? 00:41:17.700 --> 00:41:18.480 Of course. 00:41:18.480 --> 00:41:20.970 Since I'm always putting these -- so the Q, 00:41:20.970 --> 00:41:24.260 I'll put the q1 and the q2 in a matrix. 00:41:24.260 --> 00:41:25.720 And what are they? 00:41:29.270 --> 00:41:32.700 Now when I'm writing q-s I'm supposed 00:41:32.700 --> 00:41:34.200 to make things normalized. 00:41:34.200 --> 00:41:36.450 I'm supposed to make things unit vectors. 00:41:36.450 --> 00:41:39.740 So I'm going to take that A but I'm going to divide it 00:41:39.740 --> 00:41:41.650 by square root of three. 00:41:46.920 --> 00:41:48.780 And I'm going to take this B but I'm 00:41:48.780 --> 00:41:53.370 going to divide it by square root of two 00:41:53.370 --> 00:41:57.720 to make it a unit vector, and there is my matrix. 00:42:01.000 --> 00:42:05.430 That's my matrix with orthonormal columns coming from 00:42:05.430 --> 00:42:08.790 Graham-Schmidt and it sort of it -- 00:42:08.790 --> 00:42:14.740 it came from the original one, one, one, one, zero, two, 00:42:14.740 --> 00:42:15.240 right? 00:42:15.240 --> 00:42:16.620 That was my original guys. 00:42:20.880 --> 00:42:23.200 These were the two I started with. 00:42:23.200 --> 00:42:25.950 These are the two that I'm happy to end with. 00:42:25.950 --> 00:42:30.110 Because those are orthonormal. 00:42:30.110 --> 00:42:33.490 So that's what Graham-Schmidt did. 00:42:33.490 --> 00:42:38.050 It -- well, tell me about the column spaces of these 00:42:38.050 --> 00:42:40.260 matrices. 00:42:40.260 --> 00:42:44.180 How is the column space of Q related to the column space of 00:42:44.180 --> 00:42:44.900 A? 00:42:44.900 --> 00:42:47.150 So I'm always asking you things like this, 00:42:47.150 --> 00:42:49.970 and that makes you think, OK, the column space 00:42:49.970 --> 00:42:54.550 is all combinations of the columns, it's that plane, 00:42:54.550 --> 00:42:55.540 right? 00:42:55.540 --> 00:42:58.770 I've got two vectors in three-dimensional space, 00:42:58.770 --> 00:43:03.630 their column space is a plane, the column space of this matrix 00:43:03.630 --> 00:43:08.200 is a plane, what's the relation between the planes? 00:43:08.200 --> 00:43:09.740 Between the two column spaces? 00:43:12.750 --> 00:43:15.190 They're one and the same, right? 00:43:15.190 --> 00:43:17.760 It's the same column space. 00:43:17.760 --> 00:43:23.580 All I'm taking is here this B thing that I computed, 00:43:23.580 --> 00:43:30.120 this B thing that I computed is a combination of B and A, 00:43:30.120 --> 00:43:34.630 and A was little A, so I'm always working here 00:43:34.630 --> 00:43:36.920 with this in the same space. 00:43:36.920 --> 00:43:42.450 I'm just like getting ninety-degree angles in there. 00:43:42.450 --> 00:43:47.450 Where my original column space had a perfectly good basis, 00:43:47.450 --> 00:43:50.470 but it wasn't as good as this basis, 00:43:50.470 --> 00:43:53.610 because it wasn't orthonormal. 00:43:53.610 --> 00:43:59.560 Now this one is orthonormal, and I have a basis then that -- 00:43:59.560 --> 00:44:03.050 so now projections, all the calculations I would ever want 00:44:03.050 --> 00:44:09.570 to do are -- are a cinch with this orthonormal basis. 00:44:09.570 --> 00:44:12.710 One final point. 00:44:12.710 --> 00:44:14.480 One final point in this chapter. 00:44:17.200 --> 00:44:21.240 And it's -- just like elimination. 00:44:21.240 --> 00:44:23.530 We learned how to do elimination, 00:44:23.530 --> 00:44:26.160 we know all the steps, we can do it. 00:44:26.160 --> 00:44:34.720 But then I came back to it and said look at it as a matrix 00:44:34.720 --> 00:44:40.140 in matrix language and elimination gave me -- 00:44:40.140 --> 00:44:42.480 what was elimination in matrix language? 00:44:42.480 --> 00:44:44.000 I'll just put it up there. 00:44:44.000 --> 00:44:46.240 A was LU. 00:44:46.240 --> 00:44:49.720 That was matrix, that was elimination. 00:44:49.720 --> 00:44:53.180 Now, I want to do the same for Graham-Schmidt. 00:44:53.180 --> 00:44:56.200 Everybody who works in linear algebra 00:44:56.200 --> 00:44:58.530 isn't going to write out the columns 00:44:58.530 --> 00:45:01.220 are orthogonal, or orthonormal. 00:45:01.220 --> 00:45:04.530 And isn't going to write out these formulas. 00:45:04.530 --> 00:45:08.820 They're going to write out the connection between the matrix A 00:45:08.820 --> 00:45:11.320 and the matrix Q. 00:45:11.320 --> 00:45:14.230 And the two matrices have the same column space, 00:45:14.230 --> 00:45:17.720 but there's some -- some matrix is taking the -- 00:45:17.720 --> 00:45:25.110 and I'm going to call it R, so A equals QR is the magic formula 00:45:25.110 --> 00:45:25.610 here. 00:45:28.200 --> 00:45:30.260 It's the expression of Graham-Schmidt. 00:45:32.860 --> 00:45:38.430 And I'll -- let me just capture that. 00:45:38.430 --> 00:45:42.240 So that's the -- my final step then is A equal QR. 00:45:42.240 --> 00:45:44.310 Maybe I can squeeze it in here. 00:45:47.300 --> 00:45:50.960 So A has columns, let's say a1 and a2. 00:45:55.420 --> 00:45:59.150 Let me suppose n is two, just two vectors. 00:45:59.150 --> 00:46:00.290 OK. 00:46:00.290 --> 00:46:06.300 So that's some combination of q1 and q2. 00:46:06.300 --> 00:46:13.170 And times some matrix R. 00:46:13.170 --> 00:46:16.330 They have the same column space. 00:46:16.330 --> 00:46:20.420 This is just -- this matrix just includes in it whatever these 00:46:20.420 --> 00:46:23.430 numbers like three over three and one over square root 00:46:23.430 --> 00:46:25.360 of three and one over square root of two, 00:46:25.360 --> 00:46:28.400 probably that's what it's got. 00:46:28.400 --> 00:46:31.140 One over square root of three, one over square root of two, 00:46:31.140 --> 00:46:34.390 something there, but actually it's got a zero there. 00:46:37.690 --> 00:46:45.260 So the main point about this A equal QR is this R 00:46:45.260 --> 00:46:48.270 turns out to be upper triangular. 00:46:48.270 --> 00:46:50.750 It turns out that this zero is upper triangular. 00:46:53.510 --> 00:46:56.440 We could see why. 00:46:56.440 --> 00:47:00.640 Let me see, I can put in general formulas for what these 00:47:00.640 --> 00:47:05.190 This I think in here should be the inner product of a1 00:47:05.190 --> 00:47:05.740 with q1. are. 00:47:08.600 --> 00:47:12.160 And this one should be the -- 00:47:12.160 --> 00:47:16.230 the inner product of a1 with q2. 00:47:16.230 --> 00:47:18.970 And that's what I believe is zero. 00:47:21.710 --> 00:47:25.110 This will be something here, and this will be something here 00:47:25.110 --> 00:47:33.960 with inner -- a1 transpose q2, sorry a2 transpose q1 and a2 00:47:33.960 --> 00:47:35.060 transpose q2. 00:47:35.060 --> 00:47:37.130 But why is that guy zero? 00:47:40.100 --> 00:47:43.900 Why is a1 q2 zero? 00:47:43.900 --> 00:47:47.580 That's the key to this being -- this R here being upper 00:47:47.580 --> 00:47:49.110 triangular. 00:47:49.110 --> 00:47:55.100 You know why a1q2 is zero, because a1 -- 00:47:55.100 --> 00:47:57.310 that was my -- 00:47:57.310 --> 00:48:00.180 this was really a and b here. 00:48:00.180 --> 00:48:02.700 This was really a and b. 00:48:02.700 --> 00:48:05.780 So this is a transpose q2. 00:48:05.780 --> 00:48:09.380 And the whole point of Graham-Schmidt was that we 00:48:09.380 --> 00:48:14.820 constructed these later q-s to be perpendicular to the earlier 00:48:14.820 --> 00:48:19.030 vectors, to the earlier -- all the earlier vectors. 00:48:19.030 --> 00:48:21.090 So that's why we get a triangular matrix. 00:48:23.800 --> 00:48:29.750 The -- result is extremely satisfactory. 00:48:32.530 --> 00:48:37.450 That if I have a matrix with independent columns, 00:48:37.450 --> 00:48:40.540 the Graham-Schmidt produces a matrix 00:48:40.540 --> 00:48:44.780 with orthonormal columns, and the connection between those 00:48:44.780 --> 00:48:48.230 is a triangular matrix. 00:48:48.230 --> 00:48:51.200 That last point, that the connection is a triangular 00:48:51.200 --> 00:48:53.510 matrix, please look in the book, you 00:48:53.510 --> 00:48:56.390 have to see that one more time. 00:48:56.390 --> 00:48:56.930 OK. 00:48:56.930 --> 00:48:59.170 Thanks, that's great.