WEBVTT 00:00:01.161 --> 00:00:03.920 ANNOUNCER: The following content is provided under a Creative 00:00:03.920 --> 00:00:05.310 Commons license. 00:00:05.310 --> 00:00:07.520 Your support will help MIT Open Courseware 00:00:07.520 --> 00:00:11.610 continue to offer high quality educational resources for free. 00:00:11.610 --> 00:00:14.180 To make a donation or to view additional materials 00:00:14.180 --> 00:00:16.670 from hundreds of MIT courses, visit 00:00:16.670 --> 00:00:18.540 MITopencourseware@ocw.MIT.edu. 00:00:21.746 --> 00:00:23.160 PROFESSOR: So we're really moving 00:00:23.160 --> 00:00:27.550 along this review of the highlights of linear algebra. 00:00:27.550 --> 00:00:32.460 And today it's matrices Q. They get that name there. 00:00:32.460 --> 00:00:35.500 They have orthonormal columns. 00:00:35.500 --> 00:00:37.020 So that's what one looks like. 00:00:37.020 --> 00:00:40.920 And then the key fact, orthonormal columns 00:00:40.920 --> 00:00:47.100 translates directly into that simple fact that you just 00:00:47.100 --> 00:00:50.400 keep remembering every time you see Q transpose Q, 00:00:50.400 --> 00:00:52.390 you've got the identity matrix. 00:00:52.390 --> 00:00:53.820 Let's just see why. 00:00:53.820 --> 00:00:57.410 So Q transpose would be-- 00:00:57.410 --> 00:01:00.135 I'll take those columns and make them into rows. 00:01:02.970 --> 00:01:05.250 And then I multiply by Q with the columns. 00:01:08.760 --> 00:01:10.830 And what do I get? 00:01:10.830 --> 00:01:13.050 Well hopefully, I get the identity matrix. 00:01:19.170 --> 00:01:19.830 Why? 00:01:19.830 --> 00:01:24.510 Because-- oh yeah, the normal part tells me that the length 00:01:24.510 --> 00:01:25.800 of each vector-- 00:01:25.800 --> 00:01:29.190 that's the length squared Q transpose Q-- 00:01:29.190 --> 00:01:31.650 the length squared is one. 00:01:31.650 --> 00:01:35.190 So that gives me the one in the identity matrix 00:01:35.190 --> 00:01:36.900 all along the diagonal. 00:01:36.900 --> 00:01:41.040 And then Q transpose times a different Q is zero. 00:01:41.040 --> 00:01:43.050 That's the ortho part. 00:01:43.050 --> 00:01:45.510 So that gives me the zeros. 00:01:45.510 --> 00:01:48.290 So that's a simple identity, but it 00:01:48.290 --> 00:01:55.700 translates from a lot of words into a simple expression. 00:01:55.700 --> 00:02:02.130 Now does that mean that in the other order, Q, Q transpose, 00:02:02.130 --> 00:02:03.810 is that the identity? 00:02:03.810 --> 00:02:06.810 So that's a question to think about. 00:02:06.810 --> 00:02:10.680 Is Q, Q transpose equal the identity? 00:02:10.680 --> 00:02:15.420 Question, sometimes yes, sometimes no-- 00:02:15.420 --> 00:02:17.025 easy to tell which. 00:02:17.025 --> 00:02:28.830 If the answer is yes, yes when Q is square, the answer is yes. 00:02:28.830 --> 00:02:32.640 If Q is a square m a square matrix-- 00:02:32.640 --> 00:02:35.550 this is saying that a square matrix 00:02:35.550 --> 00:02:39.750 Q has that inverse on its left. 00:02:39.750 --> 00:02:42.450 But for square matrices, a left inverse, 00:02:42.450 --> 00:02:45.800 Q transpose is also a right inverse. 00:02:45.800 --> 00:02:47.490 So for a square matrix, if you have 00:02:47.490 --> 00:02:49.210 an inverse that works on one side, 00:02:49.210 --> 00:02:51.480 it will work on the other side. 00:02:51.480 --> 00:02:54.180 So the answer is yes in that case. 00:02:54.180 --> 00:03:00.720 And then in that case, that's the case when we call Q is-- 00:03:00.720 --> 00:03:01.740 well, we really-- 00:03:01.740 --> 00:03:03.330 I don't know what the right name would 00:03:03.330 --> 00:03:06.300 be, but here is the name everybody uses, 00:03:06.300 --> 00:03:07.695 an orthogonal matrix. 00:03:14.320 --> 00:03:17.900 And that's only in their square case, square. 00:03:22.820 --> 00:03:26.330 Q is an orthogonal matrix. 00:03:26.330 --> 00:03:29.060 Do you want to just see an example of how that works? 00:03:31.590 --> 00:03:34.670 So if Q is rectangular-- 00:03:34.670 --> 00:03:39.500 let me do a rectangular Q and a square Q. 00:03:39.500 --> 00:03:43.530 So I think there must be a board up there somewhere. 00:03:43.530 --> 00:03:44.030 Here. 00:03:47.000 --> 00:03:48.770 OK, square. 00:03:52.740 --> 00:03:54.730 All right. 00:03:54.730 --> 00:03:56.782 Good to see some orthogonal matrices, 00:03:56.782 --> 00:03:58.240 because my message is that they are 00:03:58.240 --> 00:04:01.340 really important in all kinds of applications. 00:04:01.340 --> 00:04:03.220 Let's start two by two. 00:04:03.220 --> 00:04:06.160 I can think of two different ways 00:04:06.160 --> 00:04:08.320 to get an orthogonal matrix. 00:04:08.320 --> 00:04:11.050 That's a two by two matrix. 00:04:11.050 --> 00:04:13.800 And one of them you will know immediately, 00:04:13.800 --> 00:04:16.970 cos theta sine theta. 00:04:16.970 --> 00:04:19.000 So that's a unit vector. 00:04:19.000 --> 00:04:19.990 It's normalized. 00:04:19.990 --> 00:04:22.360 Cos squared plus sine squared is one. 00:04:22.360 --> 00:04:25.250 And this guy has to be orthogonal to it. 00:04:25.250 --> 00:04:28.590 So I'll make that minus sine theta and cos theta. 00:04:32.020 --> 00:04:35.530 Those are both length one, they're orthogonal, 00:04:35.530 --> 00:04:40.960 then this is my Q. And the inverse of Q 00:04:40.960 --> 00:04:43.630 will be the transpose. 00:04:43.630 --> 00:04:46.900 The transpose would put the minus sign down here 00:04:46.900 --> 00:04:49.180 and would produce the inverse matrix. 00:04:49.180 --> 00:04:54.970 And what is that particular matrix represent? 00:04:54.970 --> 00:05:00.110 Geometrically, where do we see that matrix? 00:05:00.110 --> 00:05:01.880 It's a rotation, thank you. 00:05:01.880 --> 00:05:05.240 It's a rotation of the whole plane by theta. 00:05:05.240 --> 00:05:05.930 Yeah. 00:05:05.930 --> 00:05:10.190 So if I apply that to one, zero for example, 00:05:10.190 --> 00:05:14.600 I get the first column, which is cos theta sine theta. 00:05:14.600 --> 00:05:18.710 And that's just-- let me draw a picture. 00:05:18.710 --> 00:05:23.270 That vector one, zero has gotten rotated up to-- 00:05:23.270 --> 00:05:26.810 so there's the one, zero. 00:05:26.810 --> 00:05:30.260 And there, it got rotated through an angle theta. 00:05:30.260 --> 00:05:32.990 And similarly, zero, one will get 00:05:32.990 --> 00:05:36.650 rotated through an angle theta to there. 00:05:36.650 --> 00:05:40.200 So the whole plane rotates. 00:05:40.200 --> 00:05:43.920 Oh, that makes me remember a highly important, 00:05:43.920 --> 00:05:50.370 very important property of Q. It doesn't change length. 00:05:50.370 --> 00:05:55.260 The length of any vector is the same after you rotate it. 00:05:55.260 --> 00:05:59.100 The length of any vector is the same after you multiply by Q. 00:05:59.100 --> 00:06:00.810 Can I just do that? 00:06:00.810 --> 00:06:05.640 I claim any x, any vector x, I want 00:06:05.640 --> 00:06:08.890 to look at the length of Qx. 00:06:08.890 --> 00:06:12.630 And I claim it has the same length as x. 00:06:12.630 --> 00:06:15.750 Actually, that's the reason in computations 00:06:15.750 --> 00:06:20.460 that orthogonal matrix are so much loved, 00:06:20.460 --> 00:06:24.330 because no overflow can happen with orthogonal matrices. 00:06:24.330 --> 00:06:26.130 The lengths don't change. 00:06:26.130 --> 00:06:30.030 I can multiply by any number of orthogonal matrices 00:06:30.030 --> 00:06:32.820 and the length don't change. 00:06:32.820 --> 00:06:36.960 Can we just see why that's true? 00:06:36.960 --> 00:06:38.670 So what do we have to go on? 00:06:38.670 --> 00:06:43.740 What we have to go on is Q transpose Q equal I. 00:06:43.740 --> 00:06:46.350 Whatever we're going to prove, it's got to come out of that, 00:06:46.350 --> 00:06:49.260 because that's all we know. 00:06:49.260 --> 00:06:53.852 So how do I use that to get that one? 00:06:53.852 --> 00:06:55.810 Well, we haven't said a whole lot about length, 00:06:55.810 --> 00:06:57.460 but you'll see it all now. 00:06:57.460 --> 00:07:01.120 It'll be easier to prove that the squares are the same. 00:07:01.120 --> 00:07:03.760 So what's the what's the matrix expression 00:07:03.760 --> 00:07:05.410 for the length squared? 00:07:05.410 --> 00:07:08.920 What's the right hand side of that equation? 00:07:08.920 --> 00:07:11.200 X transpose x, right? 00:07:11.200 --> 00:07:14.560 X transpose x gives me the sum of the squares. 00:07:14.560 --> 00:07:17.670 Pythagoras says that's the length squared. 00:07:17.670 --> 00:07:21.030 So that right hand side is x transpose x. 00:07:21.030 --> 00:07:23.530 What's the left side? 00:07:23.530 --> 00:07:25.810 It's the length squared of this, of Qx. 00:07:25.810 --> 00:07:32.290 So it must be the same as Qx transpose Qx. 00:07:32.290 --> 00:07:36.510 And the claim is that that equation holds. 00:07:36.510 --> 00:07:40.030 And do you see it? 00:07:40.030 --> 00:07:45.710 So any property from Q has to just come out directly 00:07:45.710 --> 00:07:46.660 from that. 00:07:46.660 --> 00:07:48.430 Where is it here? 00:07:48.430 --> 00:07:52.730 Do I just like, push away a little bit at that left hand 00:07:52.730 --> 00:07:55.070 side and see it? 00:07:55.070 --> 00:08:00.500 Qx transpose is the same as x transpose Q transpose. 00:08:00.500 --> 00:08:03.500 And Qx is Qx. 00:08:03.500 --> 00:08:05.560 And now I'm seeing-- 00:08:05.560 --> 00:08:07.130 well, you might say, wait a minute. 00:08:07.130 --> 00:08:09.650 The parentheses were there and there. 00:08:09.650 --> 00:08:13.160 But I say the most important law for matrix multiplication 00:08:13.160 --> 00:08:16.560 is you can move parentheses or throw them away. 00:08:16.560 --> 00:08:17.970 Let's throw them away. 00:08:17.970 --> 00:08:21.230 So in here I'm seeing Q transpose Q, 00:08:21.230 --> 00:08:22.460 which is the identity. 00:08:22.460 --> 00:08:26.500 So it's true, yeah. 00:08:26.500 --> 00:08:29.830 So that means that you're never under flow or overflow 00:08:29.830 --> 00:08:35.770 when you're multiplying by Q. Every numerical algorithm 00:08:35.770 --> 00:08:42.309 is written to use orthogonal matrices wherever it can. 00:08:42.309 --> 00:08:44.545 And here's the first example. 00:08:49.420 --> 00:08:53.260 I think it may be good for me to think of other examples 00:08:53.260 --> 00:08:55.720 or for us to think of other examples 00:08:55.720 --> 00:08:58.790 of orthogonal matrices. 00:08:58.790 --> 00:09:03.360 So I'm using that word orthogonal matrix. 00:09:03.360 --> 00:09:06.070 We should really be saying orthonormal. 00:09:06.070 --> 00:09:11.230 And I'm really thinking mostly of square ones. 00:09:11.230 --> 00:09:16.550 So in this square case when Q transpose is Q inverse. 00:09:19.320 --> 00:09:23.140 Of course, that fact makes it easy to solve 00:09:23.140 --> 00:09:27.610 all equations that have Q as a coefficient matrix, 00:09:27.610 --> 00:09:31.180 because you want the inverse and you just use the transpose. 00:09:34.600 --> 00:09:40.342 Let's just take some minutes to think of examples of Q's. 00:09:40.342 --> 00:09:41.800 If they're so important, there have 00:09:41.800 --> 00:09:44.560 to be interesting examples. 00:09:44.560 --> 00:09:46.450 And that was a first one. 00:09:46.450 --> 00:09:52.180 Now there's one more two by two example that you should know. 00:09:52.180 --> 00:09:54.890 Do you know what that would be? 00:09:54.890 --> 00:09:58.990 This will be an example two, and it's also going to be only two 00:09:58.990 --> 00:10:01.870 by two and real. 00:10:01.870 --> 00:10:07.310 And what possibility have I got left here? 00:10:07.310 --> 00:10:11.240 I'll use this same first column, cos theta sine theta, 00:10:11.240 --> 00:10:17.620 because that's more or less any unit vector in two dimensions, 00:10:17.620 --> 00:10:20.030 this has got that form. 00:10:20.030 --> 00:10:25.036 So what do you propose for the second column? 00:10:25.036 --> 00:10:25.994 Yes? 00:10:25.994 --> 00:10:29.347 AUDIENCE: [INAUDIBLE]. 00:10:29.347 --> 00:10:33.550 PROFESSOR: Yeah, put the minus sign down here. 00:10:33.550 --> 00:10:35.890 So you think does that make any difference? 00:10:35.890 --> 00:10:41.240 So sine theta and minus cos theta. 00:10:41.240 --> 00:10:45.100 I don't know if you've ever looked at that matrix. 00:10:45.100 --> 00:10:47.990 We're trying to collect together a few matrices that 00:10:47.990 --> 00:10:51.050 are worth knowing, are worth looking at. 00:10:51.050 --> 00:10:53.150 Now what's happened here? 00:10:53.150 --> 00:10:57.290 You may say that was a trivial change, which it kind of was. 00:10:57.290 --> 00:10:59.150 But it's a different matrix now. 00:10:59.150 --> 00:11:01.520 It's not a rotation anymore. 00:11:01.520 --> 00:11:03.530 That's not a rotation. 00:11:03.530 --> 00:11:08.630 And yeah, somehow now it's symmetric. 00:11:08.630 --> 00:11:15.300 And yeah, it's eigenvectors must be something or other. 00:11:15.300 --> 00:11:17.670 We'll get to those. 00:11:17.670 --> 00:11:20.970 But what does that matrix do? 00:11:20.970 --> 00:11:22.680 I don't know if you've seen it. 00:11:22.680 --> 00:11:29.190 If you haven't, it doesn't jump out, but it's a important case. 00:11:29.190 --> 00:11:30.960 This is a reflection matrix. 00:11:39.640 --> 00:11:42.290 Notice that it's determinant is minus one. 00:11:42.290 --> 00:11:45.500 You have minus cos square theta, minus sine squared theta. 00:11:45.500 --> 00:11:46.940 It's determinant is minus one. 00:11:46.940 --> 00:11:54.350 There's some eigenvalue coming up that's got a minus. 00:11:57.410 --> 00:11:59.960 So what do I mean by a reflection matrix? 00:11:59.960 --> 00:12:01.265 Let me draw the plane. 00:12:04.080 --> 00:12:07.460 So one, zero, let's follow that, follow again. 00:12:07.460 --> 00:12:09.470 One, zero, where does that go? 00:12:09.470 --> 00:12:10.950 That gives me the first column. 00:12:10.950 --> 00:12:15.800 So as before, it goes to cos theta sine theta. 00:12:20.060 --> 00:12:22.600 And when I say reflection, let me 00:12:22.600 --> 00:12:26.170 put the mirror into the picture so you 00:12:26.170 --> 00:12:28.000 see what reflection it is. 00:12:28.000 --> 00:12:34.330 The mirror is along here at angle theta over two, 00:12:34.330 --> 00:12:37.780 theta over two line. 00:12:37.780 --> 00:12:42.520 So sure enough, one, zero at angle zero 00:12:42.520 --> 00:12:48.960 got reflected into a unit vector at angle theta, 00:12:48.960 --> 00:12:53.730 and halfway between was theta over two line. 00:12:53.730 --> 00:12:55.300 That's OK. 00:12:55.300 --> 00:12:59.590 Now what about the other guy, zero, one? 00:12:59.590 --> 00:13:01.950 Here's zero, one. 00:13:01.950 --> 00:13:03.350 I multiply that. 00:13:03.350 --> 00:13:07.440 Can I put the zero, one up here, so your I 00:13:07.440 --> 00:13:09.480 does the multiplication? 00:13:09.480 --> 00:13:13.630 Where is the result of zero, one? 00:13:13.630 --> 00:13:19.400 What's the output from Q applied to zero, one? 00:13:19.400 --> 00:13:22.170 Sine theta minus cos theta, right? 00:13:22.170 --> 00:13:24.300 It's the second column. 00:13:24.300 --> 00:13:26.990 And so where is that? 00:13:26.990 --> 00:13:29.000 Well, it's perpendicular to that guy. 00:13:29.000 --> 00:13:31.100 That's what I know. 00:13:31.100 --> 00:13:34.970 That was the point, that the two columns are perpendicular. 00:13:34.970 --> 00:13:37.910 So it must go down this way, right? 00:13:37.910 --> 00:13:41.630 Sine theta cos theta. 00:13:41.630 --> 00:13:43.970 And it doesn't change the length. 00:13:43.970 --> 00:13:47.690 All these facts that we just learned are key. 00:13:47.690 --> 00:13:50.630 So there's zero, one. 00:13:50.630 --> 00:13:56.960 And it goes to this guy, which is whatever that second column 00:13:56.960 --> 00:14:00.190 is sine theta minus cos theta. 00:14:02.960 --> 00:14:07.730 And if you check that, actually, gosh! 00:14:07.730 --> 00:14:09.800 This is like plain geometry. 00:14:09.800 --> 00:14:14.150 I believe-- it never occurred to me before-- but I believe it, 00:14:14.150 --> 00:14:18.525 that this angle going down to there, 00:14:18.525 --> 00:14:22.850 that that goes straight through, and that the halfway one 00:14:22.850 --> 00:14:23.650 is that line. 00:14:27.200 --> 00:14:29.580 Yeah, I think that picture has got it. 00:14:29.580 --> 00:14:33.070 And I think it's in the note. 00:14:33.070 --> 00:14:35.390 So that's a reflection matrix. 00:14:35.390 --> 00:14:38.650 Well, that's a two by two reflection matrix. 00:14:38.650 --> 00:14:45.060 Would you like to see some other matrices like this one, 00:14:45.060 --> 00:14:47.590 but larger? 00:14:47.590 --> 00:14:53.940 They're named after a guy named Householder. 00:14:53.940 --> 00:14:57.660 So these are Householder reflections. 00:15:02.520 --> 00:15:04.210 What am I doing here? 00:15:04.210 --> 00:15:07.260 I'm collecting together some orthogonal matrices 00:15:07.260 --> 00:15:09.930 that are useful and important. 00:15:09.930 --> 00:15:15.530 And Householder found a whole bunch of them. 00:15:15.530 --> 00:15:18.990 And his algorithm is a much used part 00:15:18.990 --> 00:15:21.330 of numerical linear algebra. 00:15:21.330 --> 00:15:24.400 So he started with a unit vector. 00:15:24.400 --> 00:15:31.040 Start with a unit vector u transpose u equal one. 00:15:31.040 --> 00:15:33.940 So the length of the vector is one. 00:15:33.940 --> 00:15:34.890 And then he created-- 00:15:34.890 --> 00:15:36.970 let's name it after him-- 00:15:36.970 --> 00:15:37.470 H. 00:15:37.470 --> 00:15:41.865 He created this matrix, the identity minus two u, 00:15:41.865 --> 00:15:42.780 u transpose. 00:15:46.920 --> 00:15:50.325 And I believe that that's a really useful matrix. 00:15:53.910 --> 00:15:57.780 I think this review is like going beyond 18.06, 00:15:57.780 --> 00:16:04.230 into what ones are really worth knowing, 00:16:04.230 --> 00:16:06.960 worth knowing individually. 00:16:06.960 --> 00:16:10.470 Could we just check what are the properties of Householders 00:16:10.470 --> 00:16:13.200 reflection of that I minus two? 00:16:13.200 --> 00:16:17.800 You recognize here a column times a row. 00:16:17.800 --> 00:16:19.690 So that's a matrix. 00:16:19.690 --> 00:16:27.000 And what could you tell me about that matrix u, u transpose? 00:16:27.000 --> 00:16:29.810 It's yeah? 00:16:32.810 --> 00:16:39.080 What can we say about H? 00:16:39.080 --> 00:16:44.540 So I guess I'm believing that H is a orthogonal matrix, 00:16:44.540 --> 00:16:46.160 otherwise it wouldn't be here today. 00:16:46.160 --> 00:16:49.070 So I believe that-- and that not only is it orthogonal, 00:16:49.070 --> 00:16:51.580 it is also-- 00:16:51.580 --> 00:16:52.940 have a look at it-- 00:16:52.940 --> 00:16:54.130 symmetric. 00:16:54.130 --> 00:16:55.460 It's also symmetric. 00:16:55.460 --> 00:16:57.260 The identity is symmetric. 00:16:57.260 --> 00:16:59.290 u, u transpose is symmetric. 00:16:59.290 --> 00:17:03.380 So this is a family of symmetric orthogonal matrices. 00:17:03.380 --> 00:17:06.109 And that was one of them. 00:17:06.109 --> 00:17:08.450 That's a symmetric orthogonal matrix. 00:17:08.450 --> 00:17:13.130 These matrices are really great to have. 00:17:13.130 --> 00:17:17.660 In using linear algebra, you just 00:17:17.660 --> 00:17:22.550 get a collection of useful matrices that you can call on. 00:17:22.550 --> 00:17:26.780 And these are definitely one family. 00:17:26.780 --> 00:17:28.369 Well, it's obviously symmetric. 00:17:28.369 --> 00:17:31.040 Shall we check that it's orthogonal? 00:17:31.040 --> 00:17:33.110 So to check that it's orthogonal, 00:17:33.110 --> 00:17:40.540 so I'm going to check that H transpose H is the identity. 00:17:40.540 --> 00:17:43.380 Can I just check that? 00:17:43.380 --> 00:17:46.800 Well, H transpose is the same as H because it was symmetric. 00:17:46.800 --> 00:17:49.070 So I'm going to square this guy. 00:17:49.070 --> 00:17:51.650 This is really H times H. 00:17:51.650 --> 00:17:53.360 I'm squaring it. 00:17:53.360 --> 00:17:55.730 And what do I get if I square-- 00:17:55.730 --> 00:17:57.830 So I get-- 00:17:57.830 --> 00:18:01.740 I hope I get the identity, but let's see it. 00:18:01.740 --> 00:18:05.835 What do I get when I square this? 00:18:05.835 --> 00:18:09.020 I get little-- multiply it out. 00:18:09.020 --> 00:18:12.170 So I times I is I. 00:18:12.170 --> 00:18:15.590 And then I get some number of u, u transposes. 00:18:15.590 --> 00:18:19.162 How many do I get from that? 00:18:19.162 --> 00:18:22.400 So I'm squaring this thing because H transpose 00:18:22.400 --> 00:18:26.760 J is the same as H times H. So I'm squaring it. 00:18:26.760 --> 00:18:29.080 So what do I put here? 00:18:29.080 --> 00:18:30.740 Four, thanks. 00:18:30.740 --> 00:18:33.720 And now I've got this guy squared with a plus. 00:18:33.720 --> 00:18:38.360 So that's four, u, you, transpose u, u transpose. 00:18:43.690 --> 00:18:45.700 Yeah, I'm totally realizing I've practiced 00:18:45.700 --> 00:18:49.660 for a lifetime doing these dinky little calculations. 00:18:49.660 --> 00:18:51.970 But they are dinky. 00:18:51.970 --> 00:18:55.750 And you'll get the hang of it quickly. 00:18:55.750 --> 00:18:59.390 Now what am I hoping out of that bottom line? 00:18:59.390 --> 00:19:04.450 That it is I. We're hoping we're going to get I. Do we get I? 00:19:04.450 --> 00:19:05.770 Yes. 00:19:05.770 --> 00:19:09.760 Who sees how to get I out of that thing? 00:19:09.760 --> 00:19:15.190 Yeah, u transpose u in here is a number. 00:19:15.190 --> 00:19:18.730 That was u-- that was column times row times column times 00:19:18.730 --> 00:19:19.570 row. 00:19:19.570 --> 00:19:23.750 And I look at in the middle here is row times column. 00:19:23.750 --> 00:19:28.430 And that's the number one right because it's there. 00:19:28.430 --> 00:19:31.240 So that's one and then I have minus 4 00:19:31.240 --> 00:19:32.650 of that plus 4 that that. 00:19:32.650 --> 00:19:37.690 They cancel each other, and I get I. 00:19:37.690 --> 00:19:40.260 So those are good matrices. 00:19:43.130 --> 00:19:43.920 We'll use them. 00:19:46.610 --> 00:19:47.330 We'll use them. 00:19:47.330 --> 00:19:49.400 Actually, they're better than Gram-Schmidt. 00:19:49.400 --> 00:19:53.750 So we'll use them in making things orthogonal. 00:19:53.750 --> 00:19:57.750 So what other orthogonal matrices? 00:19:57.750 --> 00:19:59.430 Let's create some. 00:19:59.430 --> 00:20:05.410 Creating good orthogonal matrices is-- 00:20:05.410 --> 00:20:08.030 you know, it pays off. 00:20:08.030 --> 00:20:11.200 Let's think. 00:20:11.200 --> 00:20:22.790 So there a family named after this French guy 00:20:22.790 --> 00:20:23.900 who lived to 100. 00:20:23.900 --> 00:20:27.820 He was a real old timer. 00:20:27.820 --> 00:20:32.890 Well, MIT had a faculty member in math, when I came, 00:20:32.890 --> 00:20:36.100 Professor Struik, who lived to 106. 00:20:36.100 --> 00:20:40.420 And I heard him give a lecture at Brown University at age 100. 00:20:40.420 --> 00:20:42.100 And it was perfect. 00:20:42.100 --> 00:20:44.020 You could not have done it better. 00:20:44.020 --> 00:20:45.550 So he's my inspiration. 00:20:45.550 --> 00:20:46.540 I'm keep going. 00:20:46.540 --> 00:20:50.710 I only have I like, n more years to get there. 00:20:50.710 --> 00:20:53.440 And then it's-- well, it's too many anyway. 00:20:56.410 --> 00:20:58.240 So Hadamard, he created-- 00:20:58.240 --> 00:21:06.750 well, that's the simplest, the smallest. 00:21:06.750 --> 00:21:10.410 Now the next guy is going to be four by four. 00:21:10.410 --> 00:21:12.060 I'm going to put that-- 00:21:12.060 --> 00:21:16.940 so where I see a one, I'm going to put 00:21:16.940 --> 00:21:22.580 Hadamard one, one, one minus one, one, one, one minus one. 00:21:22.580 --> 00:21:25.610 And then when I say a minus, I'm going to put an n with a minus. 00:21:29.990 --> 00:21:32.300 You saw that picture? 00:21:32.300 --> 00:21:39.440 It was a picture of H2, H2, H2, and minus H2. 00:21:39.440 --> 00:21:40.640 That's what I've got there. 00:21:43.380 --> 00:21:46.303 And I believe those columns are orthogonal. 00:21:49.081 --> 00:21:51.860 Right? 00:21:51.860 --> 00:21:53.670 Now what could I do? 00:21:53.670 --> 00:21:56.090 Well it's not quite an orthogonal matrix. 00:21:56.090 --> 00:21:59.690 What do I have to do to make-- this isn't quite 00:21:59.690 --> 00:22:01.520 an orthogonal matrix either. 00:22:01.520 --> 00:22:05.630 What do I do to make that an orthogonal matrix? 00:22:05.630 --> 00:22:11.000 Divide by square root of two? 00:22:11.000 --> 00:22:15.120 I need unit vectors there. 00:22:15.120 --> 00:22:19.190 And here, those links are one squared, one squared, got four, 00:22:19.190 --> 00:22:20.710 square root of four is two. 00:22:20.710 --> 00:22:23.720 So I better divide by the two there. 00:22:23.720 --> 00:22:26.190 And now here I'm up to-- 00:22:26.190 --> 00:22:26.810 yeah. 00:22:26.810 --> 00:22:29.080 So that was that one. 00:22:29.080 --> 00:22:31.010 Tell me the next one up. 00:22:31.010 --> 00:22:32.140 What's that going to be? 00:22:32.140 --> 00:22:34.040 Eight by eight? 00:22:34.040 --> 00:22:34.640 What's that? 00:22:34.640 --> 00:22:36.290 So this is H4 here. 00:22:39.080 --> 00:22:41.720 Oops, four. 00:22:41.720 --> 00:22:45.330 So tell me, what I should do for eight by eight. 00:22:45.330 --> 00:22:47.240 You know, it's simple. 00:22:47.240 --> 00:22:51.560 But that's a good thing to say about it. 00:22:51.560 --> 00:22:54.380 You know, they're in coding theory, 00:22:54.380 --> 00:22:58.040 all sorts of places you want matrices of ones and minus 00:22:58.040 --> 00:22:59.450 ones. 00:22:59.450 --> 00:23:01.770 What's H8? 00:23:01.770 --> 00:23:04.850 I'm going to build it out of H4. 00:23:04.850 --> 00:23:07.810 So what's it going to be? 00:23:07.810 --> 00:23:09.620 I'm going to put an H4 there. 00:23:09.620 --> 00:23:12.230 What am I going to put here? 00:23:12.230 --> 00:23:14.190 Another H4. 00:23:14.190 --> 00:23:16.260 And up here? 00:23:16.260 --> 00:23:17.430 Another H4. 00:23:17.430 --> 00:23:19.530 And finally, here? 00:23:19.530 --> 00:23:20.370 Minus H4. 00:23:23.030 --> 00:23:26.190 And I think I've got orthogonal columns again. 00:23:29.310 --> 00:23:36.210 Because the columns within these dot products with themselves 00:23:36.210 --> 00:23:38.230 give zero and zero. 00:23:38.230 --> 00:23:42.030 The dot products from these columns and these columns 00:23:42.030 --> 00:23:43.980 obviously, have the minus. 00:23:43.980 --> 00:23:48.120 And the dot products in here are zero from that and zero 00:23:48.120 --> 00:23:48.690 from that. 00:23:48.690 --> 00:23:50.010 Yeah, it works. 00:23:52.750 --> 00:23:59.770 And we could keep going to 16 and 32 and 64. 00:23:59.770 --> 00:24:04.600 But then up comes a question. 00:24:04.600 --> 00:24:07.930 What about H12? 00:24:07.930 --> 00:24:14.920 Is there ones and minus ones matrix of size 12, 12 by 12? 00:24:19.020 --> 00:24:22.590 It doesn't come directly from our little pattern, which 00:24:22.590 --> 00:24:25.110 is doubling size every time. 00:24:25.110 --> 00:24:27.060 But you could still hope. 00:24:27.060 --> 00:24:28.890 And it works. 00:24:28.890 --> 00:24:30.600 I don't know what it is, but there's 00:24:30.600 --> 00:24:36.000 a there's a matrix of ones and minus ones 12 00:24:36.000 --> 00:24:37.890 orthogonal columns. 00:24:37.890 --> 00:24:38.925 So the answer is yes. 00:24:42.480 --> 00:24:47.550 So we make an 18.065 conjecture. 00:24:47.550 --> 00:24:52.090 Every matrix size-- well, not every matrix size, 00:24:52.090 --> 00:24:54.850 because three by three is not going to work. 00:24:54.850 --> 00:24:57.570 One by one is not going to work either. 00:24:57.570 --> 00:25:00.970 But H12 works, H8, H16-- 00:25:00.970 --> 00:25:02.860 What's our conjecture? 00:25:02.860 --> 00:25:05.770 We won't be the first to conjecture it, 00:25:05.770 --> 00:25:09.440 that there is a ones and minus ones 00:25:09.440 --> 00:25:13.290 orthogonal matrix with orthogonal columns 00:25:13.290 --> 00:25:15.570 of every size n. 00:25:18.490 --> 00:25:25.320 So I'll start the conjecture, always possible 00:25:25.320 --> 00:25:31.470 if n, which is the size of the matrix, let's say. 00:25:31.470 --> 00:25:34.840 What would you guess? 00:25:34.840 --> 00:25:36.040 Just take a shot. 00:25:36.040 --> 00:25:39.100 You won't be asked for a proof, because nobody has a proof. 00:25:42.050 --> 00:25:43.700 Even? 00:25:43.700 --> 00:25:46.310 Well, you could hope for even. 00:25:46.310 --> 00:25:49.860 You could look for try six, I guess, would be the first guy 00:25:49.860 --> 00:25:50.360 there. 00:25:50.360 --> 00:25:52.700 And I don't think it's possible. 00:25:52.700 --> 00:25:55.430 I think six is not possible. 00:25:55.430 --> 00:25:59.220 So every even size is, I think, not 00:25:59.220 --> 00:26:02.870 going to work, but a natural idea to try. 00:26:02.870 --> 00:26:05.200 What's the next thought? 00:26:05.200 --> 00:26:06.370 Every multiple of four. 00:26:06.370 --> 00:26:10.890 If n over a four is a whole number. 00:26:20.860 --> 00:26:25.340 But nobody has a systematic way to create these things. 00:26:25.340 --> 00:26:27.830 So like some of them, at this point, 00:26:27.830 --> 00:26:30.080 we're down to doing it one at a time. 00:26:30.080 --> 00:26:34.430 And we're up to 668. 00:26:34.430 --> 00:26:36.680 But we haven't got that one yet. 00:26:36.680 --> 00:26:37.520 Isn't that crazy? 00:26:37.520 --> 00:26:39.440 So all this is coming from Wikipedia. 00:26:42.020 --> 00:26:44.450 My source of all that's good in mathematics 00:26:44.450 --> 00:26:46.520 is there on Wikipedia. 00:26:46.520 --> 00:26:49.340 Anyway, this is the conjecture. 00:26:49.340 --> 00:26:51.800 Conjecture means you don't have any damn idea 00:26:51.800 --> 00:26:55.280 of whether it's true or not. 00:26:55.280 --> 00:26:57.710 Is that divisible by four? 00:26:57.710 --> 00:26:59.480 Yeah, I guess it would be. 00:26:59.480 --> 00:27:02.200 600 certainly is, and 68 certainly is. 00:27:02.200 --> 00:27:03.680 Yeah, OK. 00:27:03.680 --> 00:27:06.570 So I think that's the first one. 00:27:06.570 --> 00:27:10.910 If you find one of size 668, just skip the homework 00:27:10.910 --> 00:27:13.620 and tell us about that one. 00:27:13.620 --> 00:27:18.340 But I don't think I'll assign that. 00:27:18.340 --> 00:27:21.320 Yeah, I must have searched for it online. 00:27:21.320 --> 00:27:25.050 But yeah. 00:27:25.050 --> 00:27:29.865 Anyway, so those are the Hadamard matrices. 00:27:33.100 --> 00:27:37.430 Now where else do I remember orthogonal matrices coming 00:27:37.430 --> 00:27:37.930 from? 00:27:37.930 --> 00:27:44.760 Well, yeah really, the biggest source. 00:27:44.760 --> 00:27:47.400 So when I'm looking for orthogonal matrices, 00:27:47.400 --> 00:27:52.020 I'm looking for a basis of orthogonal vectors. 00:27:52.020 --> 00:27:56.320 And where in math am I going to find vectors 00:27:56.320 --> 00:27:59.700 that come out to be orthogonal? 00:27:59.700 --> 00:28:02.340 We haven't seen-- that's the next section of the notes, 00:28:02.340 --> 00:28:05.130 but maybe you are remembering. 00:28:05.130 --> 00:28:08.580 Where will we sort of like, automatically show up 00:28:08.580 --> 00:28:11.920 with orthogonal vectors? 00:28:11.920 --> 00:28:14.440 They could be the eye eigenvectors 00:28:14.440 --> 00:28:17.830 of symmetric matrix. 00:28:17.830 --> 00:28:22.270 And that's where the most important ones come from. 00:28:22.270 --> 00:28:24.970 Oh, I could tell you about wavelets though. 00:28:24.970 --> 00:28:26.910 Wavelets are more like this picture. 00:28:26.910 --> 00:28:30.220 They're ones and minus ones, or the simplest wavelets 00:28:30.220 --> 00:28:32.530 are ones and minus ones. 00:28:32.530 --> 00:28:36.540 Before I go on to the eigenvector business, 00:28:36.540 --> 00:28:40.450 can I mention the wavelets matrices? 00:28:40.450 --> 00:28:44.860 Yeah, these are really important simple and important 00:28:44.860 --> 00:28:46.540 constructions. 00:28:46.540 --> 00:28:55.570 So wavelets- let me draw a picture of-- 00:28:55.570 --> 00:28:57.250 I'm going to come up with four-- 00:28:57.250 --> 00:28:59.440 I'll do the four by four case. 00:28:59.440 --> 00:29:07.410 And these are the orthogonal guys. 00:29:07.410 --> 00:29:14.680 And then the next one is and down and zero. 00:29:14.680 --> 00:29:20.980 And the last one is zero and up and down. 00:29:20.980 --> 00:29:22.940 So that's four things. 00:29:22.940 --> 00:29:24.850 But let me show you the matrix. 00:29:24.850 --> 00:29:27.880 So I'll call it W for wavelets. 00:29:27.880 --> 00:29:31.810 So that guy, I'm thinking of was one, one, one, one. 00:29:31.810 --> 00:29:35.320 This guy I'm thinking of as one, one, minus one, minus one. 00:29:35.320 --> 00:29:38.770 It's looking sort of Hadamard's way. 00:29:38.770 --> 00:29:40.870 But there's a difference here. 00:29:40.870 --> 00:29:44.770 This guy is one minus one, zero, zero. 00:29:44.770 --> 00:29:48.630 So the wavelengths rescale. 00:29:48.630 --> 00:29:51.390 That's the difference between Hadamard and wavelets. 00:29:51.390 --> 00:29:56.880 Wavelets are self-scaling. 00:29:56.880 --> 00:29:59.010 And what's the last guy here? 00:29:59.010 --> 00:30:00.480 What's the fourth column? 00:30:00.480 --> 00:30:03.430 From that fourth wavelet? 00:30:03.430 --> 00:30:11.460 Zero, zero, one minus one, yeah. 00:30:14.460 --> 00:30:17.560 So Haar came up with that. 00:30:17.560 --> 00:30:23.610 This is the Haar wavelet, which was many years before the word 00:30:23.610 --> 00:30:26.970 wavelets was invented. 00:30:26.970 --> 00:30:32.690 He came up with this construction, the Haar matrix, 00:30:32.690 --> 00:30:34.460 the Haar functions. 00:30:34.460 --> 00:30:38.840 So they're very simple functions, but you know, 00:30:38.840 --> 00:30:42.680 that makes them usable that the fact that they're so simple. 00:30:42.680 --> 00:30:47.030 Now I don't know if you want to see the pattern in eight 00:30:47.030 --> 00:30:50.600 by eight but let me start the eight by eight so you'll 00:30:50.600 --> 00:30:53.240 know what wavelets are about. 00:30:53.240 --> 00:30:55.840 You'll know what these Haar wavelets are about, anyway. 00:30:55.840 --> 00:31:02.540 They're the ones that were kind of easy to visualize. 00:31:02.540 --> 00:31:08.330 So if that's W4, let's just take a minute. 00:31:08.330 --> 00:31:12.680 It won't take long for W8. 00:31:12.680 --> 00:31:14.815 So the first column is going to be eight, one. 00:31:21.150 --> 00:31:23.370 And what's the next column going to be? 00:31:26.590 --> 00:31:30.480 Four ones and four minus ones, like so. 00:31:30.480 --> 00:31:34.350 So four ones and four minus ones. 00:31:36.990 --> 00:31:46.420 And now the next column, two ones two minus ones and zeros. 00:31:46.420 --> 00:31:50.660 One, one, minus one, minus one and zero. 00:31:50.660 --> 00:31:58.310 And the fourth will be zeros and two ones and two minus ones. 00:31:58.310 --> 00:31:59.480 We got half a matrix now. 00:32:02.660 --> 00:32:07.290 Now if we just tell me the fifth, what do you think? 00:32:07.290 --> 00:32:10.410 What do I put in the fifth one? 00:32:10.410 --> 00:32:15.810 So again, it's going to squeeze down and rescale. 00:32:15.810 --> 00:32:18.740 And what's fifth column up here that's 00:32:18.740 --> 00:32:22.740 going to be ones and minus ones and zeros now? 00:32:22.740 --> 00:32:25.440 So it's not Hadamard, it's Haar. 00:32:25.440 --> 00:32:28.430 And what shall I put? 00:32:28.430 --> 00:32:29.630 One, one. 00:32:29.630 --> 00:32:31.757 Shall I start with one, one? 00:32:31.757 --> 00:32:32.590 AUDIENCE: 1 minus 1. 00:32:32.590 --> 00:32:35.510 PROFESSOR: Oh! 00:32:35.510 --> 00:32:36.330 And then all zeros? 00:32:39.860 --> 00:32:40.700 Oh yeah, thanks! 00:32:40.700 --> 00:32:41.870 Perfect! 00:32:41.870 --> 00:32:44.240 One minus and then all zeros. 00:32:44.240 --> 00:32:46.340 And then the next three columns, we'll 00:32:46.340 --> 00:32:50.330 have the one minus one here and the one minus one here, 00:32:50.330 --> 00:32:51.620 and the one minus one here. 00:32:51.620 --> 00:32:53.080 And otherwise all zeros. 00:32:53.080 --> 00:32:53.930 Yeah. 00:32:53.930 --> 00:32:55.130 So you see the pattern. 00:32:55.130 --> 00:32:57.200 It's scaling at every step. 00:33:01.010 --> 00:33:04.640 So that matrix has the advantage of being 00:33:04.640 --> 00:33:07.670 quite sparse, short of-- 00:33:07.670 --> 00:33:13.680 This, in my mind is a-- 00:33:13.680 --> 00:33:21.670 or four ones, that get involved with like, taking the average. 00:33:21.670 --> 00:33:26.740 Then this guy is like, taking the differences 00:33:26.740 --> 00:33:28.270 between those and those. 00:33:28.270 --> 00:33:32.920 And then this is like taking the difference at a smaller scale. 00:33:32.920 --> 00:33:35.240 And that also at a smaller scale. 00:33:35.240 --> 00:33:36.700 So that's what we keep doing. 00:33:36.700 --> 00:33:37.270 Yeah. 00:33:37.270 --> 00:33:37.600 Yeah. 00:33:37.600 --> 00:33:38.380 So that wavelets. 00:33:41.750 --> 00:33:48.876 It looks so simple right, but just 00:33:48.876 --> 00:33:51.030 a one minute history of wavelets, 00:33:51.030 --> 00:33:54.480 so Haar invented this in like, 1910. 00:33:54.480 --> 00:33:58.700 I mean, a long-- 00:33:58.700 --> 00:34:00.260 forever. 00:34:00.260 --> 00:34:03.560 But then you wanted wavelets said 00:34:03.560 --> 00:34:09.110 we're a little better, and not just 00:34:09.110 --> 00:34:11.510 ones and minus ones and zeros. 00:34:11.510 --> 00:34:16.670 And that took a lot of thinking. 00:34:16.670 --> 00:34:18.690 A lot of people were searching for it. 00:34:18.690 --> 00:34:23.449 And Ingrid Daubechies-- so I'll just put her name-- 00:34:23.449 --> 00:34:27.280 became famous for finding them. 00:34:27.280 --> 00:34:36.560 So in about 1988 she found a whole lot 00:34:36.560 --> 00:34:38.350 of families of wavelets. 00:34:38.350 --> 00:34:40.790 And when I say wavelets, she found 00:34:40.790 --> 00:34:43.159 a whole lot of orthogonal matrices 00:34:43.159 --> 00:34:45.630 that had good properties. 00:34:45.630 --> 00:34:46.130 Yeah. 00:34:46.130 --> 00:34:49.040 So that's the wavelet picture. 00:34:49.040 --> 00:34:50.300 OK. 00:34:50.300 --> 00:34:53.600 Now to close today and to connect 00:34:53.600 --> 00:34:58.310 with the next lecture on eigenvalues, eigenvectors, 00:34:58.310 --> 00:35:01.520 positive definite matrices-- we're really getting 00:35:01.520 --> 00:35:04.520 to the heart of things here. 00:35:04.520 --> 00:35:07.220 Let me follow through on that idea. 00:35:09.840 --> 00:35:17.950 So the eigenvectors of a symmetric matrix, 00:35:17.950 --> 00:35:25.743 but also of an orthogonal matrix are orthogonal. 00:35:33.150 --> 00:35:35.970 And that is really where, people-- 00:35:35.970 --> 00:35:37.680 where you can invent-- 00:35:37.680 --> 00:35:39.570 because you don't have to work hard. 00:35:39.570 --> 00:35:42.210 You just find a symmetric matrix. 00:35:42.210 --> 00:35:46.980 Its eigenvectors are automatically orthogonal. 00:35:46.980 --> 00:35:51.300 That doesn't mean they're great for use, but some of them 00:35:51.300 --> 00:35:55.650 are really important. 00:35:55.650 --> 00:35:57.750 And that maybe the most important of all 00:35:57.750 --> 00:35:59.670 is the Fourier. 00:35:59.670 --> 00:36:03.630 So you probably have seen Fourier series sines 00:36:03.630 --> 00:36:04.530 and cosines. 00:36:04.530 --> 00:36:07.830 Those guys are orthogonal functions. 00:36:07.830 --> 00:36:11.760 But the discrete Fourier series is what everybody computes. 00:36:11.760 --> 00:36:14.500 And those are orthogonal vectors. 00:36:14.500 --> 00:36:17.970 So the are orthogonal vectors that 00:36:17.970 --> 00:36:22.900 go into the discrete Fourier transform. 00:36:22.900 --> 00:36:24.990 And then they're done at high speed 00:36:24.990 --> 00:36:27.360 by the fast Fourier transform. 00:36:27.360 --> 00:36:32.520 Those are eigenvectors of Q, eigenvectors of the right Q. 00:36:32.520 --> 00:36:39.390 So let me just tell you in the right Q that get-- 00:36:39.390 --> 00:36:41.410 who's eigenvectors-- 00:36:41.410 --> 00:36:43.110 So here we go. 00:36:43.110 --> 00:36:50.610 Eigenvectors of Q-- you will just 00:36:50.610 --> 00:36:53.880 be amazed by how simple this matrix is. 00:36:53.880 --> 00:36:56.370 It's just that matrix. 00:36:56.370 --> 00:37:01.920 It's called a permutation matrix. 00:37:01.920 --> 00:37:04.782 It just puts the-- 00:37:04.782 --> 00:37:15.840 those are the four, eight, four Fourier discrete-- 00:37:15.840 --> 00:37:18.160 let me put that word discrete up here-- 00:37:18.160 --> 00:37:18.660 transform. 00:37:22.550 --> 00:37:26.260 So I really meant to put discrete Fourier transform. 00:37:26.260 --> 00:37:27.630 Yeah. 00:37:27.630 --> 00:37:31.650 The eigenvectors of that matrix, first of all, 00:37:31.650 --> 00:37:37.680 they are orthogonal, and then second, and more important, 00:37:37.680 --> 00:37:40.380 they're tremendously useful. 00:37:40.380 --> 00:37:42.630 They're the heart of signal processing. 00:37:45.808 --> 00:37:50.620 In signal processing, they just take a discrete Fourier 00:37:50.620 --> 00:37:53.270 transform of a vector before they even look at it. 00:37:53.270 --> 00:37:55.910 I mean, like that's the way to see it, 00:37:55.910 --> 00:37:58.990 is split it into its frequencies. 00:37:58.990 --> 00:38:01.930 And that's what the eigenvectors of this will do. 00:38:01.930 --> 00:38:07.570 So we're going to see the discrete Fourier transform. 00:38:07.570 --> 00:38:15.050 But my point here is to know that they're orthogonal just 00:38:15.050 --> 00:38:18.620 comes out of this fact that the eigenvectors are 00:38:18.620 --> 00:38:22.490 orthogonal for any Q. And that is certainly 00:38:22.490 --> 00:38:27.380 a Q. Everybody can see that those columns are orthogonal. 00:38:30.270 --> 00:38:33.840 That's a permutation matrix. 00:38:33.840 --> 00:38:36.030 You've taken the columns of the identity, which 00:38:36.030 --> 00:38:38.430 are totally orthogonal, and you just 00:38:38.430 --> 00:38:40.530 put them in a different order. 00:38:40.530 --> 00:38:43.230 So a permutation matrix is a reordering 00:38:43.230 --> 00:38:44.910 of the identity matrix. 00:38:44.910 --> 00:38:51.740 It's got to be a Q, and therefore, its eigenvectors 00:38:51.740 --> 00:38:53.210 are orthogonal. 00:38:53.210 --> 00:38:56.690 And they're just the winners. 00:38:56.690 --> 00:39:01.530 I mean, the matrix of the Fourier matrix 00:39:01.530 --> 00:39:06.930 with those four eigenvectors in it, I'll show you now. 00:39:06.930 --> 00:39:13.530 We're finishing today by leading into Wednesday, eigenvectors 00:39:13.530 --> 00:39:15.000 and eigenvalues. 00:39:15.000 --> 00:39:18.000 And we happen to be doing eigenvectors and eigenvalues 00:39:18.000 --> 00:39:24.600 of a Q, not today, of an S. Most of the time, 00:39:24.600 --> 00:39:27.980 it's a symmetric matrix whose eigenvectors we take, 00:39:27.980 --> 00:39:30.960 but here that happens to be a Q. Can I 00:39:30.960 --> 00:39:35.780 show you the eigenvectors, the four eigenvectors of that? 00:39:35.780 --> 00:39:36.810 Now, oh! 00:39:40.500 --> 00:39:44.470 The complex number I is going to come in. 00:39:44.470 --> 00:39:46.510 You have to let it in here. 00:39:46.510 --> 00:39:47.270 Yeah, sorry! 00:39:52.170 --> 00:39:57.370 If S is a real symmetric matrix, its eigenvectors are real. 00:39:57.370 --> 00:40:02.190 But if Q is a orthogonal matrix, its eigenvectors-- 00:40:02.190 --> 00:40:07.470 even though you couldn't ask for a more real matrix than that-- 00:40:07.470 --> 00:40:13.450 but its eigenvectors are at least, 00:40:13.450 --> 00:40:17.210 the good way are complex numbers. 00:40:17.210 --> 00:40:19.630 So can I just show you the eigenvectors of-- 00:40:27.150 --> 00:40:29.070 So again, overall the point today 00:40:29.070 --> 00:40:33.120 is to see orthogonal matrices. 00:40:33.120 --> 00:40:36.870 So I'll just repeat now while I can-- 00:40:36.870 --> 00:40:54.240 rotations here, reflections, wavelets, the Householder idea 00:40:54.240 --> 00:40:58.350 of reflections of big, large matrices 00:40:58.350 --> 00:41:02.880 that have this form I minus 2 u, u transpose are orthogonal. 00:41:02.880 --> 00:41:06.080 And now we're going to see the big guys, 00:41:06.080 --> 00:41:12.400 the eigenvectors of Q. Yes? 00:41:12.400 --> 00:41:14.540 AUDIENCE: [INAUDIBLE]? 00:41:14.540 --> 00:41:17.138 PROFESSOR: Ah yeah, we don't have orthonormal. 00:41:17.138 --> 00:41:17.680 That's right. 00:41:17.680 --> 00:41:18.940 We don't have orthonormal. 00:41:18.940 --> 00:41:21.820 I better divide by the square root of eight. 00:41:21.820 --> 00:41:24.100 AUDIENCE: [INAUDIBLE]. 00:41:24.100 --> 00:41:25.200 PROFESSOR: Oh yes! 00:41:25.200 --> 00:41:27.470 Oh, you're right! 00:41:27.470 --> 00:41:27.970 Sorry. 00:41:27.970 --> 00:41:31.390 I just thought I'd get away with that, but I didn't. 00:41:31.390 --> 00:41:32.620 Yeah. 00:41:32.620 --> 00:41:34.870 So these guys these words of eight. 00:41:34.870 --> 00:41:35.830 So are these. 00:41:35.830 --> 00:41:38.350 But these guys are square roots of four. 00:41:38.350 --> 00:41:39.970 These guys are square roots of two. 00:41:39.970 --> 00:41:40.750 Thank you. 00:41:40.750 --> 00:41:42.130 Absolutely right. 00:41:42.130 --> 00:41:43.270 Absolutely. 00:41:43.270 --> 00:41:45.040 Yeah. 00:41:45.040 --> 00:41:50.120 OK, so what are the eigenvectors of a permutation. 00:41:50.120 --> 00:41:54.830 This is going to be nice to see. 00:41:54.830 --> 00:41:58.010 And I'll use the matrix F for the eigenvector 00:41:58.010 --> 00:42:12.280 matrix of that Q up there, and F is for Fourier. 00:42:12.280 --> 00:42:14.840 And it'd be the four by four Fourier matrix. 00:42:14.840 --> 00:42:15.380 OK. 00:42:15.380 --> 00:42:19.240 What are the eigenvectors of Q? 00:42:19.240 --> 00:42:20.320 OK. 00:42:20.320 --> 00:42:23.850 So Q is a permutation. 00:42:23.850 --> 00:42:27.450 So like, I'm going to ask you for one eigenvector 00:42:27.450 --> 00:42:29.340 of every permutation matrix. 00:42:29.340 --> 00:42:34.170 What vector can you tell me that actually the eigenvalue 00:42:34.170 --> 00:42:35.610 will be one? 00:42:35.610 --> 00:42:39.180 What vector can you tell me where if I permute it, 00:42:39.180 --> 00:42:41.670 I don't change it? 00:42:41.670 --> 00:42:43.650 One, one, one, one. 00:42:43.650 --> 00:42:46.260 Like, it's everywhere here, one, one, one, one. 00:42:49.160 --> 00:42:52.220 So that's the zero frequency for a vector, 00:42:52.220 --> 00:42:55.220 the constant vector, all ones. 00:42:55.220 --> 00:42:59.610 Everybody sees if I'm multiply by Q, it doesn't change. 00:42:59.610 --> 00:43:00.110 OK. 00:43:00.110 --> 00:43:01.880 Now the next one-- 00:43:01.880 --> 00:43:04.430 I'll show you the four now. 00:43:04.430 --> 00:43:08.720 The next one will be 1 I, I squared and I cubed. 00:43:12.610 --> 00:43:15.070 Of course, I squared-- 00:43:15.070 --> 00:43:19.720 I don't know how many course 6J people are in this audience, 00:43:19.720 --> 00:43:21.600 but this is a math building. 00:43:21.600 --> 00:43:22.560 We paid for it. 00:43:22.560 --> 00:43:23.820 It's 2-190. 00:43:23.820 --> 00:43:25.380 And it's I in this room. 00:43:30.840 --> 00:43:36.280 Anyway, I is the first letter in what? 00:43:36.280 --> 00:43:36.970 Imaginary. 00:43:36.970 --> 00:43:39.270 Thank you, the first letter in imaginary. 00:43:39.270 --> 00:43:42.080 You can't say jimaginary. 00:43:42.080 --> 00:43:43.150 So that's it. 00:43:43.150 --> 00:43:44.080 OK. 00:43:44.080 --> 00:43:49.510 And then the next one is 1 I squared, I fourth, I sixth. 00:43:49.510 --> 00:43:55.060 And the next one is 1 I cubed, I6 and I9. 00:43:55.060 --> 00:43:57.340 Isn't that just beautiful? 00:43:57.340 --> 00:44:01.960 And you could show that every one of those four columns, 00:44:01.960 --> 00:44:06.970 if you multiply them by Q, you would get the eigenvalues. 00:44:06.970 --> 00:44:11.590 And you would see that it's an eigenvector. 00:44:11.590 --> 00:44:17.080 And this is just sort of like a discrete Fourier stuff, instead 00:44:17.080 --> 00:44:22.460 of e to the I, e to the Ix, e to the 2Ix, e to the 3Ix 00:44:22.460 --> 00:44:22.960 and so on. 00:44:22.960 --> 00:44:23.835 We just have vectors. 00:44:27.640 --> 00:44:32.680 So those are the four eigenvectors 00:44:32.680 --> 00:44:34.720 of that permutation. 00:44:34.720 --> 00:44:38.960 And those are orthogonal. 00:44:38.960 --> 00:44:41.720 Could I just check that? 00:44:41.720 --> 00:44:45.050 How do you know that this first column and the second-- well, 00:44:45.050 --> 00:44:48.950 I should really say zeros column and first column, 00:44:48.950 --> 00:44:53.620 if I'm talking frequencies. 00:44:53.620 --> 00:44:56.330 Do you see that that's orthogonal to that? 00:44:56.330 --> 00:44:59.480 Well, y is one plus I plus I squared 00:44:59.480 --> 00:45:01.850 plus I cubed equal zero. 00:45:01.850 --> 00:45:06.360 That's the dot product is this column one dot column two. 00:45:10.100 --> 00:45:12.440 Yeah, you're right. 00:45:12.440 --> 00:45:14.420 This happens to come out zero. 00:45:14.420 --> 00:45:17.205 Is that right? 00:45:17.205 --> 00:45:19.130 AUDIENCE: Yeah, that will come out zero. 00:45:19.130 --> 00:45:22.540 PROFESSOR: That will come out zero. 00:45:22.540 --> 00:45:27.040 But somebody mentioned that this isn't right. 00:45:27.040 --> 00:45:29.590 It's true that that came out zero, 00:45:29.590 --> 00:45:36.550 but when I have imaginary numbers anywhere around, 00:45:36.550 --> 00:45:40.540 this isn't a correct dot product to test orthogonal. 00:45:40.540 --> 00:45:42.680 If I have imaginary-- 00:45:42.680 --> 00:45:45.160 complex vectors-- if I have complex numbers, 00:45:45.160 --> 00:45:52.060 complex vectors, I should test column one conjugate dotted 00:45:52.060 --> 00:45:53.770 with column two-- 00:45:53.770 --> 00:46:00.730 column I conjugate-- well, let me take column see, 00:46:00.730 --> 00:46:02.230 which ones shall I take? 00:46:02.230 --> 00:46:04.660 Maybe that guy and that guy. 00:46:04.660 --> 00:46:09.030 Many of them, you luck out here. 00:46:09.030 --> 00:46:15.000 But really, I should be taking that conjugate. 00:46:15.000 --> 00:46:16.670 So these ones-- 00:46:16.670 --> 00:46:20.970 But the thing is, the complex conjugate of one is one. 00:46:20.970 --> 00:46:23.700 So that was OK. 00:46:23.700 --> 00:46:25.980 But in general, if I wanted to take 00:46:25.980 --> 00:46:33.570 column two dotted with column maybe four 00:46:33.570 --> 00:46:35.820 would be a little dodgy-- 00:46:35.820 --> 00:46:36.600 yeah. 00:46:36.600 --> 00:46:37.980 Look what happens. 00:46:37.980 --> 00:46:41.760 Take that column and that column. 00:46:41.760 --> 00:46:44.450 Take their dot product. 00:46:44.450 --> 00:46:47.370 Do it the wrong way. 00:46:47.370 --> 00:46:49.500 So what's the wrong way? 00:46:49.500 --> 00:46:52.110 Forget about the complex conjugate 00:46:52.110 --> 00:46:54.195 and just do it the usual way. 00:46:54.195 --> 00:46:55.950 So one times one is one. 00:46:55.950 --> 00:46:57.570 I times I cube is? 00:47:00.930 --> 00:47:02.430 One. 00:47:02.430 --> 00:47:05.110 I squared times I6 is? 00:47:05.110 --> 00:47:05.610 One. 00:47:05.610 --> 00:47:07.380 I'm getting all ones. 00:47:07.380 --> 00:47:10.260 I'm not getting orthogonality there. 00:47:10.260 --> 00:47:14.430 And that's because I forgot that I should take the complex 00:47:14.430 --> 00:47:17.010 conjugate-- well, of these guys-- one-- 00:47:17.010 --> 00:47:21.750 I should take minus I, minus I, minus I squared-- 00:47:21.750 --> 00:47:22.500 well, that's real. 00:47:22.500 --> 00:47:23.760 So it's OK. 00:47:23.760 --> 00:47:26.310 Minus there. 00:47:26.310 --> 00:47:32.316 So minus I squared is still minus one. 00:47:32.316 --> 00:47:36.250 So now if I do it, it comes out zero. 00:47:36.250 --> 00:47:39.330 So let me repeat again. 00:47:39.330 --> 00:47:41.730 Let me just make this statement. 00:47:41.730 --> 00:47:50.280 If Q transpose Q is I, and Qx is lambda x, 00:47:50.280 --> 00:47:57.230 and Qy is different eigenvalue y-- 00:47:57.230 --> 00:48:00.680 so I'm setting up the main fact. 00:48:00.680 --> 00:48:03.440 In the last minute, I'm just going to write down. 00:48:03.440 --> 00:48:05.930 So I have an orthogonal matrix. 00:48:05.930 --> 00:48:08.990 I have an eigenvector with eigenvalue lambda. 00:48:08.990 --> 00:48:12.740 I have another eigenvector with a different eigenvalue, mu. 00:48:12.740 --> 00:48:14.930 Then the claim is that-- 00:48:14.930 --> 00:48:18.830 what is the claim about eigenvectors? 00:48:18.830 --> 00:48:22.310 So here, this has an eigenvalue. 00:48:22.310 --> 00:48:24.700 This has a different eigenvalue. 00:48:24.700 --> 00:48:27.170 I need them to be different to really know 00:48:27.170 --> 00:48:31.520 that the x's and the y's can't be the same. 00:48:31.520 --> 00:48:32.990 So what is it that I want to show? 00:48:32.990 --> 00:48:34.598 AUDIENCE: [INAUDIBLE]. 00:48:34.598 --> 00:48:35.630 PROFESSOR: Yes! 00:48:35.630 --> 00:48:39.980 That x-- and I have to remember to do that. 00:48:39.980 --> 00:48:43.640 x transpose y is zero. 00:48:43.640 --> 00:48:46.340 That's orthogonality. 00:48:46.340 --> 00:48:49.460 That's orthogonality for a complex vectors. 00:48:49.460 --> 00:48:54.440 I have to remember to change are every I to a minus 00:48:54.440 --> 00:48:56.780 I in one of the vectors. 00:48:56.780 --> 00:49:00.770 And I can prove that fact by playing with these. 00:49:03.470 --> 00:49:07.560 By starting from here, I can get to that. 00:49:07.560 --> 00:49:08.060 OK. 00:49:08.060 --> 00:49:08.560 That's it. 00:49:08.560 --> 00:49:10.820 We've done a lot today, a lot of stuff 00:49:10.820 --> 00:49:13.340 about orthogonal matrices. 00:49:13.340 --> 00:49:18.560 Important ones, and those sources of important ones, 00:49:18.560 --> 00:49:19.400 eigenvectors. 00:49:19.400 --> 00:49:23.950 And so it will be eigenvectors on Wednesday.