WEBVTT 00:00:01.550 --> 00:00:03.920 The following content is provided under a Creative 00:00:03.920 --> 00:00:05.310 Commons license. 00:00:05.310 --> 00:00:07.520 Your support will help MIT OpenCourseWare 00:00:07.520 --> 00:00:11.610 continue to offer high quality educational resources for free. 00:00:11.610 --> 00:00:14.180 To make a donation, or to view additional materials 00:00:14.180 --> 00:00:18.140 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:18.140 --> 00:00:19.026 at ocw.mit.edu. 00:00:22.510 --> 00:00:27.340 GILBERT STRANG: So last time was orthogonal matrices-- 00:00:27.340 --> 00:00:32.995 Q. And this time is symmetric matrices, S. 00:00:32.995 --> 00:00:37.620 So we're really talking about the best matrices of all. 00:00:37.620 --> 00:00:43.620 Well, I'll start with any square matrix and about eigenvectors. 00:00:43.620 --> 00:00:47.130 But you've heard of eigenvectors more than once-- 00:00:47.130 --> 00:00:48.540 more than twice-- 00:00:48.540 --> 00:00:50.580 more than 10 times, probably. 00:00:50.580 --> 00:00:51.690 OK. 00:00:51.690 --> 00:00:54.270 So eigenvectors. 00:00:54.270 --> 00:00:58.440 And then, let's be sure we know why they're useful, 00:00:58.440 --> 00:01:02.820 and maybe compute one or two. 00:01:02.820 --> 00:01:05.910 But then we'll move to symmetric matrices 00:01:05.910 --> 00:01:08.260 and what is special about those. 00:01:08.260 --> 00:01:12.060 And then, even more special and more important 00:01:12.060 --> 00:01:16.350 will be positive definite symmetric matrices-- 00:01:16.350 --> 00:01:20.280 so that when I say, positive definite, I mean symmetric. 00:01:20.280 --> 00:01:26.850 So start with A. Next comes S. Then come the special S-- 00:01:26.850 --> 00:01:29.190 special symmetric matrices that have 00:01:29.190 --> 00:01:31.950 this extra positive definite property. 00:01:31.950 --> 00:01:33.270 OK. 00:01:33.270 --> 00:01:39.360 So start with A. So an eigenvector-- 00:01:39.360 --> 00:01:44.470 if I multiply A by x, I get some vector. 00:01:44.470 --> 00:01:51.010 And sometimes, if x is especially chosen well, 00:01:51.010 --> 00:01:55.988 Ax comes out in the same direction as x. 00:01:55.988 --> 00:02:01.830 Ax comes out some number times x. 00:02:01.830 --> 00:02:04.530 So there are-- normally, there would be, 00:02:04.530 --> 00:02:06.120 for an n by n matrix-- 00:02:06.120 --> 00:02:13.040 so let's say A is n by n today. 00:02:13.040 --> 00:02:16.910 Normally, if we live right, there 00:02:16.910 --> 00:02:21.260 will be n different independent vectors-- 00:02:21.260 --> 00:02:26.740 x eigenvectors-- that have this special property. 00:02:26.740 --> 00:02:32.680 And we can compute them by hand if n is 2 or 3-- 00:02:32.680 --> 00:02:35.140 2, mostly. 00:02:35.140 --> 00:02:38.920 But the computation of the x's and the lambdas-- 00:02:38.920 --> 00:02:46.230 so this is for i equal 1 up to n, 00:02:46.230 --> 00:02:49.920 if I use this sort of math shorthand-- 00:02:49.920 --> 00:02:53.880 that I have n of these almost always. 00:02:53.880 --> 00:02:59.190 And my first question is, what are they good for? 00:02:59.190 --> 00:03:05.840 Why does course after course introduce eigenvectors? 00:03:05.840 --> 00:03:12.440 And to me the key property is seen by looking at A squared. 00:03:12.440 --> 00:03:13.910 So let me look at A squared. 00:03:17.430 --> 00:03:20.690 So it's another n by n matrix. 00:03:20.690 --> 00:03:24.140 And we would ask, suppose we know these guys? 00:03:24.140 --> 00:03:27.890 Suppose we've found those somehow. 00:03:27.890 --> 00:03:30.410 What about A squared? 00:03:30.410 --> 00:03:35.020 Is x an eigenvector of A squared also? 00:03:35.020 --> 00:03:39.310 Well, the way to find out is to multiply A squared by x, 00:03:39.310 --> 00:03:41.390 and see what happens. 00:03:41.390 --> 00:03:43.640 Do you see what's going to happen here? 00:03:43.640 --> 00:03:50.960 This is A times Ax, which is A times-- 00:03:50.960 --> 00:03:54.370 Ax is lambda x-- 00:03:54.370 --> 00:03:58.150 and now what do I do now? 00:03:58.150 --> 00:04:01.150 Because I'm shooting for the answer yes. 00:04:01.150 --> 00:04:04.300 X is an eigenvector of A squared also. 00:04:04.300 --> 00:04:05.250 So what do I do? 00:04:05.250 --> 00:04:09.010 That number-- that lambda is just a number. 00:04:09.010 --> 00:04:10.930 I can put it anywhere I like. 00:04:10.930 --> 00:04:13.210 So I can put it out front. 00:04:13.210 --> 00:04:16.170 And then I have Ax, which is? 00:04:16.170 --> 00:04:17.395 AUDIENCE: Lambda x. 00:04:17.395 --> 00:04:18.000 GILBERT STRANG: Lambda x. 00:04:18.000 --> 00:04:18.500 Thanks. 00:04:18.500 --> 00:04:19.800 So I have another lambda x. 00:04:19.800 --> 00:04:21.200 So there's lambda squared x. 00:04:21.200 --> 00:04:24.870 So I learned the crucial thing here-- 00:04:24.870 --> 00:04:28.270 that x is also an eigenvector of A squared, 00:04:28.270 --> 00:04:33.700 and the eigenvalue is lambda squared. 00:04:33.700 --> 00:04:36.710 And of course, I can keep going. 00:04:36.710 --> 00:04:39.025 So A to the nth-- 00:04:39.025 --> 00:04:42.580 x is lambda to the nth x. 00:04:42.580 --> 00:04:46.660 We have found the right vectors for that particular matrix A. 00:04:46.660 --> 00:04:49.970 What about A inverse x? 00:04:49.970 --> 00:04:53.230 That will be-- if everything is good-- 00:04:53.230 --> 00:04:57.040 1 over lambda x. 00:04:57.040 --> 00:04:58.750 Well, yeah. 00:04:58.750 --> 00:05:01.420 So anytime I write 1 over lambda, 00:05:01.420 --> 00:05:07.810 my mind says, you gotta make some comment 00:05:07.810 --> 00:05:11.680 on the special case where it doesn't work, which is? 00:05:11.680 --> 00:05:13.405 AUDIENCE: Lambda is not equal to 0. 00:05:13.405 --> 00:05:14.280 GILBERT STRANG: Yeah. 00:05:14.280 --> 00:05:17.720 If lambda is not 0, I'm golden. 00:05:17.720 --> 00:05:22.590 If lambda is 0, it doesn't look good. 00:05:22.590 --> 00:05:26.137 And what's happening if lambda is 0? 00:05:26.137 --> 00:05:27.470 AUDIENCE: A inverse [INAUDIBLE]. 00:05:27.470 --> 00:05:29.428 GILBERT STRANG: A doesn't even have an inverse. 00:05:29.428 --> 00:05:32.010 If lambda was 0-- 00:05:32.010 --> 00:05:33.620 which it could be-- 00:05:33.620 --> 00:05:36.710 no rule against it. 00:05:36.710 --> 00:05:40.520 If lambda was 0, this would say, A times the eigenvector 00:05:40.520 --> 00:05:42.860 is 0 times the eigenvector. 00:05:42.860 --> 00:05:44.900 So that would tell me that the eigenvector 00:05:44.900 --> 00:05:47.750 is in the null space. 00:05:47.750 --> 00:05:51.040 It would tell me the matrix A isn't invertible. 00:05:51.040 --> 00:05:55.220 It's taking some vector x to 0. 00:05:55.220 --> 00:05:58.550 And so everything clicks. 00:05:58.550 --> 00:06:01.760 This works when it should work. 00:06:01.760 --> 00:06:05.950 And if we have other fun-- any function of the matrix, 00:06:05.950 --> 00:06:08.590 we could define the exponential of a matrix. 00:06:08.590 --> 00:06:10.570 18.03 would do that. 00:06:10.570 --> 00:06:15.340 Let's just write it down, as if we know what it means. 00:06:15.340 --> 00:06:17.740 Does it have the same eigenvector? 00:06:17.740 --> 00:06:18.820 Well, sure. 00:06:18.820 --> 00:06:20.330 Because e to the At-- 00:06:20.330 --> 00:06:22.900 the exponential of a matrix-- 00:06:22.900 --> 00:06:25.120 if I see e to the something-- 00:06:25.120 --> 00:06:27.610 I think of that long, infinite series 00:06:27.610 --> 00:06:29.980 that gives the exponential. 00:06:29.980 --> 00:06:34.720 Those-- all the terms in that series have powers of A. 00:06:34.720 --> 00:06:36.250 So everything is working. 00:06:36.250 --> 00:06:38.430 Every term in that series-- 00:06:38.430 --> 00:06:39.830 x is an eigenvector. 00:06:39.830 --> 00:06:41.830 And when I put it all together, I 00:06:41.830 --> 00:06:47.740 learn that the eigenvalue is e to the lambda t. 00:06:47.740 --> 00:06:54.670 That's just a typical and successful work use. 00:06:54.670 --> 00:06:55.660 OK. 00:06:55.660 --> 00:06:58.720 So that's eigenvectors and eigenvalues, 00:06:58.720 --> 00:07:02.810 and we'll find some in a minute. 00:07:05.800 --> 00:07:07.965 Now, so I'm claiming that this-- 00:07:11.720 --> 00:07:16.120 that from this first thing-- which was just about certain 00:07:16.120 --> 00:07:17.950 vectors are special-- 00:07:17.950 --> 00:07:21.490 now we're beginning to see why they're useful. 00:07:21.490 --> 00:07:22.720 So special is good. 00:07:22.720 --> 00:07:25.060 Useful is even better. 00:07:25.060 --> 00:07:37.810 So let me take any vector, say v. And OK, 00:07:37.810 --> 00:07:39.910 what do I want to do? 00:07:39.910 --> 00:07:42.190 I want to use eigenvectors. 00:07:42.190 --> 00:07:45.280 This v is probably not an eigenvector. 00:07:45.280 --> 00:07:49.330 But I'm supposing that I've got n of them. 00:07:49.330 --> 00:07:51.400 You and I are agreed that there are 00:07:51.400 --> 00:07:54.760 some matrices for which there are not 00:07:54.760 --> 00:07:57.290 a full set of eigenvectors. 00:07:57.290 --> 00:08:02.480 That's really the main sort of annoying point 00:08:02.480 --> 00:08:05.440 in the whole subject of linear algebra, 00:08:05.440 --> 00:08:08.870 is some matrices don't have enough eigenvectors. 00:08:08.870 --> 00:08:14.280 But almost all do, and let's go forward 00:08:14.280 --> 00:08:16.730 assuming our matrix has. 00:08:16.730 --> 00:08:17.820 OK. 00:08:17.820 --> 00:08:22.200 So if I've got n independent eigenvectors, that's a basis. 00:08:22.200 --> 00:08:26.250 I can write any vector v as a combination 00:08:26.250 --> 00:08:30.620 of those eigenvectors. 00:08:30.620 --> 00:08:33.250 Right. 00:08:33.250 --> 00:08:40.090 And then I can find out what A to any power. 00:08:40.090 --> 00:08:42.820 So that's the point. 00:08:42.820 --> 00:08:49.290 This is going to be the simple and reason why 00:08:49.290 --> 00:08:50.520 we like to have-- 00:08:50.520 --> 00:08:52.840 we like to know the eigenvectors. 00:08:52.840 --> 00:08:56.470 Because if I choose those as my basis vectors, 00:08:56.470 --> 00:08:58.750 v is a combination of them. 00:08:58.750 --> 00:09:04.030 Now if I multiply by A, or A squared, or A to the k power, 00:09:04.030 --> 00:09:05.890 then it's linear. 00:09:05.890 --> 00:09:08.950 So I can multiply each one by A to the k. 00:09:08.950 --> 00:09:15.300 And what do I get if I multiply that guy by A to the kth power? 00:09:15.300 --> 00:09:16.240 OK. 00:09:16.240 --> 00:09:18.580 Well, I'm just going to use-- or, here 00:09:18.580 --> 00:09:21.122 I said n, but let me say k. 00:09:21.122 --> 00:09:23.260 Because n-- I'm sorry. 00:09:23.260 --> 00:09:26.210 I'm using n for the size of the matrix. 00:09:26.210 --> 00:09:31.870 So I better use k for the typical case here. 00:09:31.870 --> 00:09:34.460 So what do I get? 00:09:34.460 --> 00:09:39.140 Just help me through this and we're happy. 00:09:39.140 --> 00:09:44.330 So what happens when I multiply that by A to the k? 00:09:44.330 --> 00:09:47.720 It's an eigenvector, remember, so when I 00:09:47.720 --> 00:09:49.670 multiply by A to the k, I get? 00:09:49.670 --> 00:09:50.360 AUDIENCE: C1. 00:09:50.360 --> 00:09:51.610 GILBERT STRANG: C1. 00:09:51.610 --> 00:09:53.350 That's just a number. 00:09:53.350 --> 00:09:56.840 And A to the k times that eigenvector gives? 00:09:56.840 --> 00:09:57.650 AUDIENCE: Lambda 1. 00:09:57.650 --> 00:10:00.653 GILBERT STRANG: Lambda 1 to the k times the eigenvector. 00:10:03.970 --> 00:10:05.730 Right? 00:10:05.730 --> 00:10:07.170 That's the whole point. 00:10:07.170 --> 00:10:11.340 And linearity says keep going. 00:10:11.340 --> 00:10:18.270 Cn, lambda n to the kth power, Xn. 00:10:18.270 --> 00:10:22.080 In other words, I can take-- 00:10:22.080 --> 00:10:24.820 I can apply any power of a matrix. 00:10:24.820 --> 00:10:27.250 I can apply the exponential of a matrix. 00:10:27.250 --> 00:10:33.380 I can do anything quickly, because I've 00:10:33.380 --> 00:10:35.210 got the eigenvector. 00:10:35.210 --> 00:10:38.630 So really, I'm saying the first use 00:10:38.630 --> 00:10:41.900 for eigenvectors-- maybe the principle use for which they 00:10:41.900 --> 00:10:47.660 were invented-- is to be able to solve difference equations. 00:10:47.660 --> 00:10:51.560 So if I call that Vk-- 00:10:51.560 --> 00:10:55.790 the kth power-- then the equation I'm solving here 00:10:55.790 --> 00:11:01.530 is a one step difference equation. 00:11:01.530 --> 00:11:03.790 This is my difference equation. 00:11:03.790 --> 00:11:06.140 And if I wanted to use exponentials, 00:11:06.140 --> 00:11:12.090 the equation I would be solving would be dv, dt equal Av. 00:11:17.050 --> 00:11:29.820 Solution to discrete steps, or continuous time evolution comes 00:11:29.820 --> 00:11:32.290 is trivial, if I know the eigenvectors. 00:11:32.290 --> 00:11:35.560 Because here is the solution to this one. 00:11:35.560 --> 00:11:40.325 And the solution to this one is the same thing, C1, e 00:11:40.325 --> 00:11:43.480 to the lambda, 1, t, x1. 00:11:43.480 --> 00:11:49.180 Is that what you were expecting for the solution here? 00:11:49.180 --> 00:11:50.830 Because if I takes the derivative, 00:11:50.830 --> 00:11:52.810 it brings down a lambda. 00:11:52.810 --> 00:11:56.680 If I multiply by A, it brings down a lambda-- 00:11:56.680 --> 00:11:59.020 so, plus the other guys. 00:12:03.520 --> 00:12:04.020 OK. 00:12:09.600 --> 00:12:15.110 Not news, but important to remember what eigenvectors 00:12:15.110 --> 00:12:17.280 are for in the first place. 00:12:17.280 --> 00:12:17.780 Good. 00:12:21.960 --> 00:12:22.620 Yeah. 00:12:22.620 --> 00:12:24.220 Let me move ahead. 00:12:24.220 --> 00:12:34.270 Oh-- one matrix fact is about something 00:12:34.270 --> 00:12:35.800 called similar matrices. 00:12:35.800 --> 00:12:38.680 So I have on my matrix A. Then I have 00:12:38.680 --> 00:12:41.980 the idea of what it means to be similar to A, 00:12:41.980 --> 00:12:56.740 so B is similar to A. What does that mean? 00:12:56.740 --> 00:12:59.410 So here's what it means, first of all. 00:12:59.410 --> 00:13:04.450 It means that B can be found from A, by-- 00:13:04.450 --> 00:13:07.310 this is the key operation here-- 00:13:07.310 --> 00:13:10.930 multiplying by a matrix M, and its inverse-- 00:13:10.930 --> 00:13:12.940 M inverse AM. 00:13:12.940 --> 00:13:18.550 When I see two matrices, B and A, 00:13:18.550 --> 00:13:24.620 that are connected by that kind of a change, 00:13:24.620 --> 00:13:28.160 M could be any invertible matrix. 00:13:28.160 --> 00:13:34.410 Then I would say B was similar to A. And that changed-- 00:13:34.410 --> 00:13:38.920 that appearance of AM is pretty natural. 00:13:38.920 --> 00:13:43.770 If I change variables here by M, then I get-- 00:13:43.770 --> 00:13:47.850 that similar matrix will show up. 00:13:47.850 --> 00:13:49.110 So what's the key factor? 00:13:49.110 --> 00:13:53.760 Do you remember the key fact about similar matrices? 00:13:53.760 --> 00:13:56.972 If B and A are connected like that-- 00:13:56.972 --> 00:13:58.680 AUDIENCE: They have the same eigenvalues. 00:13:58.680 --> 00:14:01.380 GILBERT STRANG: They have the same eigenvalues. 00:14:01.380 --> 00:14:04.170 So this is just a useful point to remember. 00:14:04.170 --> 00:14:13.320 So I'll-- this is like one fact in the discussion 00:14:13.320 --> 00:14:15.780 of eigenvalues and eigenvectors. 00:14:15.780 --> 00:14:28.600 So similar matrices, same eigenvalues. 00:14:37.140 --> 00:14:38.100 Yeah. 00:14:38.100 --> 00:14:42.810 So in some way in the eigenvalue, eigenvector world, 00:14:42.810 --> 00:14:45.090 they're in this-- they belong together. 00:14:49.390 --> 00:14:54.503 They're connected by this relation that just turns out 00:14:54.503 --> 00:14:55.420 to be the right thing. 00:14:58.120 --> 00:15:03.820 Actually, that is-- it gives us a clue of how eigenvalues 00:15:03.820 --> 00:15:05.950 are actually computed. 00:15:05.950 --> 00:15:10.510 Well, they're actually computed by typing eig of A, 00:15:10.510 --> 00:15:14.080 with parentheses around A. That's how they're-- 00:15:14.080 --> 00:15:17.890 in real life. 00:15:17.890 --> 00:15:22.000 But what happens when you type eig of A? 00:15:22.000 --> 00:15:24.010 Well, you could say the eigenvalue shows up 00:15:24.010 --> 00:15:26.080 on the screen. 00:15:26.080 --> 00:15:28.660 But something had to happen in there. 00:15:28.660 --> 00:15:33.250 And what happened was that MATLAB-- 00:15:33.250 --> 00:15:40.540 or whoever-- took that matrix A, started using good choices 00:15:40.540 --> 00:15:41.140 of m-- 00:15:43.860 --> 00:15:44.610 better and better. 00:15:47.170 --> 00:15:50.550 Took a bunch of steps with different m's. 00:15:50.550 --> 00:15:53.860 Because if I do another m, I still have a similar matrix, 00:15:53.860 --> 00:15:54.360 right? 00:15:54.360 --> 00:16:00.300 If I take B and do a different m2 to B-- 00:16:00.300 --> 00:16:02.220 so I get something similar to B, then 00:16:02.220 --> 00:16:04.230 that's also similar to A. I've got 00:16:04.230 --> 00:16:07.020 a whole family of similar things there. 00:16:07.020 --> 00:16:13.410 And what does MATLAB do with all these m's, m1 and m2 and m3 00:16:13.410 --> 00:16:14.580 and so on? 00:16:14.580 --> 00:16:22.140 It brings the matrix to a triangular matrix. 00:16:22.140 --> 00:16:24.915 It gets the eigenvalues showing up on the diagonal. 00:16:27.620 --> 00:16:33.110 It's just tremendously-- it was an inspiration when that-- 00:16:33.110 --> 00:16:35.990 when the good choice of m appeared. 00:16:35.990 --> 00:16:38.060 And let me just say-- 00:16:38.060 --> 00:16:41.120 because I'm going on to symmetric matrices-- 00:16:41.120 --> 00:16:47.360 that for a symmetric matrices, everything is sort of clean. 00:16:47.360 --> 00:16:51.770 You not only go to a triangular matrix, 00:16:51.770 --> 00:16:54.380 you go toward a diagonal matrix. 00:16:54.380 --> 00:16:57.380 They off-- you choose m's that make 00:16:57.380 --> 00:17:00.710 the off diagonal stuff smaller and smaller and smaller. 00:17:00.710 --> 00:17:03.230 And the eigenvalues are not changing. 00:17:03.230 --> 00:17:08.800 So there, shooting up on the diagonal, are the eigenvalues. 00:17:08.800 --> 00:17:13.329 So I guess I should verify that fact, 00:17:13.329 --> 00:17:16.329 that similar matrices have the same eigenvalues. 00:17:16.329 --> 00:17:19.300 Can we-- there can't be much to show. 00:17:19.300 --> 00:17:24.710 There can't be much in the proof because that's all I know. 00:17:24.710 --> 00:17:27.400 And I want to know its eigenvalues and eigenvectors. 00:17:27.400 --> 00:17:32.920 So let me say, suppose m inverse Am has the eigenvector 00:17:32.920 --> 00:17:35.125 y and the eigenvalue of lambda. 00:17:41.360 --> 00:17:44.610 And I want to show-- 00:17:44.610 --> 00:17:48.810 do I want to show that y is an eigenvector also, of A itself? 00:17:48.810 --> 00:17:50.010 No. 00:17:50.010 --> 00:17:52.140 Eigenvectors are changing. 00:17:52.140 --> 00:17:56.610 Do I want to show that lambda is an eigenvalue of A itself? 00:17:56.610 --> 00:17:57.150 Yes. 00:17:57.150 --> 00:17:58.560 That's my point. 00:17:58.560 --> 00:18:00.656 So can we see that? 00:18:00.656 --> 00:18:01.650 Ha. 00:18:01.650 --> 00:18:05.250 Can I see that lambda is an eigenvector? 00:18:05.250 --> 00:18:07.440 There's not a lot to do here. 00:18:07.440 --> 00:18:10.200 I mean, if I can't do it soon, I'm never going to do it, 00:18:10.200 --> 00:18:12.300 because-- 00:18:12.300 --> 00:18:13.845 so what am I going to do? 00:18:13.845 --> 00:18:15.930 AUDIENCE: Define the vector x equals my-- 00:18:15.930 --> 00:18:17.530 GILBERT STRANG: Yeah, I could. 00:18:17.530 --> 00:18:19.320 Yeah. 00:18:19.320 --> 00:18:24.120 X is-- m-y is going to be a key, and I can see m-y coming. 00:18:24.120 --> 00:18:26.550 Just-- when I see m inverse over there, 00:18:26.550 --> 00:18:28.345 what am I going to do with the darn thing? 00:18:28.345 --> 00:18:29.220 AUDIENCE: [INAUDIBLE] 00:18:29.220 --> 00:18:31.500 GILBERT STRANG: I'm going to put it on the other side. 00:18:31.500 --> 00:18:34.020 I'm going to multiply that equation by m. 00:18:34.020 --> 00:18:37.560 So I'll have-- that will put the m over here. 00:18:37.560 --> 00:18:44.630 And I'll have A-M-y equals lambda My, right? 00:18:48.680 --> 00:18:50.520 And is that telling me what I want to know? 00:18:50.520 --> 00:18:51.790 Yes. 00:18:51.790 --> 00:18:54.560 That's saying that My-- 00:18:54.560 --> 00:18:58.340 that you wisely suggested to give a name x to-- 00:18:58.340 --> 00:19:01.025 is lambda times My. 00:19:01.025 --> 00:19:02.285 Do you see that? 00:19:02.285 --> 00:19:06.650 That the eigenvalue lambda didn't change. 00:19:06.650 --> 00:19:08.970 The eigenvector did change. 00:19:08.970 --> 00:19:11.480 It changed from y to My. 00:19:11.480 --> 00:19:13.350 That's the x. 00:19:13.350 --> 00:19:14.930 The eigenvector of x. 00:19:14.930 --> 00:19:18.530 This is lambda x. 00:19:18.530 --> 00:19:19.910 Yeah. 00:19:19.910 --> 00:19:24.110 So that's the role of M. It just gives you a different basis 00:19:24.110 --> 00:19:25.610 for eigenvectors. 00:19:25.610 --> 00:19:28.010 But it does not change eigenvalues. 00:19:28.010 --> 00:19:28.880 Right. 00:19:28.880 --> 00:19:30.250 Yeah. 00:19:30.250 --> 00:19:31.540 OK. 00:19:31.540 --> 00:19:35.380 So those are similar matrices. 00:19:35.380 --> 00:19:37.180 Yeah, some other good things happen. 00:19:37.180 --> 00:19:39.190 A lot of people don't know-- in fact, 00:19:39.190 --> 00:19:42.610 I wasn't very conscious of the fact 00:19:42.610 --> 00:19:48.500 that A times B has the same eigenvalues as B times A. Well, 00:19:48.500 --> 00:19:51.130 I should maybe write that down. 00:19:51.130 --> 00:19:58.190 AB has the same eigenvalues-- 00:19:58.190 --> 00:20:00.300 the same non-zero ones-- 00:20:00.300 --> 00:20:01.720 you'll see. 00:20:01.720 --> 00:20:07.240 I have to-- as BA. 00:20:07.240 --> 00:20:10.690 This is any A and B same size. 00:20:10.690 --> 00:20:13.390 I'm not talking similar matrices here. 00:20:13.390 --> 00:20:19.270 I'm talking any two A and B. Yeah. 00:20:19.270 --> 00:20:23.140 So that's a good thing that happens. 00:20:23.140 --> 00:20:29.880 Now could we see y? 00:20:29.880 --> 00:20:35.240 And then I'm going to be really pretty happy with basic fact 00:20:35.240 --> 00:20:37.870 about eigenvalues. 00:20:37.870 --> 00:20:41.070 So if I want to show that two things have 00:20:41.070 --> 00:20:45.760 the same eigenvalues, what do you propose? 00:20:45.760 --> 00:20:49.660 Show that they are similar. 00:20:49.660 --> 00:20:51.440 I already said, if they are similar. 00:20:51.440 --> 00:20:53.540 So is there an m? 00:20:53.540 --> 00:20:58.370 Is there an m that will connect this matrix? 00:20:58.370 --> 00:21:05.820 So is there an m that will multiply this matrix that way? 00:21:05.820 --> 00:21:07.880 So that would be similar to AB. 00:21:07.880 --> 00:21:09.950 And can I produce BA then? 00:21:16.770 --> 00:21:19.110 So I'll just put the word want up here. 00:21:22.960 --> 00:21:28.000 I want-- if I have that, then I'm 00:21:28.000 --> 00:21:32.140 done, because that's saying that those two matrices, AB and BA, 00:21:32.140 --> 00:21:33.160 are similar. 00:21:33.160 --> 00:21:36.930 And I know that then they have the same eigenvalues. 00:21:36.930 --> 00:21:41.425 So what should m be? 00:21:41.425 --> 00:21:49.640 M should be-- so what is M here? 00:21:49.640 --> 00:21:50.870 I want that to be true. 00:21:54.700 --> 00:21:57.890 Should M be B? 00:21:57.890 --> 00:21:58.660 Yeah. 00:21:58.660 --> 00:22:01.020 M equal B. Boy. 00:22:01.020 --> 00:22:05.710 Not the most hidden fact here. 00:22:05.710 --> 00:22:11.110 Take M equal B. 00:22:11.110 --> 00:22:14.950 So then I have B times A, times BB inverse-- 00:22:14.950 --> 00:22:16.130 which is the identity. 00:22:16.130 --> 00:22:18.130 So I have B times A. Yes. 00:22:18.130 --> 00:22:19.690 OK. 00:22:19.690 --> 00:22:23.440 So AB and BA are fine. 00:22:23.440 --> 00:22:28.240 Now, what do you think about this question? 00:22:28.240 --> 00:22:31.610 Are the eigenvalues-- I now know that AB and BA 00:22:31.610 --> 00:22:33.230 have the same eigenvalues. 00:22:33.230 --> 00:22:40.730 And the reason I had to be careful about non-zero is that 00:22:40.730 --> 00:22:45.065 if I had zero eigenvalues, then-- 00:22:45.065 --> 00:22:46.055 AUDIENCE: [INAUDIBLE] 00:22:46.055 --> 00:22:46.930 GILBERT STRANG: Yeah. 00:22:46.930 --> 00:22:49.370 I can't count on those inverses. 00:22:49.370 --> 00:22:50.380 Right. 00:22:50.380 --> 00:22:51.570 Right. 00:22:51.570 --> 00:22:56.100 So that's why I put it in that little qualifier. 00:22:56.100 --> 00:22:59.190 But now I want to ask this question. 00:22:59.190 --> 00:23:02.070 If I know the eigenvalues of A-- 00:23:02.070 --> 00:23:05.570 separately, by itself, A-- and of B-- 00:23:05.570 --> 00:23:09.850 now I'm talking about any two matrices, A and B. 00:23:09.850 --> 00:23:12.855 If I have two matrices, A-- 00:23:12.855 --> 00:23:15.580 I have a matrix A and a matrix B. 00:23:15.580 --> 00:23:19.170 And I know their eigenvalues and their eigenvalues. 00:23:19.170 --> 00:23:21.790 What about AB? 00:23:21.790 --> 00:23:25.510 A times B. Can I multiply the eigenvalues of A times 00:23:25.510 --> 00:23:27.640 the eigenvalues of B? 00:23:27.640 --> 00:23:28.540 Don't do it. 00:23:28.540 --> 00:23:29.080 Right. 00:23:29.080 --> 00:23:29.730 Yes. 00:23:29.730 --> 00:23:30.230 Right. 00:23:30.230 --> 00:23:33.340 The eigenvalues of A times the eigenvalues of B 00:23:33.340 --> 00:23:35.620 could be damn near anything. 00:23:35.620 --> 00:23:36.680 Right. 00:23:36.680 --> 00:23:40.890 They're not connected to the eigenvalues of AB specially. 00:23:40.890 --> 00:23:45.800 And maybe something could be discovered, but not much. 00:23:45.800 --> 00:23:50.920 And similarly, for A plus B. So yeah. 00:23:50.920 --> 00:23:54.670 So let me just write down this point. 00:23:54.670 --> 00:24:00.100 Eigenvalues of A plus B are generally not 00:24:00.100 --> 00:24:08.830 eigenvalues of A plus eigenvalues of B. 00:24:08.830 --> 00:24:09.670 Generally not. 00:24:09.670 --> 00:24:12.200 Just-- there is no reason. 00:24:12.200 --> 00:24:15.580 And the reason that that's-- 00:24:15.580 --> 00:24:20.260 I get that no answer is, that the eigenvectors can 00:24:20.260 --> 00:24:21.040 be all different. 00:24:21.040 --> 00:24:23.770 If the eigenvectors for A are totally 00:24:23.770 --> 00:24:26.300 different from the eigenvectors for B, 00:24:26.300 --> 00:24:30.160 then A plus B will have probably some other, totally different 00:24:30.160 --> 00:24:34.360 eigenvectors, and there's nothing happening there. 00:24:38.420 --> 00:24:43.650 That's sort of thoughts about eigenvalues in general. 00:24:43.650 --> 00:24:50.230 And I could-- there'd be a whole section on eigenvectors, 00:24:50.230 --> 00:24:53.830 but I'm really interested in eigenvectors 00:24:53.830 --> 00:24:56.450 of symmetric matrices. 00:24:56.450 --> 00:25:02.550 So I'm going to move on to that topic. 00:25:02.550 --> 00:25:06.410 So now, having talked about any matrix A, 00:25:06.410 --> 00:25:09.710 I'm going to specialize to symmetric matrices, 00:25:09.710 --> 00:25:12.020 see what's special about the eigenvalues 00:25:12.020 --> 00:25:14.790 there, what's special about eigenvectors there. 00:25:14.790 --> 00:25:17.720 And I think we've already said it in class. 00:25:17.720 --> 00:25:20.070 So let me-- let me ask you to tell me 00:25:20.070 --> 00:25:21.590 about it-- tell me again. 00:25:21.590 --> 00:25:26.270 So I'll call that matrix S now, as a reminder 00:25:26.270 --> 00:25:29.970 always that I'm talking here about symmetric matrices. 00:25:29.970 --> 00:25:33.710 So what do I-- what are the key facts to know? 00:25:33.710 --> 00:25:44.360 Eigenvalues are real numbers, if the matrix is. 00:25:44.360 --> 00:25:48.840 I'm thinking of real symmetric matrices. 00:25:48.840 --> 00:25:51.060 Of course, other real matrices could 00:25:51.060 --> 00:25:55.400 have imaginary eigenvalues. 00:25:55.400 --> 00:25:57.620 Other real matrices-- so just-- 00:25:57.620 --> 00:26:00.860 let's just think for a moment. 00:26:00.860 --> 00:26:01.360 Yeah. 00:26:01.360 --> 00:26:02.620 Maybe I'll just put it here. 00:26:02.620 --> 00:26:09.760 Can I back up, before I keep going with symmetric matrices? 00:26:09.760 --> 00:26:15.610 So you take a matrix like that. 00:26:20.150 --> 00:26:20.850 Q, yeah. 00:26:20.850 --> 00:26:25.500 That would be a Q. But it's not specially a Q. Maybe 00:26:25.500 --> 00:26:28.320 the most remarkable thing about that matrix 00:26:28.320 --> 00:26:31.080 is that it's anti-symmetric. 00:26:31.080 --> 00:26:33.630 So I'll call it A. Right. 00:26:33.630 --> 00:26:37.942 If I transpose that matrix, what do I get? 00:26:37.942 --> 00:26:38.900 AUDIENCE: The negative. 00:26:38.900 --> 00:26:40.108 GILBERT STRANG: The negative. 00:26:40.108 --> 00:26:42.400 So that's like anti-symmetric. 00:26:42.400 --> 00:26:45.430 And I claim that an anti-symmetric matrix 00:26:45.430 --> 00:26:47.830 has imaginary eigenvalues. 00:26:47.830 --> 00:26:51.010 So that's a 90 degree rotation. 00:26:54.330 --> 00:26:57.040 And you might say, what could be simpler than that? 00:26:57.040 --> 00:27:01.050 A 90 degree rotation-- that's not a weird matrix. 00:27:01.050 --> 00:27:03.740 But from the point of view of eigenvectors, 00:27:03.740 --> 00:27:07.620 something a little odd has to happen, right? 00:27:07.620 --> 00:27:11.010 Because if I have a 90 degree rotation-- 00:27:11.010 --> 00:27:12.720 if I take a vector x-- 00:27:12.720 --> 00:27:18.250 any vector x-- could it possibly be an eigenvector? 00:27:18.250 --> 00:27:20.910 Well, apply A to it. 00:27:20.910 --> 00:27:24.510 You'd be off in this direction, Ax. 00:27:24.510 --> 00:27:30.390 And there is no way that Ax can be a multiple of x. 00:27:30.390 --> 00:27:34.950 So there's no real eigenvector for that anti-symmetric matrix, 00:27:34.950 --> 00:27:38.710 or any anti-symmetric matrix. 00:27:38.710 --> 00:27:43.480 So you see that when we say that the eigenvalues 00:27:43.480 --> 00:27:46.090 of a symmetric matrix are real, we're 00:27:46.090 --> 00:27:48.180 saying that this couldn't happen-- 00:27:48.180 --> 00:27:51.160 that this couldn't happen if A were symmetric. 00:27:51.160 --> 00:27:54.010 And here, it's the very opposite, it's anti-symmetric. 00:27:56.880 --> 00:27:59.820 Well, while that's on the board, you might say, wait a minute. 00:27:59.820 --> 00:28:02.160 How could that have any eigenvector whatsoever? 00:28:05.950 --> 00:28:09.300 So what is an eigenvector of that matrix A? 00:28:09.300 --> 00:28:13.080 How do you find the eigenvectors of A? 00:28:13.080 --> 00:28:19.850 When they're 2 by 2, that's a calculation we know how to do. 00:28:19.850 --> 00:28:21.950 You remember the steps there? 00:28:21.950 --> 00:28:26.690 I'm looking for Ax equal lambda x. 00:28:26.690 --> 00:28:30.020 So right now I'm looking for both lambda and x. 00:28:30.020 --> 00:28:31.050 I've got 2. 00:28:31.050 --> 00:28:35.900 It's not linear, but I'm going to bring this over to this side 00:28:35.900 --> 00:28:39.405 and write it as A minus lambda I, x equals 0. 00:28:43.263 --> 00:28:44.680 And then I'm going to look at that 00:28:44.680 --> 00:28:48.850 and say, wow, A minus lambda I must be not invertible, 00:28:48.850 --> 00:28:52.690 b because it's got this x in its null space. 00:28:52.690 --> 00:28:56.680 So the determinant of this matrix must be 0. 00:28:59.490 --> 00:29:06.300 I couldn't have a null space unless the determinant is 0. 00:29:06.300 --> 00:29:12.590 And then when I look at A minus lambda I, for this A, 00:29:12.590 --> 00:29:20.350 I've got minus lambdas, minus A-- 00:29:20.350 --> 00:29:21.820 oh, A is just the 1. 00:29:21.820 --> 00:29:23.700 And that's minus 1. 00:29:23.700 --> 00:29:26.250 I'm going to take the determinant. 00:29:26.250 --> 00:29:29.100 And what am I going to get for the determinant? 00:29:29.100 --> 00:29:30.732 Lambda squared-- 00:29:30.732 --> 00:29:31.722 AUDIENCE: Plus 1. 00:29:31.722 --> 00:29:32.680 GILBERT STRANG: Plus 1. 00:29:36.620 --> 00:29:38.156 And I set that to 0. 00:29:44.220 --> 00:29:48.020 So I'm just following all the rules, 00:29:48.020 --> 00:29:51.322 but it's showing me that the lambda-- 00:29:51.322 --> 00:29:55.150 the two lambdas-- there are two lambdas here-- 00:29:55.150 --> 00:29:58.300 but they're not real, because that equation, the roots 00:29:58.300 --> 00:29:59.770 are i and minus i. 00:30:02.660 --> 00:30:04.250 So those are the eigenvalues. 00:30:07.060 --> 00:30:08.710 And they have the nice-- 00:30:08.710 --> 00:30:10.180 they have all the-- 00:30:10.180 --> 00:30:11.530 well, they are the eigenvalues. 00:30:11.530 --> 00:30:12.520 No doubt about it. 00:30:15.560 --> 00:30:20.360 With 2 by 2 there are two quick checks that tell you, yeah, 00:30:20.360 --> 00:30:22.790 you did a calculation right. 00:30:22.790 --> 00:30:31.970 If I add up the two eigenvalues in this-- 00:30:31.970 --> 00:30:34.550 if I add up the two eigenvalues for any matrix, 00:30:34.550 --> 00:30:37.310 and I'm going to do it for this one-- 00:30:37.310 --> 00:30:38.420 I get what answer? 00:30:38.420 --> 00:30:39.500 AUDIENCE: The trace? 00:30:39.500 --> 00:30:43.430 GILBERT STRANG: I get the same answer from the adding-- 00:30:43.430 --> 00:30:48.410 add the lambdas gives me the same answer 00:30:48.410 --> 00:30:56.690 as add the diagonal of the matrix-- 00:30:56.690 --> 00:31:03.130 which I'm calling A. So if I add the diagonal I get 0 and 0. 00:31:03.130 --> 00:31:03.910 So it's 0 plus 0. 00:31:07.790 --> 00:31:11.165 And this number adding the diagonal is called the trace. 00:31:13.790 --> 00:31:19.130 And we'll see it again because it's so simple. 00:31:19.130 --> 00:31:22.910 Just adding the diagonal entries gives you 00:31:22.910 --> 00:31:25.370 a key bit of information. 00:31:25.370 --> 00:31:27.230 When you add down the diagonal it 00:31:27.230 --> 00:31:29.960 tells you the sum of the eigenvalue-- some 00:31:29.960 --> 00:31:32.670 of the lambdas. 00:31:32.670 --> 00:31:35.970 Doesn't tell you each lambda separately, 00:31:35.970 --> 00:31:37.890 but it tells you the sum. 00:31:37.890 --> 00:31:41.380 So it tells you one fact by doing one thing. 00:31:41.380 --> 00:31:42.210 Yeah. 00:31:42.210 --> 00:31:45.170 That's pretty handy. 00:31:45.170 --> 00:31:49.010 Gives you a quick check if you've-- 00:31:49.010 --> 00:31:50.840 when you compute this determinant 00:31:50.840 --> 00:31:54.414 and solve for lambda-- 00:31:54.414 --> 00:32:03.440 the thing you-- this is a way to compute eigenvalues by hand. 00:32:03.440 --> 00:32:05.120 You could make a mistake, because it's 00:32:05.120 --> 00:32:10.770 a quadratic formula for 2 by 2, but you can 00:32:10.770 --> 00:32:13.270 check by adding the two roots. 00:32:13.270 --> 00:32:17.640 Do you get the same as the trace 0 plus 0? 00:32:20.420 --> 00:32:25.370 Well, there's one other check, equally quick, for 2 by 2, 00:32:25.370 --> 00:32:26.690 so 2 by 2s-- 00:32:26.690 --> 00:32:28.940 you really get them right. 00:32:28.940 --> 00:32:31.070 What's the other check to-- 00:32:31.070 --> 00:32:33.860 we add the eigenvalues, we get the trace. 00:32:33.860 --> 00:32:34.970 AUDIENCE: [INAUDIBLE] 00:32:34.970 --> 00:32:37.010 GILBERT STRANG: We multiply the eigenvalues. 00:32:37.010 --> 00:32:45.640 So we take-- so now multiply the lambdas. 00:32:45.640 --> 00:32:50.600 So then I get i times minus i. 00:32:50.600 --> 00:32:54.550 And that should equal-- let's-- don't look yet. 00:32:54.550 --> 00:32:58.060 What should it equal if I multiply the eigenvalues 00:32:58.060 --> 00:32:59.870 I should get the? 00:32:59.870 --> 00:33:00.810 AUDIENCE: Determinant. 00:33:00.810 --> 00:33:02.700 GILBERT STRANG: Determinant, right. 00:33:02.700 --> 00:33:12.680 Of A. So that's two handy checks. 00:33:12.680 --> 00:33:14.900 Add the eigenvalues-- for any size-- 00:33:14.900 --> 00:33:18.230 3 by 3, 4 by 4-- but it's only two checks. 00:33:18.230 --> 00:33:21.080 So for 2 by 2, it's kind of, you've got it. 00:33:21.080 --> 00:33:23.300 3 by 3, 4 by 4-- you could still have 00:33:23.300 --> 00:33:29.630 made an error and the two checks could potentially still work. 00:33:29.630 --> 00:33:30.920 Let's just check it out here. 00:33:30.920 --> 00:33:32.570 What's i times minus i? 00:33:35.530 --> 00:33:36.030 AUDIENCE: 1. 00:33:36.030 --> 00:33:37.230 GILBERT STRANG: 1. 00:33:37.230 --> 00:33:40.290 Because it's minus i squared, and that's plus 1. 00:33:40.290 --> 00:33:44.950 And the determinant of that matrix is 0 minus-- 00:33:44.950 --> 00:33:45.450 is 1. 00:33:45.450 --> 00:33:47.170 Yeah. 00:33:47.170 --> 00:33:47.820 OK. 00:33:47.820 --> 00:33:49.150 So we got 1. 00:33:49.150 --> 00:33:49.650 Good. 00:33:53.610 --> 00:33:56.340 Those are really the key fact about eigenvalues. 00:33:59.010 --> 00:34:02.550 But of course they're not-- it's not 00:34:02.550 --> 00:34:06.360 as simple as solving Ax equal B to find them, 00:34:06.360 --> 00:34:13.980 but if you follow through on this idea of similar matrices, 00:34:13.980 --> 00:34:18.510 and sort of chop down the off diagonal part, then 00:34:18.510 --> 00:34:22.900 sure enough, the eigenvalue's gotta show up. 00:34:22.900 --> 00:34:24.790 OK. 00:34:24.790 --> 00:34:25.449 Symmetric. 00:34:27.989 --> 00:34:28.905 Symmetric matrices. 00:34:33.830 --> 00:34:38.830 So now we're going to have symmetric, 00:34:38.830 --> 00:34:42.639 and then we'll have the special, even better than symmetric, 00:34:42.639 --> 00:34:45.690 is symmetric positive definite. 00:34:45.690 --> 00:34:46.190 OK. 00:34:46.190 --> 00:34:56.330 Symmetric-- you told me the main facts are the eigenvalues real, 00:34:56.330 --> 00:35:02.040 the eigenvectors orthogonal. 00:35:08.030 --> 00:35:10.280 And I guess, actually-- 00:35:10.280 --> 00:35:11.420 yeah. 00:35:11.420 --> 00:35:17.380 So I want to put those into math symbols instead of words. 00:35:22.480 --> 00:35:26.500 So yeah. 00:35:26.500 --> 00:35:31.270 I guess-- shall I just jump in? 00:35:31.270 --> 00:35:36.830 And the other thing hidden there-- but very important is-- 00:35:36.830 --> 00:35:39.790 there's a full set of eigenvectors, 00:35:39.790 --> 00:35:42.400 even if some eigenvalues happen to be repeated, 00:35:42.400 --> 00:35:44.650 like the identity matrix. 00:35:44.650 --> 00:35:47.940 It's still got plenty of eigenvectors. 00:35:47.940 --> 00:35:51.150 So that's a added point that I've not made there. 00:35:51.150 --> 00:35:54.720 And I could prove those two statements, 00:35:54.720 --> 00:35:59.998 but why don't I ask you to accept them and go onward? 00:36:02.810 --> 00:36:05.240 What are we going to do with them? 00:36:05.240 --> 00:36:05.740 OK. 00:36:11.600 --> 00:36:13.430 Can you just-- let's have an example. 00:36:16.550 --> 00:36:18.690 Let me put an example here. 00:36:18.690 --> 00:36:22.910 Suppose S-- now I'm calling it S-- 00:36:22.910 --> 00:36:26.610 is 0s, 1 and 1. 00:36:26.610 --> 00:36:29.960 So that's symmetric. 00:36:29.960 --> 00:36:33.080 What are its eigenvalues? 00:36:33.080 --> 00:36:37.140 What are the eigenvalues of that symmetric matrix, S? 00:36:37.140 --> 00:36:38.370 AUDIENCE: Plus and minus 1. 00:36:38.370 --> 00:36:40.878 GILBERT STRANG: Plus and minus 1. 00:36:40.878 --> 00:36:43.660 Well, if you propose two eigenvalues, 00:36:43.660 --> 00:36:46.730 I'll write them down, 1 and minus 1. 00:36:46.730 --> 00:36:49.727 And then what will I do to check them? 00:36:49.727 --> 00:36:51.060 AUDIENCE: Trace and determinant. 00:36:51.060 --> 00:36:53.810 GILBERT STRANG: Trace and determinant. 00:36:53.810 --> 00:36:54.330 OK. 00:36:54.330 --> 00:36:57.600 So are they-- is it true that the eigenvalues 00:36:57.600 --> 00:37:01.600 are 1 and minus 1? 00:37:01.600 --> 00:37:02.370 OK. 00:37:02.370 --> 00:37:04.020 How do I check the trace? 00:37:04.020 --> 00:37:07.640 What is the trace of that matrix? 00:37:07.640 --> 00:37:08.350 0. 00:37:08.350 --> 00:37:10.630 And what's the sum of the eigenvalues-- 00:37:10.630 --> 00:37:11.450 0. 00:37:11.450 --> 00:37:12.570 Good. 00:37:12.570 --> 00:37:13.710 What about determinant? 00:37:13.710 --> 00:37:15.380 What's the determinant of S? 00:37:15.380 --> 00:37:16.290 AUDIENCE: Minus 1. 00:37:16.290 --> 00:37:17.290 GILBERT STRANG: Minus 1. 00:37:17.290 --> 00:37:19.290 The product of the eigenvalues-- minus 1. 00:37:19.290 --> 00:37:20.590 So we've got it. 00:37:20.590 --> 00:37:21.480 OK. 00:37:21.480 --> 00:37:24.820 What are the eigenvectors? 00:37:24.820 --> 00:37:29.070 What vector can you multiply by and it 00:37:29.070 --> 00:37:32.010 doesn't change direction-- in fact, doesn't change at all? 00:37:32.010 --> 00:37:35.735 I'm looking for the eigenvector that's a steady state? 00:37:35.735 --> 00:37:37.180 AUDIENCE: 0, 1? 00:37:37.180 --> 00:37:38.115 GILBERT STRANG: 0, 1? 00:37:38.115 --> 00:37:41.090 AUDIENCE: 1, 1. 00:37:41.090 --> 00:37:42.780 GILBERT STRANG: I think it's 1, 1. 00:37:42.780 --> 00:37:43.280 Yeah. 00:37:43.280 --> 00:37:45.380 So here is the lambdas. 00:37:45.380 --> 00:37:47.420 And then the eigenvectors are-- 00:37:47.420 --> 00:37:48.800 I think 1, 1. 00:37:51.670 --> 00:37:52.760 Is that right? 00:37:52.760 --> 00:37:53.260 Yeah. 00:37:53.260 --> 00:37:54.190 Sure. 00:37:54.190 --> 00:37:57.010 S is just a permutation here. 00:37:57.010 --> 00:37:59.230 It's just exchanging the two entries. 00:37:59.230 --> 00:38:01.770 So 1 and 1 won't change. 00:38:01.770 --> 00:38:04.378 And what's the other eigenvector? 00:38:04.378 --> 00:38:05.940 AUDIENCE: Minus 1? 00:38:05.940 --> 00:38:07.190 GILBERT STRANG: 1 and minus 1. 00:38:15.220 --> 00:38:19.090 And then, I'm thinking-- remembering about this similar 00:38:19.090 --> 00:38:20.140 stuff-- 00:38:20.140 --> 00:38:27.610 I'm thinking that S is similar to a matrix that 00:38:27.610 --> 00:38:29.710 just shows the eigenvalues. 00:38:29.710 --> 00:38:31.690 So S is similar to-- 00:38:31.690 --> 00:38:34.410 I'm going to put in an M-- 00:38:34.410 --> 00:38:37.960 well, I'm going to connect S-- that matrix-- 00:38:37.960 --> 00:38:45.160 with the eigenvalue matrix, which has the eigenvalues. 00:38:45.160 --> 00:38:48.430 So here is my-- 00:38:50.950 --> 00:38:53.770 everybody calls that matrix capital lambda, 00:38:53.770 --> 00:38:57.550 because everybody calls the eigenvalues little lambda. 00:38:57.550 --> 00:39:02.330 So the matrix that has them is called capital lambda. 00:39:02.330 --> 00:39:06.610 And I-- my claim is that these guys are similar-- 00:39:06.610 --> 00:39:09.650 that this matrix, S, that you're seeing up there-- 00:39:09.650 --> 00:39:12.890 I believe there is an M I believe 00:39:12.890 --> 00:39:15.650 there is an M. So that S-- 00:39:15.650 --> 00:39:17.510 what did I put in here? 00:39:17.510 --> 00:39:19.370 So I'm following this pattern. 00:39:19.370 --> 00:39:24.280 I believe that there would be an M and an M inverse, 00:39:24.280 --> 00:39:27.590 so that this would mean that. 00:39:27.590 --> 00:39:29.870 And that's nice. 00:39:29.870 --> 00:39:33.530 First of all, it would confirm that the eigenvalues 00:39:33.530 --> 00:39:38.210 stay the same, which was certain to happen. 00:39:38.210 --> 00:39:43.720 And then it would also mean that I had got a diagonal matrix. 00:39:43.720 --> 00:39:45.520 And of course, that's a natural goal-- 00:39:45.520 --> 00:39:46.945 to get a diagonal matrix. 00:39:49.540 --> 00:39:52.210 So we might hope that the M that gets us there 00:39:52.210 --> 00:39:57.510 is like an important matrix. 00:39:57.510 --> 00:39:59.520 So do you see what I'm doing here? 00:39:59.520 --> 00:40:04.350 It comes under the heading of diagonalizing a matrix. 00:40:04.350 --> 00:40:08.840 I start with a matrix, S. I find it's eigenvalues. 00:40:08.840 --> 00:40:11.420 They go on into lambda. 00:40:11.420 --> 00:40:19.980 And I believe I can find an M, so that I see they're similar. 00:40:19.980 --> 00:40:23.660 They have the same eigenvalues, 1 and minus 1, both sides. 00:40:23.660 --> 00:40:27.650 So only remaining question is, what's M? 00:40:27.650 --> 00:40:33.262 What's the matrix that diagonalizes S? 00:40:33.262 --> 00:40:35.745 The-- what have we got left to use? 00:40:35.745 --> 00:40:36.870 AUDIENCE: The eigenvectors. 00:40:36.870 --> 00:40:39.140 GILBERT STRANG: The eigenvectors. 00:40:39.140 --> 00:40:43.010 The matrix that-- so, can I put the M over there? 00:40:45.530 --> 00:40:46.460 Yeah. 00:40:46.460 --> 00:40:48.590 I'll put-- that M inverse is going 00:40:48.590 --> 00:40:52.180 to go over to the other side. 00:40:52.180 --> 00:40:52.680 Oh. 00:40:52.680 --> 00:40:54.800 It goes here, doesn't it? 00:40:54.800 --> 00:40:56.240 I was worried there. 00:40:56.240 --> 00:40:58.630 It didn't look good, but yeah. 00:40:58.630 --> 00:41:02.192 So this is all going to be right, if-- 00:41:07.990 --> 00:41:09.460 this is what I'd like to have-- 00:41:09.460 --> 00:41:11.980 SM equal M lambda. 00:41:11.980 --> 00:41:14.810 SM equal M lambda. 00:41:14.810 --> 00:41:17.120 That's diagonalizing a matrix. 00:41:17.120 --> 00:41:22.390 That's finding the M using the eigenvectors. 00:41:22.390 --> 00:41:25.850 That produces a similar matrix lambda, 00:41:25.850 --> 00:41:27.270 which has the eigenvalues. 00:41:27.270 --> 00:41:35.810 That's the great fact about diagonalizing. 00:41:35.810 --> 00:41:38.160 That's how you use-- that's another way to say, 00:41:38.160 --> 00:41:40.610 this is how the eigenvectors pay off. 00:41:40.610 --> 00:41:43.880 You put them into M. You take the similar matrix 00:41:43.880 --> 00:41:45.830 and it's nice and diagonal. 00:41:45.830 --> 00:41:47.970 And do you see that this will happen? 00:41:47.970 --> 00:41:51.410 S times-- so M has the first eigenvector 00:41:51.410 --> 00:41:53.870 and the second eigenvector. 00:41:53.870 --> 00:41:59.690 And I believe that first eigenvector times the second-- 00:41:59.690 --> 00:42:03.660 and the second eigenvector-- that's M again, on this side. 00:42:03.660 --> 00:42:08.375 Let me just write in 1, 0, 0, minus 1. 00:42:13.110 --> 00:42:17.620 I believe is has got to be confirming that we've 00:42:17.620 --> 00:42:19.510 done the thing right-- 00:42:19.510 --> 00:42:22.030 confirming that the eigenvectors work here. 00:42:24.550 --> 00:42:27.410 Please make sense out of that last line. 00:42:30.060 --> 00:42:33.180 When you see that last line, what do I 00:42:33.180 --> 00:42:35.460 mean to make sense out of it? 00:42:35.460 --> 00:42:37.860 I want to see that that's true. 00:42:37.860 --> 00:42:39.180 How do I see that-- 00:42:39.180 --> 00:42:40.770 how do I do this-- 00:42:40.770 --> 00:42:43.050 so what's the left side and what's the right side? 00:42:47.850 --> 00:42:50.910 So what-- if I multiply S by a couple 00:42:50.910 --> 00:42:54.013 of columns, what's the answer? 00:42:54.013 --> 00:42:55.615 AUDIENCE: Sx1 and Sx2. 00:42:55.615 --> 00:42:56.920 GILBERT STRANG: Sx1 and Sx2. 00:42:56.920 --> 00:42:59.590 That's the beauty of matrix multiplication. 00:42:59.590 --> 00:43:02.090 If I multiply a matrix by another matrix, 00:43:02.090 --> 00:43:05.400 I can do it a column at a time. 00:43:05.400 --> 00:43:07.690 There are four great ways to multiply matrices, 00:43:07.690 --> 00:43:10.900 so this is another one-- 00:43:10.900 --> 00:43:12.010 a column at a time. 00:43:12.010 --> 00:43:16.810 So this left hand side is Sx1, Sx2. 00:43:16.810 --> 00:43:19.810 I just do each column. 00:43:19.810 --> 00:43:23.030 And what about the right hand side? 00:43:23.030 --> 00:43:25.404 I can do that multiplication. 00:43:25.404 --> 00:43:26.730 AUDIENCE: X1 minus x2. 00:43:26.730 --> 00:43:29.860 GILBERT STRANG: X1 minus x2 did somebody say? 00:43:29.860 --> 00:43:31.840 Death. 00:43:31.840 --> 00:43:32.340 No. 00:43:32.340 --> 00:43:33.520 I don't want-- Oh, x1-- 00:43:33.520 --> 00:43:34.020 sorry. 00:43:34.020 --> 00:43:35.360 You said it right. 00:43:35.360 --> 00:43:36.330 OK. 00:43:36.330 --> 00:43:39.120 When you said x1 minus x2, I was subtracting. 00:43:39.120 --> 00:43:42.660 But you meant that that's-- the first column is x1, 00:43:42.660 --> 00:43:44.820 and the second column is minus x2. 00:43:44.820 --> 00:43:45.758 Correct. 00:43:45.758 --> 00:43:46.734 Sorry about that. 00:43:50.640 --> 00:43:53.170 And did we come out right? 00:43:53.170 --> 00:43:54.310 Yes. 00:43:54.310 --> 00:43:56.140 Of course, now I compare. 00:43:56.140 --> 00:43:59.990 Sx1 is lambda one x1. 00:43:59.990 --> 00:44:02.860 Sx2 is lambda two x2. 00:44:02.860 --> 00:44:03.520 And I'm golden. 00:44:08.040 --> 00:44:12.310 So what was the point of this board? 00:44:12.310 --> 00:44:15.640 What did we learn? 00:44:15.640 --> 00:44:20.750 We learned-- well, we kind of expected that the original S 00:44:20.750 --> 00:44:26.690 would be similar to the lambdas, because the eigenvalues match. 00:44:26.690 --> 00:44:28.790 S has eigenvalues lambda. 00:44:28.790 --> 00:44:30.860 And this diagonal matrix certainly 00:44:30.860 --> 00:44:33.260 has eigenvalues 1n minus 1. 00:44:33.260 --> 00:44:35.210 A diagonal matrix-- the eigenvalues 00:44:35.210 --> 00:44:37.050 are right in front of you. 00:44:37.050 --> 00:44:38.280 So they're similar. 00:44:38.280 --> 00:44:40.560 S is similar to the lambda. 00:44:40.560 --> 00:44:44.310 And there should be an M. And then somebody suggested, maybe 00:44:44.310 --> 00:44:46.260 the M is the eigenvectors. 00:44:46.260 --> 00:44:48.550 And that's the right answer. 00:44:48.550 --> 00:44:53.070 So finally, let me write that conclusion here-- 00:44:55.730 --> 00:44:59.650 which isn't just for symmetric matrices. 00:44:59.650 --> 00:45:04.710 So maybe I should put it for matrix A. 00:45:04.710 --> 00:45:12.480 So if it has lambdas and eigenvectors, 00:45:12.480 --> 00:45:22.300 and the claim is that A times the eigenvector matrix 00:45:22.300 --> 00:45:28.793 is the eigenvector matrix times the eigenvalues. 00:45:34.220 --> 00:45:39.750 And I would shorten that to Ax equals x lambda. 00:45:42.310 --> 00:45:44.200 And I could rewrite that, and then I'll 00:45:44.200 --> 00:45:49.270 slow down, as A equal x lambda x inverse. 00:45:58.120 --> 00:46:00.040 Really, this is bringing it all together 00:46:00.040 --> 00:46:02.740 in a simple, small formula. 00:46:02.740 --> 00:46:07.360 It's telling us that A is similar to lambda. 00:46:07.360 --> 00:46:10.360 It's telling us the matrix M, that does the job-- 00:46:10.360 --> 00:46:13.390 it's a matrix of eigenvectors. 00:46:13.390 --> 00:46:21.580 And so it's like a shorthand way to write the main fact 00:46:21.580 --> 00:46:24.390 about eigenvalues and eigenvectors. 00:46:24.390 --> 00:46:26.470 What about A squared? 00:46:26.470 --> 00:46:28.840 Can I go back to the very first-- 00:46:28.840 --> 00:46:31.510 I see time is close to the end here. 00:46:31.510 --> 00:46:33.850 What about A squared? 00:46:33.850 --> 00:46:36.810 What are the eigenvectors of A squared? 00:46:36.810 --> 00:46:39.410 What are the eigenvalues of A squared? 00:46:39.410 --> 00:46:42.700 That's like the whole point of eigenvalues. 00:46:42.700 --> 00:46:45.160 Well, or I could just square that stupid thing. 00:46:45.160 --> 00:46:50.140 X lambda, x inverse, x lambda, x inverse. 00:46:50.140 --> 00:46:52.600 And what have I got? 00:46:52.600 --> 00:46:54.730 X inverse, x in the middle is-- 00:46:54.730 --> 00:46:55.570 AUDIENCE: Identity. 00:46:55.570 --> 00:46:58.220 GILBERT STRANG: Identity. 00:46:58.220 --> 00:47:03.010 So I have x, lambda squared, x inverse. 00:47:03.010 --> 00:47:06.910 And to me and to you that says, the eigenvalues 00:47:06.910 --> 00:47:07.810 have been squared. 00:47:07.810 --> 00:47:10.840 The eigenvectors didn't change. 00:47:10.840 --> 00:47:11.740 Yeah. 00:47:11.740 --> 00:47:12.300 OK. 00:47:12.300 --> 00:47:15.250 And now finally, last breath is, what 00:47:15.250 --> 00:47:18.280 if the matrix is symmetric? 00:47:18.280 --> 00:47:19.870 Then we have different letters. 00:47:19.870 --> 00:47:23.850 That's the only-- that's the significant change. 00:47:23.850 --> 00:47:30.310 The eigenvector matrix is now an orthogonal matrix. 00:47:30.310 --> 00:47:34.850 I'm coming back to the key fact of what makes symmetric-- 00:47:34.850 --> 00:47:35.930 how do I read-- 00:47:35.930 --> 00:47:39.350 how do I see symmetric helping me in the eigenvector 00:47:39.350 --> 00:47:41.180 and eigenvalue world? 00:47:41.180 --> 00:47:46.960 Well, it tells me that the eigenvectors are orthogonal. 00:47:46.960 --> 00:47:53.140 So the x is Q. The eigenvalues are real. 00:47:53.140 --> 00:47:56.880 And the eigenvectors is x inverse. 00:47:56.880 --> 00:48:00.910 But now I'm going to make those eigenvectors unit vectors. 00:48:00.910 --> 00:48:02.230 I'm going to normalize it. 00:48:02.230 --> 00:48:04.330 So I'm really allowing-- 00:48:04.330 --> 00:48:08.020 I have an orthogonal matrix Q. So I have a different way 00:48:08.020 --> 00:48:10.780 to write this, and this is the end of the-- 00:48:10.780 --> 00:48:12.370 today's class. 00:48:12.370 --> 00:48:15.430 Q lambda. 00:48:15.430 --> 00:48:18.355 And what can you tell me about Q inverse? 00:48:18.355 --> 00:48:19.480 AUDIENCE: It's Q transpose. 00:48:19.480 --> 00:48:20.630 GILBERT STRANG: It's Q transpose. 00:48:20.630 --> 00:48:21.130 Thanks. 00:48:21.130 --> 00:48:23.240 So that was the last lecture. 00:48:23.240 --> 00:48:27.490 So now the orthogonal lecture is coming up 00:48:27.490 --> 00:48:31.600 at the last second of the symmetric matrices lecture. 00:48:31.600 --> 00:48:35.380 And this has the name spectral theorem, 00:48:35.380 --> 00:48:36.730 which I'll just put there. 00:48:40.560 --> 00:48:46.640 And the whole point is that it tells you 00:48:46.640 --> 00:48:50.430 what every symmetric matrix looks like-- 00:48:50.430 --> 00:48:54.965 orthogonal eigenvectors, real eigenvalues.