WEBVTT 00:00:10.510 --> 00:00:11.010 OK. 00:00:11.010 --> 00:00:11.635 Shall we start? 00:00:11.635 --> 00:00:15.360 This is the second lecture on eigenvalues. 00:00:15.360 --> 00:00:18.350 So the first lecture was -- 00:00:18.350 --> 00:00:22.350 reached the key equation, A x equal lambda x. 00:00:22.350 --> 00:00:27.090 x is the eigenvector and lambda's the eigenvalue. 00:00:27.090 --> 00:00:30.220 Now to use that. 00:00:30.220 --> 00:00:36.330 And the, the good way to, after we've found -- 00:00:36.330 --> 00:00:39.400 so, so job one is to find the eigenvalues 00:00:39.400 --> 00:00:41.990 and find the eigenvectors. 00:00:41.990 --> 00:00:44.750 Now after we've found them, what do we do with them? 00:00:44.750 --> 00:00:50.360 Well, the good way to see that is diagonalize the matrix. 00:00:50.360 --> 00:00:52.850 So the matrix is A. 00:00:52.850 --> 00:00:55.130 And I want to show -- first of all, 00:00:55.130 --> 00:00:57.190 this is like the basic fact. 00:00:57.190 --> 00:00:58.720 This, this formula. 00:00:58.720 --> 00:01:03.210 That's, that's the key to today's lecture. 00:01:03.210 --> 00:01:06.810 This matrix A, I put its eigenvectors 00:01:06.810 --> 00:01:09.660 in the columns of a matrix S. 00:01:09.660 --> 00:01:12.740 So S will be the eigenvector matrix. 00:01:12.740 --> 00:01:19.560 And I want to look at this magic combination S inverse A S. 00:01:19.560 --> 00:01:22.830 So can I show you how that -- 00:01:22.830 --> 00:01:24.200 what happens there? 00:01:24.200 --> 00:01:27.900 And notice, there's an S inverse. 00:01:27.900 --> 00:01:33.690 We have to be able to invert this eigenvector matrix S. 00:01:33.690 --> 00:01:38.330 So for that, we need n independent eigenvectors. 00:01:38.330 --> 00:01:41.440 So that's the, that's the case. 00:01:41.440 --> 00:01:42.130 OK. 00:01:42.130 --> 00:01:53.740 So suppose we have n linearly independent eigenvectors 00:01:53.740 --> 00:02:11.830 of A. Put them in the columns of this matrix S. 00:02:11.830 --> 00:02:15.430 So I'm naturally going to call that the eigenvector matrix, 00:02:15.430 --> 00:02:18.730 because it's got the eigenvectors in its columns. 00:02:18.730 --> 00:02:21.730 And all I want to do is show you what happens 00:02:21.730 --> 00:02:24.960 when you multiply A times S. 00:02:24.960 --> 00:02:29.200 So A times S. 00:02:29.200 --> 00:02:34.490 So this is A times the matrix with the first eigenvector 00:02:34.490 --> 00:02:37.300 in its first column, the second eigenvector 00:02:37.300 --> 00:02:42.510 in its second column, the n-th eigenvector in its n-th column. 00:02:42.510 --> 00:02:46.300 And how I going to do this matrix multiplication? 00:02:46.300 --> 00:02:49.890 Well, certainly I'll do it a column at a time. 00:02:49.890 --> 00:02:52.510 And what do I get. 00:02:52.510 --> 00:02:55.480 A times the first column gives me the first column 00:02:55.480 --> 00:02:57.820 of the answer, but what is it? 00:02:57.820 --> 00:03:00.010 That's an eigenvector. 00:03:00.010 --> 00:03:04.280 A times x1 is equal to the lambda times the x1. 00:03:04.280 --> 00:03:06.870 And that lambda's we're -- we'll call lambda one, 00:03:06.870 --> 00:03:09.590 of course. 00:03:09.590 --> 00:03:11.420 So that's the first column. 00:03:11.420 --> 00:03:14.000 Ax1 is the same as lambda one x1. 00:03:14.000 --> 00:03:16.730 A x2 is lambda two x2. 00:03:16.730 --> 00:03:22.120 So on, along to in the n-th column we now how lambda n xn. 00:03:24.690 --> 00:03:27.110 Looking good, but the next step is even 00:03:27.110 --> 00:03:28.120 better. 00:03:28.120 --> 00:03:31.280 So for the next step, I want to separate out 00:03:31.280 --> 00:03:36.100 those eigenvalues, those, those multiplying numbers, 00:03:36.100 --> 00:03:38.880 from the x-s. 00:03:38.880 --> 00:03:41.920 So then I'll have just what I want. 00:03:41.920 --> 00:03:42.530 OK. 00:03:42.530 --> 00:03:44.990 So how, how I going to separate out? 00:03:44.990 --> 00:03:47.150 So that, that number lambda one is 00:03:47.150 --> 00:03:49.760 multiplying the first column. 00:03:49.760 --> 00:03:52.840 So if I want to factor it out of the first column, 00:03:52.840 --> 00:03:55.400 I better put -- 00:03:55.400 --> 00:03:58.830 here is going to be x1, and that's 00:03:58.830 --> 00:04:02.180 going to multiply this matrix lambda 00:04:02.180 --> 00:04:05.050 one in the first entry and all zeros. 00:04:05.050 --> 00:04:08.230 Do you see that that, that's going to come out 00:04:08.230 --> 00:04:10.220 right for the first column? 00:04:10.220 --> 00:04:13.370 Because w- we remember how -- how we're going back to that 00:04:13.370 --> 00:04:15.310 original punchline. 00:04:15.310 --> 00:04:21.260 That if I want a number to multiply x1 then 00:04:21.260 --> 00:04:24.310 I can do it by putting x1 in that column, 00:04:24.310 --> 00:04:27.280 in the first column, and putting that number there. 00:04:27.280 --> 00:04:28.850 Th- u- what I going to have here? 00:04:28.850 --> 00:04:30.040 I'm going to have lambda -- 00:04:30.040 --> 00:04:33.800 I'm going to have x1, x2, ... 00:04:33.800 --> 00:04:34.890 ,xn. 00:04:34.890 --> 00:04:37.150 These are going to be my columns again. 00:04:37.150 --> 00:04:39.740 I'm getting S back again. 00:04:39.740 --> 00:04:41.380 I'm getting S back again. 00:04:41.380 --> 00:04:44.560 But now what's it multiplied by, on the right it's 00:04:44.560 --> 00:04:48.080 multiplied by? 00:04:48.080 --> 00:04:54.730 If I want lambda n xn in the last column, how do I do it? 00:04:54.730 --> 00:04:57.670 Well, the last column here will be -- 00:04:57.670 --> 00:05:02.630 I'll take the last column, use these coefficients, 00:05:02.630 --> 00:05:05.050 put the lambda n down there, and it 00:05:05.050 --> 00:05:09.970 will multiply that n-th column and give me lambda n xn. 00:05:09.970 --> 00:05:13.130 There, there you see matrix multiplication just working 00:05:13.130 --> 00:05:14.850 for us. 00:05:14.850 --> 00:05:17.410 So I started with A S. 00:05:17.410 --> 00:05:21.010 I wrote down what it meant, A times each eigenvector. 00:05:21.010 --> 00:05:23.690 That gave me lambda time the eigenvector. 00:05:23.690 --> 00:05:25.700 And then when I peeled off the lambdas, 00:05:25.700 --> 00:05:30.690 they were on the right-hand side, so I've got S, my matrix, 00:05:30.690 --> 00:05:31.750 back again. 00:05:31.750 --> 00:05:45.500 And this matrix, this diagonal matrix, the eigenvalue matrix, 00:05:45.500 --> 00:05:49.210 and I call it capital lambda. 00:05:49.210 --> 00:05:52.830 Using capital letters for matrices and lambda 00:05:52.830 --> 00:05:54.980 to prompt me that it's, that it's 00:05:54.980 --> 00:05:56.850 eigenvalues that are in there. 00:05:56.850 --> 00:05:58.840 So you see that the eigenvalues are just 00:05:58.840 --> 00:06:01.480 sitting down that diagonal? 00:06:01.480 --> 00:06:07.340 If I had a column x2 here, I would want the lambda two 00:06:07.340 --> 00:06:10.970 in the two two position, in the diagonal position, 00:06:10.970 --> 00:06:15.140 to multiply that x2 and give me the lambda two x2. 00:06:15.140 --> 00:06:16.730 That's my formula. 00:06:16.730 --> 00:06:19.770 A S is S lambda. 00:06:23.000 --> 00:06:23.720 OK. 00:06:23.720 --> 00:06:27.530 That's the -- you see, it's just a calculation. 00:06:27.530 --> 00:06:31.390 Now -- I mentioned, and I have to mention again, 00:06:31.390 --> 00:06:37.210 this business about n independent eigenvectors. 00:06:37.210 --> 00:06:40.050 As it stands, this is all fine, whether -- 00:06:40.050 --> 00:06:43.240 I mean, I could be repeating the same eigenvector, but -- 00:06:43.240 --> 00:06:45.550 I'm not interested in that. 00:06:45.550 --> 00:06:50.600 I want to be able to invert S, and that's where this comes in. 00:06:50.600 --> 00:06:53.690 This n independent eigenvectors business 00:06:53.690 --> 00:06:56.990 comes in to tell me that that matrix is invertible. 00:06:56.990 --> 00:07:01.380 So let me, on the next board, write down what I've got. 00:07:01.380 --> 00:07:06.780 A S equals S lambda. 00:07:06.780 --> 00:07:11.890 And now I'm, I can multiply on the left by S inverse. 00:07:11.890 --> 00:07:13.839 So this is really -- 00:07:17.970 --> 00:07:20.660 I can do that, provided S is invertible. 00:07:20.660 --> 00:07:26.320 Provided my assumption of n independent eigenvectors is 00:07:26.320 --> 00:07:27.260 satisfied. 00:07:27.260 --> 00:07:31.470 And I mentioned at the end of last time, and I'll say again, 00:07:31.470 --> 00:07:35.930 that there's a small number of matrices for -- 00:07:35.930 --> 00:07:38.680 that don't have n independent eigenvectors. 00:07:38.680 --> 00:07:42.580 So I've got to discuss that, that technical point. 00:07:42.580 --> 00:07:48.030 But the great -- the most matrices that we see have n di- 00:07:48.030 --> 00:07:51.050 n independent eigenvectors, and we can diagonalize. 00:07:51.050 --> 00:07:54.090 This is diagonalization. 00:07:54.090 --> 00:07:57.890 I could also write it, and I often will, 00:07:57.890 --> 00:07:59.680 the other way round. 00:07:59.680 --> 00:08:03.140 If I multiply on the right by S inverse, 00:08:03.140 --> 00:08:06.130 if I took this equation at the top and multiplied on the right 00:08:06.130 --> 00:08:08.090 by S inverse, I could -- 00:08:08.090 --> 00:08:10.900 I would have A left here. 00:08:10.900 --> 00:08:15.790 Now S inverse is coming from the right. 00:08:15.790 --> 00:08:18.530 So can you keep those two straight? 00:08:18.530 --> 00:08:21.491 A multiplies its eigenvectors, that's how I keep them 00:08:21.491 --> 00:08:21.990 straight. 00:08:21.990 --> 00:08:24.530 So A multiplies S. 00:08:24.530 --> 00:08:26.020 A multiplies S. 00:08:26.020 --> 00:08:30.700 And then this S inverse makes the whole thing diagonal. 00:08:30.700 --> 00:08:33.750 And this is another way of saying the same thing, 00:08:33.750 --> 00:08:36.700 putting the Ss on the other side of the equation. 00:08:36.700 --> 00:08:40.179 A is S lambda S inverse. 00:08:40.179 --> 00:08:44.520 So that's the, that's the new factorization. 00:08:44.520 --> 00:08:50.550 That's the replacement for L U from elimination or Q R for -- 00:08:50.550 --> 00:08:52.210 from Gram-Schmidt. 00:08:52.210 --> 00:08:56.850 And notice that the matrix -- so it's, it's a matrix times 00:08:56.850 --> 00:09:00.420 a diagonal matrix times the inverse of the first one. 00:09:00.420 --> 00:09:02.410 It's, that's the combination that we'll 00:09:02.410 --> 00:09:05.530 see throughout this chapter. 00:09:05.530 --> 00:09:09.650 This combination with an S and an S inverse. 00:09:09.650 --> 00:09:10.250 OK. 00:09:10.250 --> 00:09:14.080 Can I just begin to use that? 00:09:14.080 --> 00:09:17.730 For example, what about A squared? 00:09:17.730 --> 00:09:20.880 What are the eigenvalues and eigenvectors of A squared? 00:09:20.880 --> 00:09:23.040 That's a straightforward question with a, 00:09:23.040 --> 00:09:25.380 with an absolutely clean answer. 00:09:25.380 --> 00:09:28.930 So let me, let me consider A squared. 00:09:28.930 --> 00:09:33.560 So I start with A x equal lambda x. 00:09:33.560 --> 00:09:36.070 And I'm headed for A squared. 00:09:36.070 --> 00:09:38.610 So let me multiply both sides by A. 00:09:38.610 --> 00:09:41.290 That's one way to get A squared on the left. 00:09:41.290 --> 00:09:46.190 So -- I should write these if-s in here. 00:09:46.190 --> 00:09:51.010 If A x equals lambda x, then I multiply by A, 00:09:51.010 --> 00:09:56.200 so I get A squared x equals -- well, I'm multiplying by A, 00:09:56.200 --> 00:09:58.670 so that's lambda A x. 00:09:58.670 --> 00:10:02.520 That lambda was a number, so I just put it on the left. 00:10:02.520 --> 00:10:06.740 And what do I -- tell me how to make that look better. 00:10:06.740 --> 00:10:10.220 What have I got here for if, if A 00:10:10.220 --> 00:10:13.220 has the eigenvalue lambda and eigenvector 00:10:13.220 --> 00:10:16.010 x, what's up with A squared? 00:10:16.010 --> 00:10:18.430 A squared x, I just multiplied by A, 00:10:18.430 --> 00:10:23.070 but now for Ax I'm going to substitute lambda x. 00:10:23.070 --> 00:10:25.010 So I've got lambda squared x. 00:10:28.120 --> 00:10:31.370 So from that simple calculation, I -- 00:10:31.370 --> 00:10:37.330 my conclusion is that the eigenvalues of A squared are 00:10:37.330 --> 00:10:39.310 lambda squared. 00:10:39.310 --> 00:10:40.384 And the eigenvectors -- 00:10:40.384 --> 00:10:41.550 I always think about both of 00:10:41.550 --> 00:10:42.150 those. 00:10:42.150 --> 00:10:44.390 What can I say about the eigenvalues? 00:10:44.390 --> 00:10:45.560 They're squared. 00:10:45.560 --> 00:10:48.220 What can I say about the eigenvectors? 00:10:48.220 --> 00:10:49.730 They're the same. 00:10:49.730 --> 00:10:54.090 The same x as in -- as for A. 00:10:54.090 --> 00:11:00.320 Now let me see that also from this formula. 00:11:00.320 --> 00:11:05.160 How can I see what A squared is looking like from this formula? 00:11:05.160 --> 00:11:08.480 So let me -- that was one way to do it. 00:11:08.480 --> 00:11:12.510 Let me do it by just taking A squared from that. 00:11:12.510 --> 00:11:18.990 A squared is S lambda S inverse -- that's A -- 00:11:18.990 --> 00:11:22.620 times S lambda S inverse -- that's A, which is? 00:11:25.530 --> 00:11:30.170 This is the beauty of eigenvalues, eigenvectors. 00:11:30.170 --> 00:11:34.000 Having that S inverse and S is the identity, 00:11:34.000 --> 00:11:40.220 so I've got S lambda squared S inverse. 00:11:40.220 --> 00:11:43.380 Do you see what that's telling me? 00:11:43.380 --> 00:11:46.570 It's, it's telling me the same thing that I just learned here, 00:11:46.570 --> 00:11:49.630 but in the -- in a matrix form. 00:11:49.630 --> 00:11:52.720 It's telling me that the S is the same, 00:11:52.720 --> 00:11:57.130 the eigenvectors are the same, but the eigenvalues 00:11:57.130 --> 00:11:58.530 are squared. 00:11:58.530 --> 00:12:01.020 Because this is -- what's lambda squared? 00:12:01.020 --> 00:12:02.800 That's still diagonal. 00:12:02.800 --> 00:12:05.180 It's got little lambda one squared, 00:12:05.180 --> 00:12:07.190 lambda two squared, down to lambda n 00:12:07.190 --> 00:12:09.280 squared o- on that diagonal. 00:12:09.280 --> 00:12:13.760 Those are the eigenvalues, as we just learned, of A squared. 00:12:13.760 --> 00:12:14.500 OK. 00:12:14.500 --> 00:12:19.590 So -- somehow those eigenvalues and eigenvectors are really 00:12:19.590 --> 00:12:25.060 giving you a way to -- 00:12:25.060 --> 00:12:29.460 see what's going on inside a matrix. 00:12:29.460 --> 00:12:32.100 Of course I can continue that for -- 00:12:32.100 --> 00:12:36.270 to the K-th power, A to the K-th power. 00:12:36.270 --> 00:12:39.370 If I multiply, if I have K of these together, 00:12:39.370 --> 00:12:42.920 do you see how S inverse S will keep canceling 00:12:42.920 --> 00:12:44.900 in the, in the inside? 00:12:44.900 --> 00:12:48.740 I'll have the S outside at the far left, 00:12:48.740 --> 00:12:54.390 and lambda will be in there K times, and S inverse. 00:12:54.390 --> 00:12:56.670 So what's that telling me? 00:12:56.670 --> 00:12:59.420 That's telling me that the eigenvalues 00:12:59.420 --> 00:13:04.340 of A, of A to the K-th power are the K-th powers. 00:13:04.340 --> 00:13:08.390 The eigenvalues of A cubed are the cubes of the eigenvalues of 00:13:08.390 --> 00:13:15.000 A. And the eigenvectors are the same, the same. 00:13:15.000 --> 00:13:15.600 OK. 00:13:15.600 --> 00:13:20.750 In other words, eigenvalues and eigenvectors 00:13:20.750 --> 00:13:25.910 give a great way to understand the powers of a matrix. 00:13:25.910 --> 00:13:28.380 If I take the square of a matrix, 00:13:28.380 --> 00:13:30.540 or the hundredth power of a matrix, 00:13:30.540 --> 00:13:34.780 the pivots are all over the place. 00:13:34.780 --> 00:13:39.690 L U, if I multiply L U times L U times L U times L U 00:13:39.690 --> 00:13:44.610 a hundred times, I've got a hundred L Us. 00:13:44.610 --> 00:13:46.370 I can't do anything with them. 00:13:46.370 --> 00:13:50.180 But when I multiply S lambda S inverse by itself, 00:13:50.180 --> 00:13:54.720 when I look at the eigenvector picture a hundred times, 00:13:54.720 --> 00:13:59.810 I get a hundred or ninety-nine of these guys canceling out 00:13:59.810 --> 00:14:03.390 inside, and I get A to the hundredth 00:14:03.390 --> 00:14:05.900 is S lambda to the hundredth S inverse. 00:14:05.900 --> 00:14:09.870 I mean, eigenvalues tell you about powers 00:14:09.870 --> 00:14:16.290 of a matrix in a way that we had no way to approach previously. 00:14:16.290 --> 00:14:21.220 For example, when does -- 00:14:21.220 --> 00:14:24.790 when do the powers of a matrix go to zero? 00:14:24.790 --> 00:14:29.330 I would call that matrix stable, maybe. 00:14:29.330 --> 00:14:31.940 So I could write down a theorem. 00:14:31.940 --> 00:14:36.720 I'll write it as a theorem just to use that word 00:14:36.720 --> 00:14:40.350 to emphasize that here I'm getting this great fact 00:14:40.350 --> 00:14:42.610 from this eigenvalue picture. 00:14:42.610 --> 00:14:43.230 OK. 00:14:43.230 --> 00:14:53.720 A to the K approaches zero as K goes, as K gets bigger if what? 00:14:53.720 --> 00:14:57.840 What's the w- how can I tell, for a matrix A, 00:14:57.840 --> 00:14:59.850 if its powers go to zero? 00:15:02.650 --> 00:15:07.520 What's -- somewhere inside that matrix is that information. 00:15:07.520 --> 00:15:11.510 That information is not present in the pivots. 00:15:11.510 --> 00:15:13.270 It's present in the eigenvalues. 00:15:13.270 --> 00:15:17.010 What do I need for the -- to know that if I take higher 00:15:17.010 --> 00:15:20.170 and higher powers of A, that this matrix gets smaller 00:15:20.170 --> 00:15:21.030 and smaller? 00:15:21.030 --> 00:15:24.330 Well, S and S inverse are not moving. 00:15:24.330 --> 00:15:26.810 So it's this guy that has to get small. 00:15:26.810 --> 00:15:29.720 And that's easy to -- to understand. 00:15:29.720 --> 00:15:32.595 The requirement is all eigenvalues -- 00:15:35.330 --> 00:15:38.040 so what is the requirement? 00:15:38.040 --> 00:15:41.170 The eigenvalues have to be less than one. 00:15:41.170 --> 00:15:45.310 Now I have to wrote that absolute value, 00:15:45.310 --> 00:15:48.360 because those eigenvalues could be negative, 00:15:48.360 --> 00:15:50.550 they could be complex numbers. 00:15:50.550 --> 00:15:53.170 So I'm taking the absolute value. 00:15:53.170 --> 00:15:56.750 If all of those are below one. 00:15:56.750 --> 00:16:05.000 That's, in fact, we practically see why. 00:16:05.000 --> 00:16:13.100 And let me just say that I'm operating on one assumption 00:16:13.100 --> 00:16:15.740 here, and I got to keep remembering 00:16:15.740 --> 00:16:18.690 that that assumption is still present. 00:16:18.690 --> 00:16:21.420 That assumption was that I had a full set of, 00:16:21.420 --> 00:16:24.370 of n independent eigenvectors. 00:16:24.370 --> 00:16:30.830 If I don't have that, then this approach is not working. 00:16:30.830 --> 00:16:37.090 So again, a pure eigenvalue approach, eigenvector approach, 00:16:37.090 --> 00:16:40.470 needs n independent eigenvectors. 00:16:40.470 --> 00:16:42.900 If we don't have n independent eigenvectors, 00:16:42.900 --> 00:16:46.490 we can't diagonalize the matrix. 00:16:46.490 --> 00:16:50.820 We can't get to a diagonal matrix. 00:16:50.820 --> 00:16:55.960 This diagonalization is only possible 00:16:55.960 --> 00:16:58.581 if S inverse makes sense. 00:16:58.581 --> 00:16:59.080 OK. 00:16:59.080 --> 00:17:02.800 Can I, can I follow up on that point now? 00:17:02.800 --> 00:17:07.099 So you see why -- what we get and, and why we want it, 00:17:07.099 --> 00:17:11.490 because we get information about the powers of a matrix just 00:17:11.490 --> 00:17:14.940 immediately from the eigenvalues. 00:17:14.940 --> 00:17:15.500 OK. 00:17:15.500 --> 00:17:22.329 Now let me follow up on this, business of which matrices 00:17:22.329 --> 00:17:25.030 are diagonalizable. 00:17:25.030 --> 00:17:28.220 Sorry about that long word. 00:17:28.220 --> 00:17:32.940 So a matrix is, is sure -- so here's, here's the main point. 00:17:32.940 --> 00:17:37.600 A is sure to be -- 00:17:37.600 --> 00:17:50.110 to have N independent eigenvectors and, and be -- 00:17:50.110 --> 00:18:00.210 now here comes that word -- diagonalizable if, if -- 00:18:00.210 --> 00:18:05.810 so we might as well get the nice case out in the open. 00:18:05.810 --> 00:18:13.330 The nice case is when -- if all the lambdas are different. 00:18:19.520 --> 00:18:28.326 That means, that means no repeated eigenvalues. 00:18:30.920 --> 00:18:32.000 OK. 00:18:32.000 --> 00:18:34.410 That's the nice case. 00:18:34.410 --> 00:18:39.970 If my matrix, and most -- if I do a random matrix in Matlab 00:18:39.970 --> 00:18:43.140 and compute its eigenvalues -- 00:18:43.140 --> 00:18:55.540 so if I computed if I took eig of rand of ten ten, gave, 00:18:55.540 --> 00:18:58.640 gave that Matlab command, the -- 00:18:58.640 --> 00:19:01.190 we'd get a random ten by ten matrix, 00:19:01.190 --> 00:19:03.990 we would get a list of its ten eigenvalues, 00:19:03.990 --> 00:19:08.040 and they would be different. 00:19:08.040 --> 00:19:10.470 They would be distinct is the best word. 00:19:10.470 --> 00:19:13.880 I would have -- a random matrix will have ten distinct -- 00:19:13.880 --> 00:19:18.090 a ten by ten matrix will have ten distinct eigenvalues. 00:19:18.090 --> 00:19:25.500 And if it does, the eigenvectors are automatically independent. 00:19:25.500 --> 00:19:26.930 So that's a nice fact. 00:19:26.930 --> 00:19:29.730 I'll refer you to the text for the proof. 00:19:29.730 --> 00:19:36.360 That, that A is sure to have n independent eigenvectors 00:19:36.360 --> 00:19:41.610 if the eigenvalues are different, if. 00:19:41.610 --> 00:19:43.890 If all the, if all eigenvalues are different. 00:19:43.890 --> 00:19:47.810 It's just if some lambdas are repeated, 00:19:47.810 --> 00:19:50.560 then I have to look more closely. 00:19:50.560 --> 00:19:55.050 If an eigenvalue is repeated, I have to look, I have to count, 00:19:55.050 --> 00:19:56.260 I have to check. 00:19:56.260 --> 00:19:59.560 Has it got -- say it's repeated three times. 00:19:59.560 --> 00:20:02.030 So what's a possibility for the -- 00:20:02.030 --> 00:20:05.893 so here is the, here is the repeated possibility. 00:20:11.000 --> 00:20:16.490 And, and let me emphasize the conclusion. 00:20:16.490 --> 00:20:21.410 That if I have repeated eigenvalues, I may or may not, 00:20:21.410 --> 00:20:35.271 I may or may not have, have n independent eigenvectors. 00:20:35.271 --> 00:20:35.770 I might. 00:20:35.770 --> 00:20:41.240 I, I, you know, this isn't a completely negative case. 00:20:41.240 --> 00:20:43.260 The identity matrix -- 00:20:43.260 --> 00:20:46.380 suppose I take the ten by ten identity matrix. 00:20:46.380 --> 00:20:50.750 What are the eigenvalues of that matrix? 00:20:50.750 --> 00:20:55.370 So just, just take the easiest matrix, the identity. 00:20:55.370 --> 00:21:00.470 If I look for its eigenvalues, they're all ones. 00:21:00.470 --> 00:21:04.410 So that eigenvalue one is repeated ten times. 00:21:04.410 --> 00:21:07.340 But there's no shortage of eigenvectors for the identity 00:21:07.340 --> 00:21:08.250 matrix. 00:21:08.250 --> 00:21:10.760 In fact, every vector is an eigenvector. 00:21:10.760 --> 00:21:13.610 So I can take ten independent vectors. 00:21:13.610 --> 00:21:16.530 Oh, well, what happens to everything -- 00:21:16.530 --> 00:21:18.590 if A is the identity matrix, let's 00:21:18.590 --> 00:21:21.650 just think that one through in our head. 00:21:21.650 --> 00:21:27.440 If A is the identity matrix, then it's 00:21:27.440 --> 00:21:28.830 got plenty of eigenvectors. 00:21:28.830 --> 00:21:30.910 I choose ten independent vectors. 00:21:30.910 --> 00:21:32.560 They're the columns of S. 00:21:32.560 --> 00:21:37.380 And, and what do I get from S inverse A S? 00:21:37.380 --> 00:21:39.400 I get I again, right? 00:21:39.400 --> 00:21:42.210 If A is the identity -- and of course that's the correct 00:21:42.210 --> 00:21:43.790 lambda. 00:21:43.790 --> 00:21:46.790 The matrix was already diagonal. 00:21:46.790 --> 00:21:48.970 So if the matrix is already diagonal, 00:21:48.970 --> 00:21:53.800 then the, the lambda is the same as the matrix. 00:21:53.800 --> 00:21:56.380 A diagonal matrix has got its eigenvalues 00:21:56.380 --> 00:21:59.170 sitting right there in front of you. 00:21:59.170 --> 00:22:01.790 Now if it's triangular, the eigenvalues 00:22:01.790 --> 00:22:04.820 are still sitting there, but so let's 00:22:04.820 --> 00:22:08.460 take a case where it's triangular. 00:22:08.460 --> 00:22:14.870 Suppose A is like, two one two zero. 00:22:17.920 --> 00:22:23.290 So there's a case that's going to be trouble. 00:22:23.290 --> 00:22:25.040 There's a case that's going to be trouble. 00:22:25.040 --> 00:22:26.360 First of all, what are the -- 00:22:26.360 --> 00:22:29.410 I mean, we just -- 00:22:29.410 --> 00:22:31.190 if we start with a matrix, the first thing 00:22:31.190 --> 00:22:32.890 we do, practically without thinking 00:22:32.890 --> 00:22:36.130 is compute the eigenvalues and eigenvectors. 00:22:36.130 --> 00:22:36.630 OK. 00:22:36.630 --> 00:22:38.360 So what are the eigenvalues? 00:22:38.360 --> 00:22:41.130 You can tell me right away what they are. 00:22:41.130 --> 00:22:43.880 They're two and two, right. 00:22:43.880 --> 00:22:47.770 It's a triangular matrix, so when I do this determinant, 00:22:47.770 --> 00:22:51.680 shall I do this determinant of A minus lambda I? 00:22:51.680 --> 00:22:59.990 I'll get this two minus lambda one zero two minus lambda, 00:22:59.990 --> 00:23:01.700 right? 00:23:01.700 --> 00:23:06.890 I take that determinant, so I make those into vertical bars 00:23:06.890 --> 00:23:09.130 to mean determinant. 00:23:09.130 --> 00:23:10.810 And what's the determinant? 00:23:10.810 --> 00:23:13.140 It's two minus lambda squared. 00:23:13.140 --> 00:23:14.570 What are the roots? 00:23:14.570 --> 00:23:17.410 Lambda equal two twice. 00:23:17.410 --> 00:23:22.640 So the eigenvalues are lambda equals two and two. 00:23:22.640 --> 00:23:23.580 OK, fine. 00:23:23.580 --> 00:23:26.810 Now the next step, find the eigenvectors. 00:23:26.810 --> 00:23:31.940 So I look for eigenvectors, and what do I find for this guy? 00:23:31.940 --> 00:23:33.530 Eigenvectors for this guy, when I 00:23:33.530 --> 00:23:38.340 subtract two minus the identity, so A minus two 00:23:38.340 --> 00:23:42.280 I has zeros here. 00:23:45.420 --> 00:23:48.740 And I'm looking for the null space. 00:23:48.740 --> 00:23:50.390 What's, what are the eigenvectors? 00:23:50.390 --> 00:23:56.540 They're the -- the null space of A minus lambda I. 00:23:56.540 --> 00:23:59.200 The null space is only one dimensional. 00:23:59.200 --> 00:24:03.700 This is a case where I don't have enough eigenvectors. 00:24:03.700 --> 00:24:07.940 My algebraic multiplicity is two. 00:24:07.940 --> 00:24:10.520 I would say, when I see, when I count 00:24:10.520 --> 00:24:16.210 how often the eigenvalue is repeated, 00:24:16.210 --> 00:24:18.410 that's the algebraic multiplicity. 00:24:18.410 --> 00:24:20.650 That's the multiplicity, how many times 00:24:20.650 --> 00:24:22.770 is it the root of the polynomial? 00:24:22.770 --> 00:24:28.340 My polynomial is two minus lambda squared. 00:24:28.340 --> 00:24:30.040 It's a double root. 00:24:30.040 --> 00:24:33.240 So my algebraic multiplicity is two. 00:24:33.240 --> 00:24:37.110 But the geometric multiplicity, which looks for vectors, 00:24:37.110 --> 00:24:42.130 looks for eigenvectors, and -- which means the null space 00:24:42.130 --> 00:24:46.500 of this thing, and the only eigenvector is one 00:24:46.500 --> 00:24:47.360 zero. 00:24:47.360 --> 00:24:50.140 That's in the null space. 00:24:50.140 --> 00:24:52.600 Zero one is not in the null space. 00:24:52.600 --> 00:24:54.540 The null space is only one dimensional. 00:24:54.540 --> 00:24:58.960 So there's a matrix, my -- this A or the original A, 00:24:58.960 --> 00:25:02.310 that are not diagonalizable. 00:25:02.310 --> 00:25:06.360 I can't find two independent eigenvectors. 00:25:06.360 --> 00:25:08.090 There's only one. 00:25:08.090 --> 00:25:08.700 OK. 00:25:08.700 --> 00:25:11.710 So that's the case that I'm -- 00:25:11.710 --> 00:25:15.520 that's a case that I'm not really handling. 00:25:15.520 --> 00:25:19.590 For example, when I wrote down up here 00:25:19.590 --> 00:25:24.490 that the powers went to zero if the eigenvalues were below one, 00:25:24.490 --> 00:25:29.210 I didn't really handle that case of repeated eigenvalues, 00:25:29.210 --> 00:25:33.980 because my reasoning was based on this formula. 00:25:33.980 --> 00:25:36.420 And this formula is based on n independent eigenvectors. 00:25:36.420 --> 00:25:36.920 OK. 00:25:36.920 --> 00:25:45.610 Just to say then, there are some matrices that we're, that, 00:25:45.610 --> 00:25:48.730 that we don't cover through diagonalization, 00:25:48.730 --> 00:25:51.070 but the great majority we do. 00:25:51.070 --> 00:25:51.720 OK. 00:25:51.720 --> 00:25:54.040 And we, we're always OK if we have 00:25:54.040 --> 00:25:56.550 different distinct eigenvalues. 00:25:56.550 --> 00:26:02.390 OK, that's the, like, the typical case. 00:26:02.390 --> 00:26:04.660 Because for each eigenvalue there's 00:26:04.660 --> 00:26:07.140 at least one eigenvector. 00:26:07.140 --> 00:26:11.530 The algebraic multiplicity here is one for every eigenvalue 00:26:11.530 --> 00:26:14.080 and the geometric multiplicity is one. 00:26:14.080 --> 00:26:15.580 There's one eigenvector. 00:26:15.580 --> 00:26:17.650 And they are independent. 00:26:17.650 --> 00:26:18.150 OK. 00:26:18.150 --> 00:26:18.650 OK. 00:26:21.690 --> 00:26:26.390 Now let me come back to the important case, when, 00:26:26.390 --> 00:26:27.770 when we're OK. 00:26:27.770 --> 00:26:31.820 The important case, when we are diagonalizable. 00:26:31.820 --> 00:26:38.060 Let me, look at -- 00:26:38.060 --> 00:26:42.455 so -- let me solve this equation. 00:26:46.680 --> 00:26:49.460 The equation will be each -- 00:26:49.460 --> 00:26:57.146 I start with some -- start with a given vector u0. 00:27:02.390 --> 00:27:06.080 And then my equation is at every step, 00:27:06.080 --> 00:27:11.600 I multiply what I have by A. 00:27:11.600 --> 00:27:16.550 That, that equation ought to be simple to handle. 00:27:19.940 --> 00:27:21.940 And I'd like to be able to solve it. 00:27:21.940 --> 00:27:26.840 How would I find -- if I start with a vector u0 and I multiply 00:27:26.840 --> 00:27:31.470 by A a hundred times, what have I got? 00:27:31.470 --> 00:27:35.310 Well, I could certainly write down a formula for the answer, 00:27:35.310 --> 00:27:39.295 so what, what -- so u1 is A u0. 00:27:42.170 --> 00:27:45.800 And u2 is -- what's u2 then? 00:27:45.800 --> 00:27:52.350 u2, I multiply -- u2 I get from u1 by another multiplying by A, 00:27:52.350 --> 00:27:55.830 so I've got A twice. 00:27:55.830 --> 00:28:02.120 And my formula is uk, after k steps, 00:28:02.120 --> 00:28:07.580 I've multiplied by A k times the original u0. 00:28:07.580 --> 00:28:11.220 You see what I'm doing? 00:28:11.220 --> 00:28:14.050 The next section is going to solve systems 00:28:14.050 --> 00:28:17.550 of differential equations. 00:28:17.550 --> 00:28:19.690 I'm going to have derivatives. 00:28:19.690 --> 00:28:23.370 This section is the nice one. 00:28:23.370 --> 00:28:26.190 It solves difference equations. 00:28:26.190 --> 00:28:28.550 I would call that a difference equation. 00:28:28.550 --> 00:28:33.550 It's -- at first order, I would call that a first-order system, 00:28:33.550 --> 00:28:40.150 because it connects only -- it only goes up one level. 00:28:40.150 --> 00:28:43.180 And I -- it's a system because these are vectors 00:28:43.180 --> 00:28:45.960 and that's a matrix. 00:28:45.960 --> 00:28:48.470 And the solution is just that. 00:28:48.470 --> 00:28:49.090 OK. 00:28:49.090 --> 00:28:55.160 But, that's a nice formula. 00:28:55.160 --> 00:28:57.500 That's the, like, the most compact formula 00:28:57.500 --> 00:29:01.630 I could ever get. u100 would be A to the one hundred u0. 00:29:01.630 --> 00:29:06.480 But how would I actually find u100? 00:29:06.480 --> 00:29:11.520 How would I find -- how would I discover what u100 is? 00:29:11.520 --> 00:29:13.760 Let me, let me show you how. 00:29:16.620 --> 00:29:18.630 Here's the idea. 00:29:18.630 --> 00:29:23.090 If -- so to solve, to really solve -- shall I say, 00:29:23.090 --> 00:29:26.920 to really solve -- 00:29:26.920 --> 00:29:34.500 to really solve it, I would take this initial vector u0 00:29:34.500 --> 00:29:39.420 and I would write it as a combination of eigenvectors. 00:29:39.420 --> 00:29:47.320 To really solve, write u nought as a combination, 00:29:47.320 --> 00:29:50.660 say certain amount of the first eigenvector 00:29:50.660 --> 00:29:53.480 plus a certain amount of the second eigenvector 00:29:53.480 --> 00:29:55.820 plus a certain amount of the last eigenvector. 00:30:01.790 --> 00:30:04.740 Now multiply by A. 00:30:04.740 --> 00:30:07.350 You want to -- you got to see the magic of eigenvectors 00:30:07.350 --> 00:30:08.520 working here. 00:30:08.520 --> 00:30:10.360 Multiply by A. 00:30:10.360 --> 00:30:13.910 So Au0 is what? 00:30:13.910 --> 00:30:16.930 So A times that. 00:30:16.930 --> 00:30:18.800 A times -- so what's A -- 00:30:18.800 --> 00:30:21.390 I can separate it out into n separate pieces, 00:30:21.390 --> 00:30:23.430 and that's the whole point. 00:30:23.430 --> 00:30:28.800 That each of those pieces is going in its own merry way. 00:30:28.800 --> 00:30:31.320 Each of those pieces is an eigenvector, 00:30:31.320 --> 00:30:35.810 and when I multiply by A, what does this piece become? 00:30:35.810 --> 00:30:38.450 So that's some amount of the first -- 00:30:38.450 --> 00:30:41.030 let's suppose the eigenvectors are normalized to be unit 00:30:41.030 --> 00:30:41.530 vectors. 00:30:44.750 --> 00:30:48.530 So that says what the eigenvector is. 00:30:48.530 --> 00:30:51.340 It's a -- 00:30:51.340 --> 00:30:55.220 And I need some multiple of it to produce u0. 00:30:55.220 --> 00:30:56.120 OK. 00:30:56.120 --> 00:30:59.470 Now when I multiply by A, what do I get? 00:30:59.470 --> 00:31:04.350 I get c1, which is just a factor, times Ax1, 00:31:04.350 --> 00:31:07.865 but Ax1 is lambda one x1. 00:31:10.780 --> 00:31:17.060 When I multiply this by A, I get c2 lambda two x2. 00:31:17.060 --> 00:31:20.740 And here I get cn lambda n xn. 00:31:20.740 --> 00:31:27.980 And suppose I multiply by A to the hundredth power now. 00:31:27.980 --> 00:31:30.840 Can we, having done it, multiplied by A, let's 00:31:30.840 --> 00:31:32.890 multiply by A to the hundredth. 00:31:32.890 --> 00:31:36.380 What happens to this first term when I multiply by A to the one 00:31:36.380 --> 00:31:38.130 hundredth? 00:31:38.130 --> 00:31:41.620 It's got that factor lambda to the hundredth. 00:31:41.620 --> 00:31:42.890 That's the key. 00:31:42.890 --> 00:31:48.440 That -- that's what I mean by going its own merry way. 00:31:48.440 --> 00:31:52.320 It, it is pure eigenvector. 00:31:52.320 --> 00:31:55.850 It's exactly in a direction where multiplication by A 00:31:55.850 --> 00:31:59.200 just brings in a scalar factor, lambda one. 00:31:59.200 --> 00:32:02.240 So a hundred times brings in this a hundred times. 00:32:02.240 --> 00:32:06.080 Hundred times lambda two, hundred times lambda n. 00:32:06.080 --> 00:32:08.830 Actually, we're -- what are we seeing here? 00:32:08.830 --> 00:32:15.040 We're seeing, this same, lambda capital 00:32:15.040 --> 00:32:19.570 lambda to the hundredth as in the, as in the diagonalization. 00:32:19.570 --> 00:32:22.350 And we're seeing the S matrix, the, 00:32:22.350 --> 00:32:24.730 the matrix S of eigenvectors. 00:32:24.730 --> 00:32:29.440 That's what this has got to -- this has got to amount to. 00:32:29.440 --> 00:32:40.030 A lambda to the hundredth power times an S times this vector c 00:32:40.030 --> 00:32:43.490 that's telling us how much of each one 00:32:43.490 --> 00:32:45.010 is in the original thing. 00:32:45.010 --> 00:32:49.010 So if, if I had to really find the hundredth power, 00:32:49.010 --> 00:32:54.200 I would take u0, I would expand it as a combination 00:32:54.200 --> 00:32:57.210 of eigenvectors -- this is really S, 00:32:57.210 --> 00:33:01.680 the eigenvector matrix, times c, the, the coefficient vector. 00:33:04.240 --> 00:33:07.310 And then I would immediately then, 00:33:07.310 --> 00:33:10.950 by inserting these hundredth powers of eigenvalues, 00:33:10.950 --> 00:33:15.490 I'd have the answer. 00:33:15.490 --> 00:33:17.880 So -- huh, there must be -- 00:33:17.880 --> 00:33:20.570 oh, let's see, OK. 00:33:20.570 --> 00:33:22.970 It's -- so, yeah. 00:33:22.970 --> 00:33:30.790 So if u100 is A to the hundredth times u0, and u0 is S c -- 00:33:30.790 --> 00:33:36.160 then you see this formula is just this formula, 00:33:36.160 --> 00:33:40.840 which is the way I would actually get hold of this, 00:33:40.840 --> 00:33:44.690 of this u100, which is -- 00:33:44.690 --> 00:33:47.180 let me put it here. 00:33:47.180 --> 00:33:48.030 u100. 00:33:48.030 --> 00:33:51.070 The way I would actually get hold of that, see what, 00:33:51.070 --> 00:33:57.400 what the solution is after a hundred steps, would be -- 00:33:57.400 --> 00:34:05.960 expand the initial vector into eigenvectors and let each 00:34:05.960 --> 00:34:10.020 eigenvector go its own way, multiplying by a hundred at -- 00:34:10.020 --> 00:34:13.400 by lambda at every step, and therefore by lambda 00:34:13.400 --> 00:34:16.030 to the hundredth power after a hundred steps. 00:34:16.030 --> 00:34:18.050 Can I do an example? 00:34:18.050 --> 00:34:20.260 So that's the formulas. 00:34:20.260 --> 00:34:22.540 Now let me take an example. 00:34:22.540 --> 00:34:29.090 I'll use the Fibonacci sequence as an example. 00:34:29.090 --> 00:34:31.590 So, so Fibonacci example. 00:34:39.830 --> 00:34:43.050 You remember the Fibonacci numbers? 00:34:43.050 --> 00:34:48.150 If we start with one and one as F0 -- oh, 00:34:48.150 --> 00:34:50.280 I think I start with zero, maybe. 00:34:50.280 --> 00:34:54.550 Let zero and one be the first ones. 00:34:54.550 --> 00:34:58.550 So there's F0 and F1, the first two Fibonacci numbers. 00:34:58.550 --> 00:35:02.840 Then what's the rule for Fibonacci numbers? 00:35:02.840 --> 00:35:04.130 Ah, they're the sum. 00:35:04.130 --> 00:35:08.030 The next one is the sum of those, so it's one. 00:35:08.030 --> 00:35:11.110 The next one is the sum of those, so it's two. 00:35:11.110 --> 00:35:14.010 The next one is the sum of those, so it's three. 00:35:14.010 --> 00:35:16.190 Well, it looks like one two three four five, 00:35:16.190 --> 00:35:19.350 but somehow it's not going to do that way. 00:35:19.350 --> 00:35:21.380 The next one is five, right. 00:35:21.380 --> 00:35:22.640 Two and three makes five. 00:35:22.640 --> 00:35:26.090 The next one is eight. 00:35:26.090 --> 00:35:28.370 The next one is thirteen. 00:35:28.370 --> 00:35:33.245 And the one hundredth Fibonacci number is what? 00:35:35.920 --> 00:35:37.600 That's my question. 00:35:37.600 --> 00:35:40.680 How could I get a formula for the hundredth number? 00:35:40.680 --> 00:35:44.470 And, for example, how could I answer the question, 00:35:44.470 --> 00:35:47.740 how fast are they growing? 00:35:47.740 --> 00:35:52.650 How fast are those Fibonacci numbers growing? 00:35:52.650 --> 00:35:54.070 They're certainly growing. 00:35:54.070 --> 00:35:56.270 It's not a stable case. 00:35:56.270 --> 00:35:59.030 Whatever the eigenvalues of whatever matrix it is, 00:35:59.030 --> 00:36:00.720 they're not smaller than one. 00:36:00.720 --> 00:36:02.540 These numbers are growing. 00:36:02.540 --> 00:36:04.450 But how fast are they growing? 00:36:04.450 --> 00:36:10.070 The answer lies in the eigenvalue. 00:36:10.070 --> 00:36:12.450 So I've got to find the matrix, so let me write down 00:36:12.450 --> 00:36:14.495 the Fibonacci rule. 00:36:17.610 --> 00:36:22.245 F(k+2) = F(k+1)+F k, right? 00:36:25.210 --> 00:36:28.280 Now that's not in my -- 00:36:28.280 --> 00:36:32.420 I want to write that as uk plus one and Auk. 00:36:32.420 --> 00:36:38.920 But right now what I've got is a single equation, not a system, 00:36:38.920 --> 00:36:41.140 and it's second-order. 00:36:41.140 --> 00:36:44.290 It's like having a second-order differential equation 00:36:44.290 --> 00:36:45.810 with second derivatives. 00:36:45.810 --> 00:36:47.580 I want to get first derivatives. 00:36:47.580 --> 00:36:49.200 Here I want to get first differences. 00:36:49.200 --> 00:36:55.910 So the way, the way to do it is to introduce uk will be 00:36:55.910 --> 00:36:57.960 a vector -- 00:36:57.960 --> 00:36:59.125 see, a small trick. 00:37:01.920 --> 00:37:05.330 Let uk be a vector, F(k+1) and Fk. 00:37:08.230 --> 00:37:12.680 So I'm going to get a two by two system, first order, 00:37:12.680 --> 00:37:16.890 instead of a one -- instead of a scalar system, second order, 00:37:16.890 --> 00:37:18.300 by a simple trick. 00:37:18.300 --> 00:37:22.820 I'm just going to add in an equation F(k+1) equals F(k+1). 00:37:22.820 --> 00:37:28.980 That will be my second equation. 00:37:28.980 --> 00:37:33.940 Then this is my system, this is my unknown, 00:37:33.940 --> 00:37:38.690 and what's my one step equation? 00:37:38.690 --> 00:37:45.120 So, so now u(k+1), that's -- so u(k+1) is the left side, 00:37:45.120 --> 00:37:47.620 and what have I got here on the right side? 00:37:47.620 --> 00:37:52.530 I've got some matrix multiplying uk. 00:37:52.530 --> 00:37:56.510 Can you, do -- can you see that all right? 00:37:56.510 --> 00:37:59.450 if you can see it, then you can tell me what the matrix is. 00:37:59.450 --> 00:38:02.860 Do you see that I'm taking my system here. 00:38:02.860 --> 00:38:06.550 I artificially made it into a system. 00:38:06.550 --> 00:38:10.540 I artificially made the unknown into a vector. 00:38:10.540 --> 00:38:14.260 And now I'm ready to look at and see what the matrix 00:38:14.260 --> 00:38:15.020 is. 00:38:15.020 --> 00:38:20.240 So do you see the left side, u(k+1) is F(k+2) F(k+1), 00:38:20.240 --> 00:38:21.940 that's just what I want. 00:38:21.940 --> 00:38:25.590 On the right side, this remember, this uk here -- 00:38:25.590 --> 00:38:29.960 let me for the moment put it as F(k+1) Fk. 00:38:29.960 --> 00:38:33.080 So what's the matrix? 00:38:33.080 --> 00:38:41.380 Well, that has a one and a one, and that has a one and a zero. 00:38:41.380 --> 00:38:43.080 There's the matrix. 00:38:43.080 --> 00:38:47.880 Do you see that that gives me the right-hand side? 00:38:47.880 --> 00:38:52.360 So there's the matrix A. 00:38:52.360 --> 00:38:56.810 And this is our friend uk. 00:38:56.810 --> 00:39:00.650 So we've got -- so that simple trick -- 00:39:00.650 --> 00:39:03.900 changed the second-order scalar problem 00:39:03.900 --> 00:39:05.730 to a first-order system. 00:39:05.730 --> 00:39:08.750 Two b- u- with two unknowns. 00:39:08.750 --> 00:39:10.040 With a matrix. 00:39:10.040 --> 00:39:13.100 And now what do I do? 00:39:13.100 --> 00:39:16.240 Well, before I even think, I find its eigenvalues 00:39:16.240 --> 00:39:18.170 and eigenvectors. 00:39:18.170 --> 00:39:21.170 So what are the eigenvalues and eigenvectors of that matrix? 00:39:23.820 --> 00:39:24.320 Let's see. 00:39:24.320 --> 00:39:27.083 I always -- first let me just, like, think for a minute. 00:39:29.720 --> 00:39:35.440 It's two by two, so this shouldn't be impossible to do. 00:39:35.440 --> 00:39:37.020 Let's do it. 00:39:37.020 --> 00:39:37.670 OK. 00:39:37.670 --> 00:39:43.170 So my matrix, again, is one one one zero. 00:39:46.170 --> 00:39:49.070 It's symmetric, by the way. 00:39:49.070 --> 00:39:56.070 So what I will eventually know about symmetric matrices 00:39:56.070 --> 00:39:59.140 is that the eigenvalues will come out real. 00:39:59.140 --> 00:40:02.290 I won't get any complex numbers here. 00:40:02.290 --> 00:40:06.210 And the eigenvectors, once I get those, 00:40:06.210 --> 00:40:08.520 actually will be orthogonal. 00:40:08.520 --> 00:40:11.190 But two by two, I'm more interested in what 00:40:11.190 --> 00:40:13.740 the actual numbers are. 00:40:13.740 --> 00:40:16.230 What do I know about the two numbers? 00:40:16.230 --> 00:40:18.190 Well, should do you want me to find 00:40:18.190 --> 00:40:19.820 this determinant of A minus 00:40:19.820 --> 00:40:20.629 lambda I? 00:40:20.629 --> 00:40:21.128 Sure. 00:40:23.880 --> 00:40:27.910 So it's the determinant of one minus lambda one one zero, 00:40:27.910 --> 00:40:28.410 right? 00:40:31.900 --> 00:40:33.400 Minus lambda, yes. 00:40:33.400 --> 00:40:33.994 God. 00:40:33.994 --> 00:40:34.493 OK. 00:40:38.030 --> 00:40:40.110 OK. 00:40:40.110 --> 00:40:42.240 There'll be two eigenvalues. 00:40:42.240 --> 00:40:45.160 What will -- tell me again what I know about the two 00:40:45.160 --> 00:40:47.550 eigenvalues before I go any further. 00:40:47.550 --> 00:40:49.590 Tell me something about these two eigenvalues. 00:40:49.590 --> 00:40:51.770 What do they add up to? 00:40:51.770 --> 00:40:55.860 Lambda one plus lambda two is? 00:40:55.860 --> 00:41:02.390 Is the same as the trace down the diagonal of the matrix. 00:41:02.390 --> 00:41:04.660 One and zero is one. 00:41:04.660 --> 00:41:08.320 So lambda one plus lambda two should come out to be one. 00:41:08.320 --> 00:41:10.710 And lambda one times lambda one times lambda two 00:41:10.710 --> 00:41:13.300 should come out to be the determinant, which 00:41:13.300 --> 00:41:15.360 is minus one. 00:41:15.360 --> 00:41:18.440 So I'm expecting the eigenvalues to add to one 00:41:18.440 --> 00:41:20.570 and to multiply to minus one. 00:41:20.570 --> 00:41:22.720 But let's just see it happen here. 00:41:22.720 --> 00:41:26.680 If I multiply this out, I get -- that times that'll be a lambda 00:41:26.680 --> 00:41:30.290 squared minus lambda minus one. 00:41:30.290 --> 00:41:30.790 Good. 00:41:33.830 --> 00:41:36.570 Lambda squared minus lambda minus one. 00:41:36.570 --> 00:41:43.250 Actually, I -- you see the b- compare that with the original 00:41:43.250 --> 00:41:48.655 equation that I started with. 00:41:48.655 --> 00:41:49.780 F(k+2) - F(k+1)-Fk is zero. 00:41:49.780 --> 00:42:00.330 The recursion that -- that the Fibonacci numbers satisfy is 00:42:00.330 --> 00:42:05.140 somehow showing up directly here for the eigenvalues when we set 00:42:05.140 --> 00:42:06.230 that to zero. 00:42:06.230 --> 00:42:06.730 WK. 00:42:06.730 --> 00:42:09.530 Let's solve. 00:42:09.530 --> 00:42:14.200 Well, I would like to be able to factor that, that quadratic, 00:42:14.200 --> 00:42:17.450 but I'm better off to use the quadratic formula. 00:42:17.450 --> 00:42:19.880 Lambda is -- let's see. 00:42:19.880 --> 00:42:25.860 Minus b is one plus or minus the square root of b squared, 00:42:25.860 --> 00:42:30.650 which is one, minus four times that times that, 00:42:30.650 --> 00:42:33.740 which is plus four, over two. 00:42:37.614 --> 00:42:39.030 So that's the square root of five. 00:42:42.600 --> 00:42:50.230 So the eigenvalues are lambda one is one half of one 00:42:50.230 --> 00:42:57.000 plus square root of five, and lambda two is one half of one 00:42:57.000 --> 00:42:59.220 minus square root of five. 00:42:59.220 --> 00:43:04.630 And sure enough, they -- those add up to one and they multiply 00:43:04.630 --> 00:43:06.890 to give minus one. 00:43:06.890 --> 00:43:07.390 OK. 00:43:07.390 --> 00:43:09.400 Those are the two eigenvalues. 00:43:09.400 --> 00:43:12.250 How -- what are those numbers approximately? 00:43:12.250 --> 00:43:18.060 Square root of five, well, it's more than two 00:43:18.060 --> 00:43:19.030 but less than three. 00:43:19.030 --> 00:43:19.860 Hmm. 00:43:19.860 --> 00:43:25.330 It'd be nice to know these numbers. 00:43:25.330 --> 00:43:30.480 I think, I think that -- so that number comes out bigger than 00:43:30.480 --> 00:43:30.980 one, right? 00:43:30.980 --> 00:43:31.640 That's right. 00:43:31.640 --> 00:43:35.070 This number comes out bigger than one. 00:43:35.070 --> 00:43:38.210 It's about one point six one eight or something. 00:43:42.610 --> 00:43:44.700 Not exactly, but. 00:43:44.700 --> 00:43:48.300 And suppose it's one point six. 00:43:48.300 --> 00:43:52.390 Just, like, I think so. 00:43:52.390 --> 00:43:54.800 Then what's lambda two? 00:43:54.800 --> 00:43:57.870 Is, is lambda two positive or negative? 00:43:57.870 --> 00:44:01.430 Negative, right, because I'm -- it's, obviously negative, 00:44:01.430 --> 00:44:07.420 and I knew that the -- so it's minus -- 00:44:07.420 --> 00:44:16.720 and they add up to one, so minus point six one eight, I guess. 00:44:16.720 --> 00:44:17.220 OK. 00:44:17.220 --> 00:44:17.800 A- and some more. 00:44:17.800 --> 00:44:18.520 Those are the two eigenvalues. 00:44:18.520 --> 00:44:19.830 One eigenvalue bigger than one, one eigenvalue smaller than 00:44:19.830 --> 00:44:20.330 one. 00:44:20.330 --> 00:44:22.480 Actually, that's a great situation to be in. 00:44:22.480 --> 00:44:25.430 Of course, the eigenvalues are different, 00:44:25.430 --> 00:44:29.340 so there's no doubt whatever -- is this matrix diagonalizable? 00:44:32.270 --> 00:44:35.280 Is this matrix diagonalizable, that original matrix A? 00:44:35.280 --> 00:44:35.990 Sure. 00:44:35.990 --> 00:44:38.030 We've got two distinct eigenvalues 00:44:38.030 --> 00:44:44.294 and we can find the eigenvectors in a moment. 00:44:44.294 --> 00:44:46.460 But they'll be independent, we'll be diagonalizable. 00:44:46.460 --> 00:44:54.790 And now, you, you can already answer my very first question. 00:44:54.790 --> 00:44:59.530 How fast are those Fibonacci numbers increasing? 00:44:59.530 --> 00:45:01.080 How -- those -- they're increasing, 00:45:01.080 --> 00:45:01.810 right? 00:45:01.810 --> 00:45:03.970 They're not doubling at every step. 00:45:03.970 --> 00:45:07.330 Let me -- let's look again at these numbers. 00:45:07.330 --> 00:45:09.580 Five, eight, thirteen, it's not obvious. 00:45:09.580 --> 00:45:14.060 The next one would be twenty-one, thirty-four. 00:45:14.060 --> 00:45:20.924 So to get some idea of what F one hundred is, 00:45:20.924 --> 00:45:21.840 can you give me any -- 00:45:21.840 --> 00:45:24.820 I mean the crucial number -- 00:45:24.820 --> 00:45:32.280 so it -- these -- it's approximately -- 00:45:32.280 --> 00:45:37.970 what's controlling the growth of these Fibonacci numbers? 00:45:37.970 --> 00:45:39.630 It's the eigenvalues. 00:45:39.630 --> 00:45:43.031 And which eigenvalue is controlling that growth? 00:45:43.031 --> 00:45:43.530 The big one. 00:45:43.530 --> 00:45:50.380 So F100 will be approximately some constant, c1 I guess, 00:45:50.380 --> 00:45:56.110 times this lambda one, this one plus square root of five 00:45:56.110 --> 00:46:01.300 over two, to the hundredth power. 00:46:01.300 --> 00:46:04.560 And the two hundredth F -- in other words, the eigenvalue -- 00:46:04.560 --> 00:46:08.950 the Fibonacci numbers are growing by about that factor. 00:46:08.950 --> 00:46:13.780 Do you see that we, we've got precise information about the, 00:46:13.780 --> 00:46:18.230 about the Fibonacci numbers out of the eigenvalues? 00:46:18.230 --> 00:46:18.940 OK. 00:46:18.940 --> 00:46:21.880 And again, why is that true? 00:46:21.880 --> 00:46:26.750 Let me go over to this board and s- show what I'm doing here. 00:46:26.750 --> 00:46:30.720 The -- the original initial value is some combination 00:46:30.720 --> 00:46:31.730 of eigenvectors. 00:46:35.520 --> 00:46:39.470 And then when we start -- when we start going out the theories 00:46:39.470 --> 00:46:42.580 of Fibonacci numbers, when we start multiplying by A 00:46:42.580 --> 00:46:45.980 a hundred times, it's this lambda one to the hundredth. 00:46:45.980 --> 00:46:51.070 This term is, is the one that's taking over. 00:46:51.070 --> 00:46:54.920 It's -- I mean, that's big, like one point six to the hundredth 00:46:54.920 --> 00:46:55.880 power. 00:46:55.880 --> 00:47:00.610 The second term is practically nothing, right? 00:47:00.610 --> 00:47:04.300 The point six, or minus point six, to the hundredth power 00:47:04.300 --> 00:47:08.180 is an extremely small, extremely small number. 00:47:08.180 --> 00:47:11.410 So this is -- there're only two terms, 00:47:11.410 --> 00:47:13.020 because we're two by two. 00:47:13.020 --> 00:47:16.430 This number is -- this piece of it is there, 00:47:16.430 --> 00:47:21.270 but it's, it's disappearing, where this piece is there 00:47:21.270 --> 00:47:23.890 and it's growing and controlling everything. 00:47:23.890 --> 00:47:27.230 So, so really the -- we're doing, like, 00:47:27.230 --> 00:47:29.100 problems that are evolving. 00:47:29.100 --> 00:47:33.390 We're doing dynamic u- instead of Ax=b, 00:47:33.390 --> 00:47:35.440 that's a static problem. 00:47:35.440 --> 00:47:36.930 We're now we're doing dynamics. 00:47:36.930 --> 00:47:39.740 A, A squared, A cubed, things are evolving in 00:47:39.740 --> 00:47:40.440 time. 00:47:40.440 --> 00:47:44.660 And the eigenvalues are the crucial, numbers. 00:47:44.660 --> 00:47:45.640 OK. 00:47:45.640 --> 00:47:52.490 I guess to complete this, I better 00:47:52.490 --> 00:47:56.420 write down the eigenvectors. 00:47:56.420 --> 00:47:59.160 So we should complete the, the whole process 00:47:59.160 --> 00:48:01.200 by finding the eigenvectors. 00:48:01.200 --> 00:48:03.820 OK, well, I have to -- up in the corner, then, 00:48:03.820 --> 00:48:07.670 I have to look at A minus lambda I. 00:48:07.670 --> 00:48:15.800 So A minus lambda I is this one minus lambda one one and minus 00:48:15.800 --> 00:48:16.930 lambda. 00:48:16.930 --> 00:48:19.990 And now can we spot an eigenvector out of that? 00:48:19.990 --> 00:48:23.070 That's, that's, for these two lambdas, 00:48:23.070 --> 00:48:24.415 this matrix is singular. 00:48:27.380 --> 00:48:30.350 I guess the eigenvector -- two by two ought to be, I mean, 00:48:30.350 --> 00:48:31.260 easy. 00:48:31.260 --> 00:48:33.960 So if I know that this matrix is singular, 00:48:33.960 --> 00:48:37.100 then u- seems to me the eigenvector 00:48:37.100 --> 00:48:41.340 has to be lambda and one, because that multiplication 00:48:41.340 --> 00:48:43.500 will give me the zero. 00:48:43.500 --> 00:48:47.240 And this multiplication gives me -- better give me also zero. 00:48:47.240 --> 00:48:48.650 Do you see why it does? 00:48:48.650 --> 00:48:52.670 This is the minus lambda squared plus lambda plus one. 00:48:52.670 --> 00:48:56.360 It's the thing that's zero because these lambdas are 00:48:56.360 --> 00:48:56.930 special. 00:48:56.930 --> 00:48:58.490 There's the eigenvector. 00:48:58.490 --> 00:49:07.900 x1 is lambda one one, and x2 is lambda two one. 00:49:07.900 --> 00:49:12.520 I did that as a little trick that was available in the two 00:49:12.520 --> 00:49:14.130 by two case. 00:49:14.130 --> 00:49:17.680 So now I finally have to -- 00:49:17.680 --> 00:49:20.390 oh, I have to take the initial u0 now. 00:49:20.390 --> 00:49:22.710 So to complete this example entirely, 00:49:22.710 --> 00:49:26.680 I have to say, OK, what was u0? 00:49:26.680 --> 00:49:28.740 u0 was F1 F0. 00:49:28.740 --> 00:49:40.630 So u0, the starting vector is F1 F0, and those were one and 00:49:40.630 --> 00:49:41.130 zero. 00:49:43.910 --> 00:49:47.150 So I have to use that vector. 00:49:47.150 --> 00:49:50.390 So I have to look for, for a multiple 00:49:50.390 --> 00:49:56.100 of the first eigenvector and the second to produce u0, 00:49:56.100 --> 00:49:58.070 the one zero 00:49:58.070 --> 00:49:58.570 vector. 00:49:58.570 --> 00:50:05.430 This is what will find c1 and c2, and then I'm done. 00:50:05.430 --> 00:50:10.470 Do you -- so let me instead of, in the last five seconds, 00:50:10.470 --> 00:50:14.920 grinding out a formula, let me repeat the idea. 00:50:14.920 --> 00:50:19.100 Because I'd really -- it's the idea that's central. 00:50:19.100 --> 00:50:21.610 When things are evolving in time -- 00:50:21.610 --> 00:50:25.730 let me come back to this board, because the ideas are here. 00:50:25.730 --> 00:50:30.400 When things are evolving in time by a first-order system, 00:50:30.400 --> 00:50:34.480 starting from an original u0, the key 00:50:34.480 --> 00:50:39.956 is find the eigenvalues and eigenvectors of A. 00:50:39.956 --> 00:50:41.580 That will tell -- those eigenvectors -- 00:50:41.580 --> 00:50:46.710 the eigenvalues will already tell you what's happening. 00:50:46.710 --> 00:50:48.540 Is the solution blowing up, is it 00:50:48.540 --> 00:50:51.390 going to zero, what's it doing. 00:50:51.390 --> 00:50:56.240 And then to, to find out exactly a formula, 00:50:56.240 --> 00:50:59.290 you have to take your u0 and write it 00:50:59.290 --> 00:51:03.270 as a combination of eigenvectors and then 00:51:03.270 --> 00:51:05.820 follow each eigenvector separately. 00:51:05.820 --> 00:51:10.930 And that's really what this formula, the formula for, -- 00:51:10.930 --> 00:51:15.190 that's what the formula for A to the K is doing. 00:51:15.190 --> 00:51:17.180 So remember that formula for A to the K 00:51:17.180 --> 00:51:21.770 is S lambda to the K S inverse. 00:51:21.770 --> 00:51:22.280 OK. 00:51:22.280 --> 00:51:24.590 That's, that's difference equations. 00:51:24.590 --> 00:51:33.460 And you just have to -- so the, the homework will give some 00:51:33.460 --> 00:51:41.180 examples, different from Fibonacci, to follow through. 00:51:41.180 --> 00:51:48.900 And next time will be differential equations. 00:51:48.900 --> 00:51:50.450 Thanks.