1 00:00:02,940 --> 00:00:08,330 GILBERT STRANG: So today begins eigenvalues and eigenvectors. 2 00:00:08,330 --> 00:00:11,320 And the reason we want those, need 3 00:00:11,320 --> 00:00:17,540 those is to solve systems of linear equations. 4 00:00:17,540 --> 00:00:22,680 Systems meaning more than one equation, n equations. 5 00:00:22,680 --> 00:00:27,090 n equal 2 in the examples here. 6 00:00:27,090 --> 00:00:32,000 So eigenvalue is a number, eigenvector is a vector. 7 00:00:32,000 --> 00:00:35,600 They're both hiding in the matrix. 8 00:00:35,600 --> 00:00:38,640 Once we find them, we can use them. 9 00:00:38,640 --> 00:00:43,600 Let me show you the reason eigenvalues were created, 10 00:00:43,600 --> 00:00:50,150 invented, discovered was solving differential equations, which 11 00:00:50,150 --> 00:00:52,330 is our purpose. 12 00:00:52,330 --> 00:00:59,110 So why is now a vector-- so this is a system of equations. 13 00:00:59,110 --> 00:01:01,590 I'll do an example in a minute. 14 00:01:01,590 --> 00:01:04,760 A is a matrix. 15 00:01:04,760 --> 00:01:09,460 So we have n equations, n components of y. 16 00:01:09,460 --> 00:01:15,550 And A is an n by n matrix, n rows, n columns. 17 00:01:15,550 --> 00:01:16,660 Good. 18 00:01:16,660 --> 00:01:20,600 And now I can tell you right away where eigenvalues 19 00:01:20,600 --> 00:01:23,670 and eigenvectors pay off. 20 00:01:23,670 --> 00:01:26,160 They come into the solution. 21 00:01:26,160 --> 00:01:29,800 We look for solutions of that kind. 22 00:01:29,800 --> 00:01:32,790 When we had one equation, we looked for solutions 23 00:01:32,790 --> 00:01:37,890 just e to the st, and we found that number s. 24 00:01:37,890 --> 00:01:41,540 Now we have e to the lambda t-- we changed s 25 00:01:41,540 --> 00:01:46,120 to lambda, no problem-- but multiplied by a vector 26 00:01:46,120 --> 00:01:49,610 because our unknown is a vector. 27 00:01:49,610 --> 00:01:52,830 This is a vector, but that does not depend on time. 28 00:01:52,830 --> 00:01:54,510 That's the beauty of it. 29 00:01:54,510 --> 00:01:59,330 All the time dependence is in the exponential, as always. 30 00:01:59,330 --> 00:02:05,870 And x is just multiples of that exponential, as you'll see. 31 00:02:05,870 --> 00:02:08,550 So I look for solutions like that. 32 00:02:08,550 --> 00:02:12,970 I plug that into the differential equation 33 00:02:12,970 --> 00:02:14,510 and what happens? 34 00:02:14,510 --> 00:02:16,140 So here's my equation. 35 00:02:16,140 --> 00:02:21,110 I'm plugging in what e to the lambda tx, that's y. 36 00:02:21,110 --> 00:02:23,520 That's A times y there. 37 00:02:23,520 --> 00:02:27,480 Now, the derivative of y, the time derivative, 38 00:02:27,480 --> 00:02:30,750 brings down a lambda. 39 00:02:30,750 --> 00:02:33,520 To get the derivative I include the lambda. 40 00:02:33,520 --> 00:02:37,850 So do you see that substituting into the equation 41 00:02:37,850 --> 00:02:43,360 with this nice notation is just this has to be true. 42 00:02:43,360 --> 00:02:47,050 My equation changed to that form. 43 00:02:47,050 --> 00:02:51,430 OK Now I cancel either the lambda t, just the way 44 00:02:51,430 --> 00:02:54,980 I was always canceling e to the st. 45 00:02:54,980 --> 00:02:58,700 So I cancel e to the lambda t because it's never zero. 46 00:02:58,700 --> 00:03:04,080 And I have the big equation, Ax, the matrix 47 00:03:04,080 --> 00:03:07,920 times my eigenvector, is equal to lambda 48 00:03:07,920 --> 00:03:13,680 x-- the number, the eigenvalue, times the eigenvector. 49 00:03:13,680 --> 00:03:16,140 Not linear, notice. 50 00:03:16,140 --> 00:03:18,210 Two unknowns here that are multiplied. 51 00:03:18,210 --> 00:03:21,820 A number, lambda, times a vector, x. 52 00:03:21,820 --> 00:03:23,960 So what am I looking for? 53 00:03:23,960 --> 00:03:29,440 I'm looking for vectors x, the eigenvectors, so that 54 00:03:29,440 --> 00:03:36,700 multiplying by A-- multiplying A times x gives a number times x. 55 00:03:36,700 --> 00:03:41,870 It's in the same direction as x just the length is changed. 56 00:03:41,870 --> 00:03:46,590 Well, if lambda was 1, I would have Ax equal x. 57 00:03:46,590 --> 00:03:47,850 That's allowed. 58 00:03:47,850 --> 00:03:51,400 If lambda is 0, I would have Ax equals 0. 59 00:03:51,400 --> 00:03:52,810 That's all right. 60 00:03:52,810 --> 00:03:54,640 I don't want x to be 0. 61 00:03:54,640 --> 00:03:57,085 That's useless. 62 00:03:57,085 --> 00:04:01,780 That's no help to know that 0 is a solution. 63 00:04:01,780 --> 00:04:04,070 So x should be not 0. 64 00:04:04,070 --> 00:04:06,990 Lambda can be any number. 65 00:04:06,990 --> 00:04:09,880 It can be real, it could be complex number, 66 00:04:09,880 --> 00:04:11,120 as you will see. 67 00:04:11,120 --> 00:04:14,860 Even if the matrix is real, lambda could be complex. 68 00:04:14,860 --> 00:04:17,190 Anyway, Ax equal lambda x. 69 00:04:17,190 --> 00:04:19,029 That's the big equation. 70 00:04:19,029 --> 00:04:21,690 It got a box around it. 71 00:04:21,690 --> 00:04:26,820 So now I'm ready to do an example. 72 00:04:26,820 --> 00:04:29,460 And in this example, first of all, 73 00:04:29,460 --> 00:04:33,670 I'm going to spot the eigenvalues and eigenvectors 74 00:04:33,670 --> 00:04:39,090 without a system, just go for it in the 2 by 2 case. 75 00:04:39,090 --> 00:04:43,360 So I'll give a 2 by 2 matrix A. We'll find the lambdas 76 00:04:43,360 --> 00:04:48,690 and the x's, and then we'll have the solution to the system 77 00:04:48,690 --> 00:04:50,180 of differential equations. 78 00:04:50,180 --> 00:04:53,050 Good. 79 00:04:53,050 --> 00:04:55,420 There's the system. 80 00:04:55,420 --> 00:05:00,620 There's the first equation for y1-- prime meaning derivative, 81 00:05:00,620 --> 00:05:06,220 d by dt, time derivative-- is linear, a constant coefficient. 82 00:05:06,220 --> 00:05:10,280 Second one, linear, constant coefficient, 3 and 3. 83 00:05:10,280 --> 00:05:15,540 Those numbers, 5, 1, 3, 3, go into the matrix. 84 00:05:15,540 --> 00:05:21,970 Then that problem is exactly y prime, the vector, derivative 85 00:05:21,970 --> 00:05:25,890 of the vector, equal A times y. 86 00:05:25,890 --> 00:05:27,380 That's my problem. 87 00:05:27,380 --> 00:05:32,800 Now eigenvalues and eigenvectors will solve it. 88 00:05:32,800 --> 00:05:36,340 So I just look at that matrix. 89 00:05:36,340 --> 00:05:37,870 Matrix question. 90 00:05:37,870 --> 00:05:41,360 What are the eigenvalues, what are the eigenvectors 91 00:05:41,360 --> 00:05:43,270 of that matrix? 92 00:05:43,270 --> 00:05:52,182 And remember, I want Ax equals lambda x. 93 00:05:52,182 --> 00:05:55,680 I've spotted the first eigenvector. 94 00:05:55,680 --> 00:05:58,370 1, 1. 95 00:05:58,370 --> 00:06:00,750 We could just check does it work. 96 00:06:00,750 --> 00:06:04,340 If I multiply A by that eigenvector, 97 00:06:04,340 --> 00:06:08,350 1, 1, do you see what happens when I multiply by 1? 98 00:06:08,350 --> 00:06:10,380 That gives me a 6. 99 00:06:10,380 --> 00:06:12,700 That gives me a 6. 100 00:06:12,700 --> 00:06:17,210 So A times that vector is 6, 6. 101 00:06:17,210 --> 00:06:20,870 And that is 6 times 1, 1. 102 00:06:20,870 --> 00:06:22,310 So there you go. 103 00:06:22,310 --> 00:06:24,610 Found the first eigenvalue. 104 00:06:24,610 --> 00:06:29,050 If I multiply A by x, I get 6 by x. 105 00:06:29,050 --> 00:06:31,420 I get the vector 6, 6. 106 00:06:31,420 --> 00:06:33,440 Now, the second one. 107 00:06:33,440 --> 00:06:39,170 Again, I've worked in advance, produced this eigenvector, 108 00:06:39,170 --> 00:06:42,220 and I think it's 1 minus 3. 109 00:06:42,220 --> 00:06:46,430 So let's multiply by A. Try the second eigenvector. 110 00:06:46,430 --> 00:06:51,010 I should call this first one maybe x1 and lambda 1. 111 00:06:51,010 --> 00:06:54,070 And I should call this one x2 and lambda 2. 112 00:06:54,070 --> 00:06:58,400 And we can find out what lambda 2 is, once I find 113 00:06:58,400 --> 00:07:00,160 the eigenvectors of course. 114 00:07:00,160 --> 00:07:06,060 I just do A times x to recognize the lambda, the eigenvalue. 115 00:07:06,060 --> 00:07:11,580 So 5, 1 times this is 5 minus 3 is a 2. 116 00:07:11,580 --> 00:07:14,250 It's a 2. 117 00:07:14,250 --> 00:07:17,680 So here I got a 2. 118 00:07:17,680 --> 00:07:25,020 And from 3, 3 it's 3 minus 9 is minus 6. 119 00:07:25,020 --> 00:07:28,410 That's what I got for Ax. 120 00:07:28,410 --> 00:07:31,020 There was the x. 121 00:07:31,020 --> 00:07:34,920 When I did the multiplication, Ax came out to be 2 minus 6. 122 00:07:34,920 --> 00:07:36,360 Good. 123 00:07:36,360 --> 00:07:41,380 That output is two times the input. 124 00:07:41,380 --> 00:07:44,050 The eigenvalue is 2. 125 00:07:44,050 --> 00:07:45,370 Right? 126 00:07:45,370 --> 00:07:49,560 I'm looking for inputs, the eigenvector, 127 00:07:49,560 --> 00:07:54,420 so that the output is a number times that eigenvector, 128 00:07:54,420 --> 00:07:57,730 and that number is lambda, the eigenvalue. 129 00:07:57,730 --> 00:08:01,510 So I've now found the two. 130 00:08:01,510 --> 00:08:04,730 And I expect two for a 2 by 2 matrix. 131 00:08:04,730 --> 00:08:08,820 You will soon see why I expect two eigenvalues, 132 00:08:08,820 --> 00:08:12,380 and each eigenvalue should have an eigenvector. 133 00:08:12,380 --> 00:08:17,050 So here they are for this matrix. 134 00:08:17,050 --> 00:08:19,730 So I've got the answers now. 135 00:08:19,730 --> 00:08:28,400 y of t, which stands for y1 and y2 of t. 136 00:08:30,960 --> 00:08:36,280 Those are-- it's e to the lambda tx. 137 00:08:36,280 --> 00:08:39,470 Remember, that's the picture that we're looking for. 138 00:08:39,470 --> 00:08:47,770 So the first one is e to the 6t times x, which is 1, 1. 139 00:08:47,770 --> 00:08:52,490 If I put that into the equation, it will solve the equation. 140 00:08:52,490 --> 00:08:55,502 Also, I have another one. 141 00:08:55,502 --> 00:09:00,330 e to the lambda 2 was 2t. 142 00:09:00,330 --> 00:09:06,330 e to the lambda t times its eigenvector, 1 minus 3. 143 00:09:06,330 --> 00:09:08,520 That's a solution also. 144 00:09:08,520 --> 00:09:10,490 One solution, another solution. 145 00:09:10,490 --> 00:09:13,350 And what do I do with linear equations? 146 00:09:13,350 --> 00:09:16,040 I take combinations. 147 00:09:16,040 --> 00:09:21,090 Any number c1 of that, plus any number c2 of that 148 00:09:21,090 --> 00:09:23,370 is still a solution. 149 00:09:23,370 --> 00:09:28,890 That's superposition, adding solutions to linear equations. 150 00:09:28,890 --> 00:09:31,500 These are null equations. 151 00:09:31,500 --> 00:09:35,130 There's no force term in these equations. 152 00:09:35,130 --> 00:09:36,740 I'm not dealing with a force term. 153 00:09:36,740 --> 00:09:40,800 I'm looking for the null solutions, the solutions 154 00:09:40,800 --> 00:09:43,320 of the equations themselves. 155 00:09:43,320 --> 00:09:50,510 And there I have two solutions, two coefficients to choose. 156 00:09:50,510 --> 00:09:52,290 How do I choose them? 157 00:09:52,290 --> 00:09:57,130 Of course, I match the initial condition, so at t equals 0. 158 00:09:59,990 --> 00:10:01,700 At t equals 0. 159 00:10:01,700 --> 00:10:06,710 At t equals 0, I would have y of 0. 160 00:10:10,160 --> 00:10:16,350 That's my given initial condition, my y1 and y2. 161 00:10:16,350 --> 00:10:20,480 So I'm setting t equals 0, so that's one of course. 162 00:10:20,480 --> 00:10:22,530 When t is 0, that's one. 163 00:10:22,530 --> 00:10:27,240 So I just have c1 times 1, 1. 164 00:10:27,240 --> 00:10:38,980 And c2-- that's one again at t equals o-- times 1 minus 3. 165 00:10:38,980 --> 00:10:44,230 That's what determines c1 and c2. 166 00:10:44,230 --> 00:10:47,800 c1 and c2 come from the initial conditions 167 00:10:47,800 --> 00:10:51,280 just the way they always did. 168 00:10:51,280 --> 00:10:56,600 So I'm solving two first order linear constant coefficient 169 00:10:56,600 --> 00:11:01,760 equations, homogeneous, meaning no force term. 170 00:11:05,360 --> 00:11:08,850 So I get a null solution with constants to choose 171 00:11:08,850 --> 00:11:13,370 and, as always, those constants come from matching 172 00:11:13,370 --> 00:11:15,690 the initial conditions. 173 00:11:15,690 --> 00:11:19,150 So the initial condition here is a vector. 174 00:11:21,780 --> 00:11:27,580 So if, for example, y of 0 was 2 minus 2, then 175 00:11:27,580 --> 00:11:32,500 I would want one of those and one of those. 176 00:11:32,500 --> 00:11:34,740 OK. 177 00:11:34,740 --> 00:11:37,650 I've used eigenvalues and eigenvectors 178 00:11:37,650 --> 00:11:43,530 to solve a linear system, their first and primary purpose. 179 00:11:43,530 --> 00:11:44,610 OK. 180 00:11:44,610 --> 00:11:49,240 But how do I find those eigenvalues and eigenvectors? 181 00:11:49,240 --> 00:11:51,250 What about other properties? 182 00:11:51,250 --> 00:11:54,220 What's going on with eigenvalues and eigenvectors? 183 00:11:54,220 --> 00:11:58,340 May I begin on this just a couple more minutes 184 00:11:58,340 --> 00:12:00,970 about eigenvalues and eigenvectors? 185 00:12:00,970 --> 00:12:07,390 Basic facts and then I'll come next video of how to find them. 186 00:12:07,390 --> 00:12:10,000 OK, basic facts. 187 00:12:10,000 --> 00:12:10,910 Basic facts. 188 00:12:10,910 --> 00:12:21,570 So start from Ax equals lambda x. 189 00:12:21,570 --> 00:12:25,350 Let's suppose we found those. 190 00:12:25,350 --> 00:12:28,620 Could you tell me the eigenvalues and eigenvectors 191 00:12:28,620 --> 00:12:29,480 of A squared? 192 00:12:35,120 --> 00:12:38,690 I would like to know what the eigenvalues and eigenvectors 193 00:12:38,690 --> 00:12:39,860 of A squared are. 194 00:12:39,860 --> 00:12:42,190 Are they connected with these? 195 00:12:42,190 --> 00:12:48,240 So suppose I know the x and I know the lambda for A. What 196 00:12:48,240 --> 00:12:50,200 about for A squared? 197 00:12:50,200 --> 00:12:53,800 Well, the good thing is that the eigenvectors 198 00:12:53,800 --> 00:12:56,210 are the same for A squared. 199 00:12:56,210 --> 00:12:58,570 So let me show you. 200 00:12:58,570 --> 00:13:06,650 I say that same x, so this is the same x, same vector, 201 00:13:06,650 --> 00:13:07,980 same eigenvector. 202 00:13:07,980 --> 00:13:10,220 The eigenvalue would be different, of course, 203 00:13:10,220 --> 00:13:15,390 for A squared, but the eigenvector is the same. 204 00:13:15,390 --> 00:13:18,050 And let's see what happens for A squared. 205 00:13:18,050 --> 00:13:23,010 So that's A times Ax, right? 206 00:13:23,010 --> 00:13:26,090 One A, another Ax. 207 00:13:26,090 --> 00:13:30,132 But Ax is lambda x. 208 00:13:30,132 --> 00:13:31,090 Are you good with that? 209 00:13:31,090 --> 00:13:32,990 That's just A times Ax. 210 00:13:32,990 --> 00:13:34,970 So that's OK. 211 00:13:34,970 --> 00:13:37,250 Now lambda is a number. 212 00:13:37,250 --> 00:13:42,070 I like to bring it out front where I can see it. 213 00:13:42,070 --> 00:13:44,440 So I didn't do anything there. 214 00:13:44,440 --> 00:13:47,080 This number lambda was multiplying everything 215 00:13:47,080 --> 00:13:48,770 so I put it in front. 216 00:13:48,770 --> 00:13:49,740 Now Ax. 217 00:13:49,740 --> 00:13:52,020 I have, again, the Ax. 218 00:13:52,020 --> 00:13:54,830 That's, again, the lambda x because I'm 219 00:13:54,830 --> 00:13:56,480 looking at the same x. 220 00:13:56,480 --> 00:13:59,220 Same x, so I get the same lambda. 221 00:13:59,220 --> 00:14:02,120 So that's a lambda x, another lambda. 222 00:14:02,120 --> 00:14:05,150 I have lambda squared x. 223 00:14:05,150 --> 00:14:06,910 That's what I wanted. 224 00:14:06,910 --> 00:14:11,040 A squared x is lambda squared x. 225 00:14:11,040 --> 00:14:12,370 Conclusion. 226 00:14:12,370 --> 00:14:22,030 The eigenvectors stay the same, lambda goes to lambda squared. 227 00:14:22,030 --> 00:14:24,450 The eigenvalues are squared. 228 00:14:24,450 --> 00:14:31,175 So if I had my example again-- oh, let me find that matrix. 229 00:14:36,860 --> 00:14:40,590 Suppose I had that same matrix and I 230 00:14:40,590 --> 00:14:43,740 was interested in A squared, then 231 00:14:43,740 --> 00:14:50,480 the eigenvalues would be 36 and 4, the squares. 232 00:14:50,480 --> 00:14:53,300 I suppose I'm looking at the n-th power of a matrix. 233 00:14:53,300 --> 00:14:55,780 You may say why look at the n-th power? 234 00:14:55,780 --> 00:14:57,580 But there are many examples to look 235 00:14:57,580 --> 00:15:01,340 at the n-th power of a matrix, the thousandth power. 236 00:15:01,340 --> 00:15:04,670 So let's just write down the conclusion. 237 00:15:04,670 --> 00:15:13,710 Same reasoning, A to the n-th x is lambda. 238 00:15:13,710 --> 00:15:15,930 It's the same x. 239 00:15:15,930 --> 00:15:21,130 And every time I multiply by A, I multiply by a lambda. 240 00:15:21,130 --> 00:15:22,920 So I get lambda n times. 241 00:15:26,220 --> 00:15:30,940 So there is the handy rule. 242 00:15:30,940 --> 00:15:33,340 And that really tells us something about what 243 00:15:33,340 --> 00:15:35,070 eigenvalues are good for. 244 00:15:35,070 --> 00:15:39,880 Eigenvalues are good for things that move in time. 245 00:15:39,880 --> 00:15:45,450 Differential equations, that is really moving in time. 246 00:15:45,450 --> 00:15:51,100 n equal 1 is this first time, or n equals 0 is the start. 247 00:15:51,100 --> 00:15:55,470 Take one step to n equal 1, take another step to n equal 2. 248 00:15:55,470 --> 00:15:56,480 Keep going. 249 00:15:56,480 --> 00:16:01,780 Every time step brings a multiplication by lambda. 250 00:16:01,780 --> 00:16:06,840 So that is a very useful rule. 251 00:16:06,840 --> 00:16:12,215 Another handy rule is what about A plus the identity? 252 00:16:15,290 --> 00:16:23,870 Suppose I add the identity matrix to my original matrix. 253 00:16:23,870 --> 00:16:25,620 What happens to the eigenvalues? 254 00:16:25,620 --> 00:16:27,290 What happens to the eigenvectors? 255 00:16:27,290 --> 00:16:28,740 Basic question. 256 00:16:28,740 --> 00:16:33,030 Or I could multiply a constant times the identity, 2 times 257 00:16:33,030 --> 00:16:35,820 the identity, 7 times the identity. 258 00:16:35,820 --> 00:16:39,890 And I want to know what about its eigenvectors. 259 00:16:39,890 --> 00:16:45,260 And the answer is same, same x's. 260 00:16:45,260 --> 00:16:47,950 Same x. 261 00:16:47,950 --> 00:16:53,010 I show that by figuring out what I have here. 262 00:16:53,010 --> 00:16:57,630 This is Ax, which is lambda x. 263 00:16:57,630 --> 00:17:00,850 And this is c times the identity times x. 264 00:17:00,850 --> 00:17:05,760 The identity doesn't do anything so that's just cx. 265 00:17:05,760 --> 00:17:08,430 So what do I have now? 266 00:17:08,430 --> 00:17:16,319 I've seen that the eigenvalue is lambda plus c. 267 00:17:16,319 --> 00:17:19,491 So there is the eigenvalues. 268 00:17:24,685 --> 00:17:30,100 I think about this as shifting A by a multiple of the identity. 269 00:17:30,100 --> 00:17:34,830 Shifting A, adding 5 times the identity to it. 270 00:17:34,830 --> 00:17:38,780 If I add 5 times the identity to any matrix, 271 00:17:38,780 --> 00:17:43,710 the eigenvalues of that matrix go up by 5. 272 00:17:43,710 --> 00:17:46,800 And the eigenvectors stay the same. 273 00:17:46,800 --> 00:17:51,650 So as long as I keep working with that one matrix A. 274 00:17:51,650 --> 00:17:57,650 Taking powers, adding multiples of the identity, 275 00:17:57,650 --> 00:18:02,050 later taking exponentials, whatever I do I keep 276 00:18:02,050 --> 00:18:05,730 the same eigenvectors and everything is easy. 277 00:18:10,910 --> 00:18:17,390 If I had two matrices, A and B, with different eigenvectors, 278 00:18:17,390 --> 00:18:21,280 then I don't know what the eigenvectors of A plus B 279 00:18:21,280 --> 00:18:22,330 would be. 280 00:18:22,330 --> 00:18:24,430 I don't know those. 281 00:18:24,430 --> 00:18:28,040 I can't tell the eigenvectors of A times B 282 00:18:28,040 --> 00:18:30,610 because A has its own little eigenvectors 283 00:18:30,610 --> 00:18:32,420 and B has its eigenvectors. 284 00:18:32,420 --> 00:18:38,310 Unless they're the same, I can't easily combine A and B. 285 00:18:38,310 --> 00:18:47,770 But as always I'm staying with one A and its powers and steps 286 00:18:47,770 --> 00:18:50,090 like that, no problem. 287 00:18:50,090 --> 00:18:50,720 OK. 288 00:18:50,720 --> 00:18:54,240 I'll stop there for a first look at eigenvalues 289 00:18:54,240 --> 00:18:55,990 and eigenvectors.