WEBVTT 00:00:05.560 --> 00:00:14.460 OK, this is the lecture on positive definite matrices. 00:00:14.460 --> 00:00:20.770 I made a start on those briefly in a previous lecture. 00:00:20.770 --> 00:00:26.180 One point I wanted to make was the way that this topic brings 00:00:26.180 --> 00:00:30.300 the whole course together, pivots, determinants, 00:00:30.300 --> 00:00:36.190 eigenvalues, and something new- four plot instability 00:00:36.190 --> 00:00:42.050 and then something new in this expression, x transpose Ax, 00:00:42.050 --> 00:00:46.040 actually that's the guy to watch in this lecture. 00:00:46.040 --> 00:00:50.410 So, so the topic is positive definite matrix, 00:00:50.410 --> 00:00:53.580 and what's my goal? 00:00:53.580 --> 00:00:58.440 First, first goal is, how can I tell if a matrix is 00:00:58.440 --> 00:00:59.970 positive definite? 00:00:59.970 --> 00:01:02.490 So I would like to have tests to see 00:01:02.490 --> 00:01:06.100 if you give me a, a five by five matrix, 00:01:06.100 --> 00:01:09.000 how do I tell if it's positive definite? 00:01:09.000 --> 00:01:12.020 More important is, what does it mean? 00:01:12.020 --> 00:01:14.700 Why are we so interested in this property 00:01:14.700 --> 00:01:16.440 of positive definiteness? 00:01:16.440 --> 00:01:21.410 And then, at the end comes some geometry. 00:01:21.410 --> 00:01:25.080 Ellipses are connected with positive definite things. 00:01:25.080 --> 00:01:28.520 Hyperbolas are not connected with positive definite things, 00:01:28.520 --> 00:01:33.350 so we- it's this, we, there's a geometry too, 00:01:33.350 --> 00:01:38.410 but mostly it's linear algebra and -- 00:01:38.410 --> 00:01:43.020 this application of how do you recognize 'em when you have 00:01:43.020 --> 00:01:45.181 a minim is pretty neat. 00:01:45.181 --> 00:01:45.680 OK. 00:01:45.680 --> 00:01:50.090 I'm gonna begin with two by two. 00:01:50.090 --> 00:01:52.280 All matrices are symmetric, right? 00:01:52.280 --> 00:01:55.180 That's understood; the matrix is symmetric, 00:01:55.180 --> 00:02:00.650 now my question is, is it positive definite? 00:02:00.650 --> 00:02:04.030 Now, here are some -- 00:02:04.030 --> 00:02:09.570 each one of these is a complete test for positive definiteness. 00:02:09.570 --> 00:02:15.010 If I know the eigenvalues, my test is are they positive? 00:02:15.010 --> 00:02:18.140 Are they all positive? 00:02:18.140 --> 00:02:21.650 If I know these -- 00:02:21.650 --> 00:02:24.170 so, A is really -- 00:02:24.170 --> 00:02:27.570 I look at that number A, here, as the, as the one 00:02:27.570 --> 00:02:33.420 by one determinant, and here's the two by two determinant. 00:02:33.420 --> 00:02:35.510 So this is the determinant test. 00:02:35.510 --> 00:02:39.280 This is the eigenvalue test, this is the determinant test. 00:02:39.280 --> 00:02:46.040 Are the determinants growing in s- of all, of all end, sort of, 00:02:46.040 --> 00:02:48.640 can I call them leading submatrices, 00:02:48.640 --> 00:02:52.140 they're the first ones the northwest, 00:02:52.140 --> 00:02:56.510 Seattle submatrices coming down from from there, 00:02:56.510 --> 00:02:59.570 they all, all those determinants have to be positive, 00:02:59.570 --> 00:03:03.020 and then another test is the pivots. 00:03:03.020 --> 00:03:06.370 The pivots of a two by two matrix 00:03:06.370 --> 00:03:13.070 are the number A for sure, and, since the product 00:03:13.070 --> 00:03:15.410 is the determinant, the second pivot 00:03:15.410 --> 00:03:18.980 must be the determinant divided by A. 00:03:18.980 --> 00:03:24.970 And then in here is gonna come my favorite and my new idea, 00:03:24.970 --> 00:03:30.450 the, the, the the one to catch, about x transpose Ax 00:03:30.450 --> 00:03:32.640 being positive. 00:03:32.640 --> 00:03:35.280 But we'll have to look at this guy. 00:03:35.280 --> 00:03:42.780 This gets, like a star, because for most, presentations, 00:03:42.780 --> 00:03:45.610 the definition of positive definiteness 00:03:45.610 --> 00:03:49.370 would be this number four and these numbers one two three 00:03:49.370 --> 00:03:51.260 would be test four. 00:03:51.260 --> 00:03:51.760 OK. 00:03:51.760 --> 00:03:56.160 Maybe I'll tuck this, where, you know, 00:03:56.160 --> 00:03:56.660 OK. 00:03:56.660 --> 00:04:00.000 So I'll have to look at this x transpose Ax. 00:04:00.000 --> 00:04:07.760 Can you, can we just be sure, how 00:04:07.760 --> 00:04:13.130 do we know that the eigenvalue test and the determinant test, 00:04:13.130 --> 00:04:17.170 pick out the same matrices, and let me, 00:04:17.170 --> 00:04:19.797 let's just do a few examples. 00:04:19.797 --> 00:04:20.380 Some examples. 00:04:24.610 --> 00:04:31.220 Let me pick the matrix two, six, six, tell me, 00:04:31.220 --> 00:04:36.360 what number do I have to put there for the matrix 00:04:36.360 --> 00:04:37.440 to be positive definite? 00:04:40.150 --> 00:04:42.090 Tell me a sufficiently large number 00:04:42.090 --> 00:04:45.020 that would make it positive definite? 00:04:45.020 --> 00:04:47.830 Let's just practice with these conditions in the two 00:04:47.830 --> 00:04:49.370 by two case. 00:04:49.370 --> 00:04:51.490 Now, when I ask you that, you don't 00:04:51.490 --> 00:04:56.270 wanna find the eigenvalues, you would use the determinant test 00:04:56.270 --> 00:05:00.340 for that, so, the first or the pivot test, 00:05:00.340 --> 00:05:03.750 that, that guy is certainly positive, that had to happen, 00:05:03.750 --> 00:05:05.070 and it's OK. 00:05:05.070 --> 00:05:09.030 How large a number here -- the number had better be more than 00:05:09.030 --> 00:05:10.200 what? 00:05:10.200 --> 00:05:14.350 More than eighteen, right, because if it's eight -- 00:05:14.350 --> 00:05:15.400 no. 00:05:15.400 --> 00:05:18.550 More than what? 00:05:18.550 --> 00:05:21.390 Nineteen, is it? 00:05:21.390 --> 00:05:28.965 If I have a nineteen here, is that positive definite? 00:05:32.590 --> 00:05:36.160 I get thirty eight minus thirty six, that's OK. 00:05:36.160 --> 00:05:39.900 If I had an eighteen, let me play it really close. 00:05:39.900 --> 00:05:44.850 If I have an eighteen there, then I positive definite? 00:05:44.850 --> 00:05:46.640 Not quite. 00:05:46.640 --> 00:05:49.830 I would call this guy positive, so it's 00:05:49.830 --> 00:05:53.550 useful just to see that this the borderline. 00:05:53.550 --> 00:05:55.640 That matrix is on the borderline, 00:05:55.640 --> 00:05:58.880 I would call that matrix positive semi-definite. 00:06:04.670 --> 00:06:07.900 And what are the eigenvalues of that matrix, 00:06:07.900 --> 00:06:11.690 just since we're given eigenvalues of two by twos, 00:06:11.690 --> 00:06:19.460 when it's semi-definite, but not definite, then the -- 00:06:19.460 --> 00:06:22.540 I'm squeezing this eigenvalue test down, -- 00:06:22.540 --> 00:06:28.370 what's the eigenvalue that I know this matrix has? 00:06:28.370 --> 00:06:31.320 What kind of a matrix have I got here? 00:06:31.320 --> 00:06:37.680 It's a singular matrix, one of its eigenvalues is zero. 00:06:37.680 --> 00:06:43.570 That has an eigenvalue zero, and the other eigenvalue is -- 00:06:43.570 --> 00:06:47.020 from the trace, twenty. 00:06:47.020 --> 00:06:47.620 OK. 00:06:47.620 --> 00:06:51.560 So that, that matrix has eigenvalues greater than 00:06:51.560 --> 00:06:55.710 or equal to zero, and it's that "equal to" that brought this 00:06:55.710 --> 00:06:58.000 word "semi-definite" in. 00:06:58.000 --> 00:07:01.240 And, the what are the pivots of that matrix? 00:07:01.240 --> 00:07:03.810 So the pivots, so the eigenvalues 00:07:03.810 --> 00:07:10.580 are zero and twenty, the pivots are, well, the pivot is two, 00:07:10.580 --> 00:07:12.090 and what's the next pivot? 00:07:15.840 --> 00:07:17.430 There isn't one. 00:07:17.430 --> 00:07:20.900 We got a singular matrix here, it'll only have one pivot. 00:07:20.900 --> 00:07:24.920 You see that that's a rank one matrix, two six is a -- 00:07:24.920 --> 00:07:27.710 six eighteen is a multiple of two six, 00:07:27.710 --> 00:07:32.380 the matrix is singular it only has one pivot, 00:07:32.380 --> 00:07:38.050 so the pivot test doesn't quite 00:07:38.050 --> 00:07:40.710 The -- let me do the x transpose Ax. 00:07:40.710 --> 00:07:41.210 pass. 00:07:41.210 --> 00:07:45.280 Now this is -- 00:07:45.280 --> 00:07:49.130 the novelty now. 00:07:49.130 --> 00:07:49.960 OK. 00:07:49.960 --> 00:07:54.470 What I looking at when I look at this new combination, 00:07:54.470 --> 00:07:56.360 x transpose Ax. 00:07:56.360 --> 00:08:02.140 x is any vector now, so lemme compute, so any vector, 00:08:02.140 --> 00:08:08.960 lemme call its components x1 and x2, so that's x. 00:08:08.960 --> 00:08:11.300 And I put in here A. 00:08:11.300 --> 00:08:15.500 Let's, let's use this example two six, six eighteen, 00:08:15.500 --> 00:08:20.500 and here's x transposed, so x1 x2. 00:08:20.500 --> 00:08:25.580 We're back to real matrices, after that last lecture 00:08:25.580 --> 00:08:29.480 that- that said what to do in the complex case, let's 00:08:29.480 --> 00:08:32.010 come back to real matrices. 00:08:32.010 --> 00:08:36.780 So here's x transpose Ax, and what I'm interested 00:08:36.780 --> 00:08:42.720 in is, what do I get if I multiply those together? 00:08:42.720 --> 00:08:48.410 I get some function of x1 and x2, and what is it? 00:08:48.410 --> 00:08:51.700 Let's see, if I do this multiplication, so I do it, 00:08:51.700 --> 00:08:55.220 lemme, just, I'll just do it slowly, x1, x2, 00:08:55.220 --> 00:09:01.980 if I multiply that matrix, this is 2x1, and 6x2s, 00:09:01.980 --> 00:09:07.400 and the next row is 6x1s and 18x2s, 00:09:07.400 --> 00:09:11.530 and now I do this final step and what do I have? 00:09:11.530 --> 00:09:16.810 I've got 2x1 squareds, got 2X1 squareds 00:09:16.810 --> 00:09:23.080 is coming from this two, I've got this one gives me eighteen, 00:09:23.080 --> 00:09:25.630 well, shall I do the ones in the middle? 00:09:25.630 --> 00:09:29.530 How many x1 x2s do I have? 00:09:29.530 --> 00:09:33.330 Let's see, x1 times that 6x2 would be six of 'em, 00:09:33.330 --> 00:09:38.300 and then x2 times this one will be six more, I get twelve. 00:09:38.300 --> 00:09:44.290 So, in here is going, this is the number a, this is 00:09:44.290 --> 00:09:49.190 the number 2b, and in here is -- 00:09:49.190 --> 00:09:54.960 x2 times eighteen x2 will be eighteen x2 squareds and this 00:09:54.960 --> 00:09:57.630 is the number c. 00:09:57.630 --> 00:10:02.010 So it's ax1 -- it's like ax squared. 00:10:02.010 --> 00:10:05.140 2bxy and cy squared. 00:10:05.140 --> 00:10:11.280 For my first point that I wanted to make by just doing out 00:10:11.280 --> 00:10:15.720 a multiplication is, that is as soon as you give me the matrix, 00:10:15.720 --> 00:10:19.240 as soon as you give me the matrix, I can -- 00:10:19.240 --> 00:10:22.860 those are the numbers that appear in -- 00:10:22.860 --> 00:10:25.580 I'll call this thing a quadratic, 00:10:25.580 --> 00:10:29.140 you see, it's not linear anymore. 00:10:29.140 --> 00:10:32.360 Ax is linear, but now I've got an x transpose coming 00:10:32.360 --> 00:10:35.680 in, I'm up to degree two, up to degree two, 00:10:35.680 --> 00:10:38.880 maybe quadratic is the -- 00:10:38.880 --> 00:10:39.480 use the word. 00:10:39.480 --> 00:10:41.980 A quadratic form. 00:10:41.980 --> 00:10:46.610 It's purely degree two, there's no linear part, 00:10:46.610 --> 00:10:48.440 there's no constant part, there certainly 00:10:48.440 --> 00:10:53.540 no cubes or fourth powers, it's all second degree. 00:10:53.540 --> 00:10:55.780 And my question is -- 00:10:55.780 --> 00:11:00.120 is that quantity positive or not? 00:11:00.120 --> 00:11:08.250 That's -- for every x1 and x2, that is my new definition -- 00:11:08.250 --> 00:11:11.920 that's my definition of a positive definite matrix. 00:11:11.920 --> 00:11:17.970 If this quantity is positive, if, if, if, it's positive 00:11:17.970 --> 00:11:25.140 for all x's and y's, all x1 x2s, then I call them -- 00:11:25.140 --> 00:11:29.170 then that's the matrix is positive definite. 00:11:29.170 --> 00:11:34.340 Now, is this guy passing our test? 00:11:34.340 --> 00:11:38.120 Well we have, we anticipated the answer here by, 00:11:38.120 --> 00:11:42.420 by asking first about eigenvalues and pivots, 00:11:42.420 --> 00:11:44.820 and what happened? 00:11:44.820 --> 00:11:46.350 It failed. 00:11:46.350 --> 00:11:49.490 It barely failed. 00:11:49.490 --> 00:11:53.670 If I had made this eighteen down to a seven, 00:11:53.670 --> 00:11:58.120 it would've totally failed. 00:11:58.120 --> 00:12:01.540 I do that with the eraser, and then I'll put back eighteen, 00:12:01.540 --> 00:12:07.170 because, seven is such a total disaster, but if -- 00:12:07.170 --> 00:12:10.240 I'll keep seven for a second. 00:12:10.240 --> 00:12:15.170 Is that thing in any way positive definite? 00:12:15.170 --> 00:12:18.480 No, absolutely not. 00:12:18.480 --> 00:12:21.990 I don't know its eigenvalues, but I know for sure one of them 00:12:21.990 --> 00:12:24.230 is negative. 00:12:24.230 --> 00:12:28.940 Its pivots are two and then the next pivot would be 00:12:28.940 --> 00:12:32.250 the determinant over two, and the determinant is -- what, 00:12:32.250 --> 00:12:35.090 what's the determinant of this thing? 00:12:35.090 --> 00:12:37.620 Fourteen minus thirty six, I've got 00:12:37.620 --> 00:12:39.820 a determinant minus twenty two. 00:12:39.820 --> 00:12:43.330 The next pivot will be -- the pivots now, 00:12:43.330 --> 00:12:48.480 of this thing are two and minus eleven or something. 00:12:48.480 --> 00:12:51.320 Their product being minus twenty two the determinant. 00:12:51.320 --> 00:12:53.620 This thing is not positive definite. 00:12:53.620 --> 00:12:57.200 What would be -- let me look at the x transpose Ax for this 00:12:57.200 --> 00:12:57.700 guy. 00:12:57.700 --> 00:13:00.460 What's -- if I do out this multiplication, 00:13:00.460 --> 00:13:04.560 this eighteen is temporarily changing to a seven. 00:13:04.560 --> 00:13:08.130 This eighteen is temporarily changing to a seven, 00:13:08.130 --> 00:13:16.610 and I know that there's some numbers x1 and x2 00:13:16.610 --> 00:13:24.880 for which that thing, that function, is negative. 00:13:24.880 --> 00:13:28.640 And I'm desperately trying to think what they are. 00:13:28.640 --> 00:13:30.040 Maybe you can see. 00:13:30.040 --> 00:13:33.940 Can you tell me a value of x1 and x2 00:13:33.940 --> 00:13:36.675 that makes this quantity negative? 00:13:40.400 --> 00:13:43.180 Oh, maybe one and minus one? 00:13:43.180 --> 00:13:49.270 Yes, that's -- in this case, that, will work, right, 00:13:49.270 --> 00:13:54.150 if I took x1 to be one, and x2 to be minus one, 00:13:54.150 --> 00:13:57.170 then I always get something positive, the two, 00:13:57.170 --> 00:14:01.367 and the seven minus one squared, but this would be minus twelve 00:14:01.367 --> 00:14:02.950 and the whole thing would be negative; 00:14:02.950 --> 00:14:07.620 I would have two minus twelve plus seven, a negative. 00:14:07.620 --> 00:14:11.550 If I drew the graph, can I get the little picture in 00:14:11.550 --> 00:14:12.050 here? 00:14:12.050 --> 00:14:16.430 If I draw the graph of this thing? 00:14:16.430 --> 00:14:22.730 So, graphs, of the function f(x,y), or f(x), 00:14:22.730 --> 00:14:29.620 so I say here f(x,y) equal this -- x transpose Ax, this, 00:14:29.620 --> 00:14:32.040 this this ax squared, 2bxy, and cy squared. 00:14:32.040 --> 00:14:46.600 And, let's take the example, with these numbers. 00:14:49.210 --> 00:14:54.650 OK, so here's the x axis, here's the y axis, and here's -- 00:14:54.650 --> 00:14:56.380 up is the function. 00:14:56.380 --> 00:14:59.390 z, if you like, or f. 00:14:59.390 --> 00:15:03.940 I apologize, and let me, just once in my life here, 00:15:03.940 --> 00:15:08.780 put an arrow over these, these, axes so you see them. 00:15:08.780 --> 00:15:13.300 That's the vector and I just, see, instead of x1 and x2, 00:15:13.300 --> 00:15:16.220 I made them x- the components x and y. 00:15:16.220 --> 00:15:16.930 OK. 00:15:16.930 --> 00:15:23.660 So, so, what's a graph of 2x squared, twelve xy, and seven y 00:15:23.660 --> 00:15:25.710 squared? 00:15:25.710 --> 00:15:27.660 I'd like to see -- 00:15:27.660 --> 00:15:31.260 I not the greatest artist, but let's -- 00:15:31.260 --> 00:15:38.400 tell me something about this graph of this function. 00:15:42.560 --> 00:15:44.740 Whoa, tell me one point that it goes through. 00:15:47.970 --> 00:15:49.470 The origin. 00:15:49.470 --> 00:15:50.990 Right? 00:15:50.990 --> 00:15:56.450 Even this artist can get this thing to go through the origin, 00:15:56.450 --> 00:16:00.421 when these are zero, I, I certainly get zero. 00:16:00.421 --> 00:16:00.920 OK. 00:16:00.920 --> 00:16:02.540 Some more points. 00:16:02.540 --> 00:16:06.570 If x is one and y is zero, then I'm going upwards, 00:16:06.570 --> 00:16:09.060 so I'm going up this way, and I'm, I'm 00:16:09.060 --> 00:16:12.180 going up, like, two x squared in that direction. 00:16:12.180 --> 00:16:14.766 So -- that's meant to be a parabola. 00:16:18.200 --> 00:16:23.150 And, suppose x stays zero and y increases. 00:16:23.150 --> 00:16:26.760 Well, y could be positive or negative; it's seven y squared. 00:16:26.760 --> 00:16:30.530 Is this function going upward? 00:16:30.530 --> 00:16:35.640 In the x direction it's going upward, and in the y direction 00:16:35.640 --> 00:16:39.900 it's going upwards, and if x equals y 00:16:39.900 --> 00:16:42.140 then the forty-five degree direction is certainly 00:16:42.140 --> 00:16:44.810 going upwards; because then we'd have what, 00:16:44.810 --> 00:16:49.600 about, everything would be positive, but what? 00:16:49.600 --> 00:16:54.810 This function -- what's the graph of this function? 00:16:54.810 --> 00:16:55.310 Look like? 00:16:55.310 --> 00:17:00.450 Tell me the word that describes the graph of this function. 00:17:00.450 --> 00:17:02.980 This is the non-positive definite here, 00:17:02.980 --> 00:17:07.660 everybody's with me here, for some reason 00:17:07.660 --> 00:17:10.740 got started in a negative direction, your case that 00:17:10.740 --> 00:17:12.750 isn't positive definite. 00:17:12.750 --> 00:17:17.190 And what's the graph look like that goes up, but does it -- 00:17:17.190 --> 00:17:22.710 do we have a minimum here, does it go from, from the origin? 00:17:22.710 --> 00:17:24.500 Completely? 00:17:24.500 --> 00:17:28.710 No, because we just checked that this thing failed. 00:17:28.710 --> 00:17:33.490 It failed along the direction when x was minus y -- 00:17:33.490 --> 00:17:37.080 we have a saddle point, let me put myself, let me, 00:17:37.080 --> 00:17:39.350 to the least, tell you the word. 00:17:39.350 --> 00:17:46.470 This thing, goes up in some directions, 00:17:46.470 --> 00:17:53.330 but down in other directions, and if we actually knew what 00:17:53.330 --> 00:17:59.300 a saddle looked like or thinks saddles do that -- 00:17:59.300 --> 00:18:06.560 the way your legs go is, like, down, up, the way, you, 00:18:06.560 --> 00:18:16.200 looking like, forward, and, the, and drawing the thing is even 00:18:16.200 --> 00:18:17.330 worse than describing -- 00:18:17.330 --> 00:18:20.760 I'm just going to say in some directions we go up 00:18:20.760 --> 00:18:28.290 and in other directions, there is, a saddle -- 00:18:28.290 --> 00:18:30.330 Now I'm sorry I put that on the front board, 00:18:30.330 --> 00:18:34.450 you have no way to cover it, but it's a saddle. 00:18:34.450 --> 00:18:35.360 OK. 00:18:35.360 --> 00:18:38.950 And, and this is a saddle point, it's 00:18:38.950 --> 00:18:43.900 the, it's the point that's at the maximum in some directions 00:18:43.900 --> 00:18:46.770 and at the minimum in other directions. 00:18:46.770 --> 00:18:50.840 And actually, the perfect directions to look 00:18:50.840 --> 00:18:53.670 are the eigenvector directions. 00:18:53.670 --> 00:18:55.260 We'll see that. 00:18:55.260 --> 00:19:03.730 So this is, not a positive definite matrix. 00:19:03.730 --> 00:19:04.280 OK. 00:19:04.280 --> 00:19:08.190 Now I'm coming back to this example, 00:19:08.190 --> 00:19:14.400 getting rid of this seven, let's move it up to twenty. 00:19:14.400 --> 00:19:18.420 Let's, let's let's make the thing really positive definite. 00:19:18.420 --> 00:19:19.470 OK. 00:19:19.470 --> 00:19:22.990 So this is, this number's now twenty. 00:19:22.990 --> 00:19:24.360 c is now twenty. 00:19:24.360 --> 00:19:24.860 OK. 00:19:24.860 --> 00:19:30.800 Now that passes the test, which I haven't proved, of course, 00:19:30.800 --> 00:19:34.960 it passes the test for positive pivots. 00:19:34.960 --> 00:19:40.270 It passes the test for positive eigenvalues. 00:19:40.270 --> 00:19:43.160 How can you tell that the eigenvalues of that matrix 00:19:43.160 --> 00:19:45.850 are positive without actually finding them? 00:19:45.850 --> 00:19:49.400 Of course, two by two I could find them, but can you see -- 00:19:49.400 --> 00:19:51.530 how do I know they're positive? 00:19:51.530 --> 00:19:53.270 I know that their product is -- 00:19:53.270 --> 00:19:58.020 I know that lambda one times lambda two is positive, why? 00:19:58.020 --> 00:20:01.750 Because that's the determinant, right, 00:20:01.750 --> 00:20:06.000 lambda one times lambda two is the determinant, which is forty 00:20:06.000 --> 00:20:07.730 minus thirty-six is four. 00:20:07.730 --> 00:20:11.120 So the determinant is four. 00:20:11.120 --> 00:20:16.160 And the trace, the sum down the diagonal, is twenty-two. 00:20:16.160 --> 00:20:20.630 So, they multiply to give four. 00:20:20.630 --> 00:20:22.850 So that leaves the possibility they're either 00:20:22.850 --> 00:20:25.730 both positive or both negative. 00:20:25.730 --> 00:20:29.979 But if they're both negative, the trace couldn't be 00:20:29.979 --> 00:20:31.520 So they're both positive. twenty-two. 00:20:31.520 --> 00:20:34.170 So both of the eigenvalues that are positive, 00:20:34.170 --> 00:20:36.220 both of the pivots are positive -- 00:20:36.220 --> 00:20:39.820 the determinants are positive, and I believe that this 00:20:39.820 --> 00:20:47.260 function is positive everywhere except at zero, zero, 00:20:47.260 --> 00:20:48.240 of course. 00:20:48.240 --> 00:20:52.330 When I write down this condition, 00:20:52.330 --> 00:20:54.670 So I believe that x transposed, let 00:20:54.670 --> 00:21:00.280 me copy, x transpose Ax is positive, except, of course, 00:21:00.280 --> 00:21:07.450 at the minimum point, at zero, of course, 00:21:07.450 --> 00:21:08.830 I don't expect miracles. 00:21:11.760 --> 00:21:15.720 So what does its graph look like, and how do I check, 00:21:15.720 --> 00:21:18.730 and how do I check that this really is positive? 00:21:22.300 --> 00:21:24.960 So we take it's graph for a minute. 00:21:24.960 --> 00:21:27.200 What would be the graph of that function -- 00:21:27.200 --> 00:21:29.400 it does not have a saddle point. 00:21:29.400 --> 00:21:31.280 Let me -- I'll raise the board, here, 00:21:31.280 --> 00:21:33.320 and stay with this example for a while. 00:21:36.280 --> 00:21:40.660 So I want to do the graph of -- here's my function, 00:21:40.660 --> 00:21:46.940 two x squared, twelve xy-s, that could be positive or negative, 00:21:46.940 --> 00:21:48.510 and twenty y squared. 00:21:48.510 --> 00:21:56.290 But my point is, so you're seeing the underlying point is, 00:21:56.290 --> 00:21:59.910 that, the things are positive definite 00:21:59.910 --> 00:22:05.730 when in some way, these, these pure squares, squares 00:22:05.730 --> 00:22:10.180 we know to be positive, and when those kind of overwhelm 00:22:10.180 --> 00:22:13.527 this guy, who could be m- positive or negative, 00:22:13.527 --> 00:22:15.110 because some like or have same or have 00:22:15.110 --> 00:22:19.760 same or different signs, when these are big enough 00:22:19.760 --> 00:22:22.850 they overwhelm this guy and make the total thing positive, 00:22:22.850 --> 00:22:25.750 and what would the graph now look like? 00:22:25.750 --> 00:22:32.940 Let me draw the x - well, let me draw the x direction, the y 00:22:32.940 --> 00:22:40.970 direction, and the origin, at zero, zero, I'm there, 00:22:40.970 --> 00:22:45.650 where do I go as I move away from the origin? 00:22:45.650 --> 00:22:50.780 Where do I go as I move away from the origin? 00:22:50.780 --> 00:22:53.240 I'm sure that I go up. 00:22:53.240 --> 00:22:56.660 The origin, the center point here, 00:22:56.660 --> 00:23:01.730 is a minim because this thing I believe, and we better see why, 00:23:01.730 --> 00:23:07.620 it's, the graph is like a bowl, the graph is like a bowl shape, 00:23:07.620 --> 00:23:09.805 it's -- here's the minimum. 00:23:15.660 --> 00:23:19.030 And because we've got a pure quadratic, 00:23:19.030 --> 00:23:23.410 we know it sits at the origin, we know it's tangent plane, 00:23:23.410 --> 00:23:30.970 the first derivatives are zero, so, we know, first derivatives, 00:23:30.970 --> 00:23:37.410 First derivatives are all zero, but that's 00:23:37.410 --> 00:23:38.560 not enough for a minimum. 00:23:38.560 --> 00:23:43.020 It's first derivatives were zero here. 00:23:45.780 --> 00:23:51.200 So, the partial derivatives, the first derivatives, are zero. 00:23:51.200 --> 00:23:56.720 Again, because first derivatives are gonna have an x or an a y, 00:23:56.720 --> 00:23:59.730 or a y in them, they'll be zero at the origin. 00:23:59.730 --> 00:24:03.340 It's the second derivatives that control everything. 00:24:03.340 --> 00:24:07.810 It's the second derivatives that this matrix is telling us, 00:24:07.810 --> 00:24:10.630 and somehow -- 00:24:10.630 --> 00:24:11.780 here's my point. 00:24:11.780 --> 00:24:16.710 You remember in Calculus, how did you decide on a minimum? 00:24:16.710 --> 00:24:20.010 First requirement was, that the derivative had to be 00:24:20.010 --> 00:24:20.940 zero. 00:24:20.940 --> 00:24:25.890 But then you didn't know if you had a minimum or a maximum. 00:24:25.890 --> 00:24:27.500 To know that you had a minimum, you 00:24:27.500 --> 00:24:30.300 had to look at the second derivative. 00:24:30.300 --> 00:24:33.400 The second derivative had to be positive, 00:24:33.400 --> 00:24:36.900 the slope had to be increasing as you 00:24:36.900 --> 00:24:40.010 went through the minimum point. 00:24:40.010 --> 00:24:43.470 The curvature had to go upwards, and that's 00:24:43.470 --> 00:24:46.910 what we're doing now in two dimensions, 00:24:46.910 --> 00:24:49.280 and in n dimensions. 00:24:49.280 --> 00:24:52.030 So we're doing what we did in Calculus. 00:24:52.030 --> 00:24:55.150 Second derivative positive, m- will now 00:24:55.150 --> 00:24:58.590 become that the matrix of second derivatives 00:24:58.590 --> 00:25:00.810 is positive definite. 00:25:00.810 --> 00:25:02.650 Can I just -- 00:25:02.650 --> 00:25:05.670 like a translation of -- 00:25:05.670 --> 00:25:11.890 this is how minimum are coming in, ithe beginning of Calculus 00:25:11.890 --> 00:25:14.640 -- 00:25:14.640 --> 00:25:22.730 we had a minimum was associated with second derivative, 00:25:22.730 --> 00:25:24.170 being positive. 00:25:24.170 --> 00:25:27.340 And first derivative zero, of course. 00:25:27.340 --> 00:25:36.120 Derivative, first derivative, but it 00:25:36.120 --> 00:25:39.870 was the second derivative that told us we had a minimum. 00:25:39.870 --> 00:25:43.510 And now, in 18.06, in linear algebra, 00:25:43.510 --> 00:25:47.260 we'll have a minim for our function now, 00:25:47.260 --> 00:25:53.040 our function will have, for your function be a function not 00:25:53.040 --> 00:26:00.290 of just x but several variables, the way functions really 00:26:00.290 --> 00:26:03.370 are in real life, the minimum will 00:26:03.370 --> 00:26:15.140 be when the matrix of second derivatives, the matrix 00:26:15.140 --> 00:26:17.590 here was one by one, there was just one second derivative, 00:26:17.590 --> 00:26:20.560 now we've got lots. 00:26:20.560 --> 00:26:25.335 Is positive definite. 00:26:29.070 --> 00:26:31.450 So positive for a number translates 00:26:31.450 --> 00:26:34.450 into positive definite for a matrix. 00:26:34.450 --> 00:26:38.860 And it this brings everything you check pivots, 00:26:38.860 --> 00:26:41.680 you check determinants, you check all your values, 00:26:41.680 --> 00:26:44.910 or you check this minimum stuff. 00:26:44.910 --> 00:26:45.410 OK. 00:26:45.410 --> 00:26:49.190 Let me come back to this graph. 00:26:49.190 --> 00:26:50.595 That graph goes upwards. 00:26:53.220 --> 00:26:54.740 And I'll have to see why. 00:26:54.740 --> 00:26:58.360 How do I know that this, that this function is always 00:26:58.360 --> 00:26:59.980 positive? 00:26:59.980 --> 00:27:04.770 Can you look at that and tell that it's always positive? 00:27:04.770 --> 00:27:08.710 Maybe two by two, you could feel pretty sure, 00:27:08.710 --> 00:27:14.530 but what's the good way to show that this thing is always 00:27:14.530 --> 00:27:20.630 If we can express it, as, in terms of squares, positive? 00:27:20.630 --> 00:27:24.010 because that's what we know for any x and y, 00:27:24.010 --> 00:27:26.760 whatever, if we're squaring something 00:27:26.760 --> 00:27:29.470 we certainly are not negative. 00:27:29.470 --> 00:27:32.690 So I believe that this expression, this function, 00:27:32.690 --> 00:27:37.190 could be written as a sum of squares. 00:27:37.190 --> 00:27:41.170 Can you tell me -- 00:27:41.170 --> 00:27:43.460 see, because all the problems, the headaches 00:27:43.460 --> 00:27:47.230 are coming from this xy term. 00:27:47.230 --> 00:27:50.500 If we can get expressions -- if we can get that inside 00:27:50.500 --> 00:27:53.660 a square, so actually, what we're doing is something 00:27:53.660 --> 00:27:58.160 called, that you've seen called completing the square. 00:27:58.160 --> 00:28:01.210 Let me start the square and you complete it. 00:28:01.210 --> 00:28:05.820 OK, I think we have two of x plus, 00:28:05.820 --> 00:28:09.400 now I don't remember how many y-s we need, but you'll 00:28:09.400 --> 00:28:10.725 figure it out, squared. 00:28:13.750 --> 00:28:20.470 How many y-s should I put in here, to make -- 00:28:20.470 --> 00:28:23.950 what do I want to do, the two x squared-s will be correct, 00:28:23.950 --> 00:28:25.770 right? 00:28:25.770 --> 00:28:28.990 What I want to do is put in the right number of y-s 00:28:28.990 --> 00:28:33.270 to get the twelve xy correct. 00:28:33.270 --> 00:28:34.885 And what is that number of y-s? 00:28:37.430 --> 00:28:39.170 Let's see, I've got two times, and so 00:28:39.170 --> 00:28:41.770 I really want six xy-s to come out of here, 00:28:41.770 --> 00:28:44.650 I think maybe if I put three there, 00:28:44.650 --> 00:28:48.590 does that look right to you? 00:28:48.590 --> 00:28:53.940 I have two- this is, we can mentally, multiply out, 00:28:53.940 --> 00:28:56.420 that's X squared, that's right, that's 00:28:56.420 --> 00:28:59.580 six X Y, times the two gives from, right, 00:28:59.580 --> 00:29:02.660 and how many Y squareds have I now got? 00:29:02.660 --> 00:29:05.950 How many Y squareds have I now got from this term? 00:29:05.950 --> 00:29:07.750 Eighteen. 00:29:07.750 --> 00:29:12.000 Eighteen was the key number, remember? 00:29:12.000 --> 00:29:16.630 Now if I want to make it twenty, then I've got two left. 00:29:16.630 --> 00:29:18.100 Two y squared-s. 00:29:18.100 --> 00:29:25.660 That's completing the square, and it's, now 00:29:25.660 --> 00:29:28.120 I can see that that function is positive, 00:29:28.120 --> 00:29:30.090 because it's all squares. 00:29:33.340 --> 00:29:35.840 I've got two squares, added together, 00:29:35.840 --> 00:29:38.110 I couldn't go negative. 00:29:38.110 --> 00:29:42.070 What if I went back to that seven? 00:29:42.070 --> 00:29:45.080 If instead of twenty that number was a seven, then 00:29:45.080 --> 00:29:47.610 what would happen? 00:29:47.610 --> 00:29:51.010 This would still be correct, I'd still have this square, 00:29:51.010 --> 00:29:53.670 to get the two x squared and the twelve xy, 00:29:53.670 --> 00:29:58.800 and I'd have eighteen y squared and then what would I do here? 00:29:58.800 --> 00:30:03.760 I'd have to remove eleven y squared-s, right, 00:30:03.760 --> 00:30:08.200 if I only had a seven here, then instead of -- 00:30:08.200 --> 00:30:13.040 when I had a twenty I had two more to put in, when I had 00:30:13.040 --> 00:30:16.740 an eighteen, which was the marginal case, 00:30:16.740 --> 00:30:19.780 I had no more to put in. 00:30:19.780 --> 00:30:23.370 When I had a seven, which was the case below zero, 00:30:23.370 --> 00:30:29.870 the indefinite case, I had minus eleven. 00:30:29.870 --> 00:30:36.360 Now, so, you can see now, that this thing is a bowl. 00:30:36.360 --> 00:30:36.860 OK. 00:30:36.860 --> 00:30:42.200 It's going upwards, if I cut it at a plane, z equal to one, 00:30:42.200 --> 00:30:47.830 say, I would get, I would get a curve, what would 00:30:47.830 --> 00:30:50.280 be the equation for that curve? 00:30:50.280 --> 00:30:53.190 If I cut it at height one, the equation 00:30:53.190 --> 00:30:56.980 would be this thing equal to one. 00:30:56.980 --> 00:30:59.910 And that curve would be an ellipse. 00:30:59.910 --> 00:31:02.560 So actually, already, I've blocked 00:31:02.560 --> 00:31:09.380 into the lecture, the different pieces that we're aiming for. 00:31:09.380 --> 00:31:12.620 We're aiming for the tests, which this passed; 00:31:12.620 --> 00:31:17.360 we're aiming for the connection to a minimum, which this -- 00:31:17.360 --> 00:31:22.420 which we see in the graph, and if we chop it up, 00:31:22.420 --> 00:31:25.180 if we set this thing equal to one, 00:31:25.180 --> 00:31:27.560 if I set that thing equal to one, that -- 00:31:27.560 --> 00:31:30.490 what that gives me is, the cross-section. 00:31:30.490 --> 00:31:37.750 It gives me this, this curve, and its equation 00:31:37.750 --> 00:31:41.200 is this thing equals one, and that's an ellipse. 00:31:41.200 --> 00:31:44.660 Whereas if I cut through a saddle point, 00:31:44.660 --> 00:31:47.900 I get a hyperbola. 00:31:47.900 --> 00:31:53.820 But this minimum stuff is really what I'm most interested 00:31:53.820 --> 00:31:54.610 OK. 00:31:54.610 --> 00:31:55.110 in. 00:31:55.110 --> 00:31:55.610 OK. 00:31:55.610 --> 00:32:00.300 By -- I just have to ask, do you recognize, I mean, 00:32:00.300 --> 00:32:04.460 these numbers here, the two that appeared outside, 00:32:04.460 --> 00:32:08.090 the three that appeared inside, the two that appeared there -- 00:32:08.090 --> 00:32:13.690 actually, those numbers come from elimination. 00:32:13.690 --> 00:32:18.890 Completing the square is our good old method 00:32:18.890 --> 00:32:24.910 of Gaussian elimination, in expressed 00:32:24.910 --> 00:32:27.280 in terms of these squares. 00:32:27.280 --> 00:32:30.600 The -- let me show you what I mean. 00:32:30.600 --> 00:32:34.050 I just think those numbers are no accident, 00:32:34.050 --> 00:32:40.050 If I take my matrix two, six, six, and twenty, 00:32:40.050 --> 00:32:45.050 and I do elimination, then the pivot is two 00:32:45.050 --> 00:32:50.150 and I take three, what's the multiplier? 00:32:50.150 --> 00:32:54.180 How much of row one do I take away from row two? 00:32:54.180 --> 00:32:55.360 Three. 00:32:55.360 --> 00:32:58.700 So what you're seeing in this, completing the square, 00:32:58.700 --> 00:33:05.180 is the pivots outside and the multiplier inside. 00:33:05.180 --> 00:33:06.680 Just do that again? 00:33:06.680 --> 00:33:12.840 The pivot is two, three -- three of those away from that gives 00:33:12.840 --> 00:33:18.100 me two, six, zero, and what's the second pivot? 00:33:18.100 --> 00:33:21.550 Three of this away from this, three sixes'll be eighteen, 00:33:21.550 --> 00:33:23.380 and the second pivot will also be a 00:33:23.380 --> 00:33:24.040 two. 00:33:24.040 --> 00:33:32.490 So that's the U, this is the A, and of course the L 00:33:32.490 --> 00:33:37.900 was one, zero, one, and the multiplier was three. 00:33:40.540 --> 00:33:48.210 So, completing the square is elimination. 00:33:48.210 --> 00:33:54.230 Why I happy to see, happy to see that coming together? 00:33:54.230 --> 00:34:01.190 Because I know about elimination for m by m matrices. 00:34:01.190 --> 00:34:07.740 I just started talking about completing the square, here, 00:34:07.740 --> 00:34:10.320 for two by twos. 00:34:10.320 --> 00:34:12.989 But now I see what's going on. 00:34:12.989 --> 00:34:17.909 Completing the square really amounts to splitting this thing 00:34:17.909 --> 00:34:21.610 into a sum of squares, so what's the critical thing -- 00:34:21.610 --> 00:34:25.380 I have a lot of squares, and inside those squares 00:34:25.380 --> 00:34:28.020 are multipliers but they're squares, 00:34:28.020 --> 00:34:31.790 and the question is, what's outside these squares? 00:34:31.790 --> 00:34:35.210 When I complete the square, what are the numbers that go 00:34:35.210 --> 00:34:36.179 outside? 00:34:36.179 --> 00:34:37.342 They're the pivots. 00:34:39.889 --> 00:34:45.510 They're the pivots, and that's why positive pivots give me 00:34:45.510 --> 00:34:47.409 sum of squares. 00:34:47.409 --> 00:34:50.270 Positive pivots, those pivots are the numbers 00:34:50.270 --> 00:34:53.600 that go outside the squares, so positive pivots, 00:34:53.600 --> 00:34:57.850 sum of squares, everything positive, graph goes up, 00:34:57.850 --> 00:35:03.050 a minimum at the origin, it's all connected together; 00:35:03.050 --> 00:35:04.710 all connected together. 00:35:04.710 --> 00:35:08.110 And in the two by two case, you can see those connections, 00:35:08.110 --> 00:35:16.100 but linear algebra now can go up to three by three, m by m. 00:35:16.100 --> 00:35:17.950 Let's do that next. 00:35:17.950 --> 00:35:20.680 Can I just, before I leave two by two, 00:35:20.680 --> 00:35:24.940 I've written this expression "matrix of second derivatives." 00:35:24.940 --> 00:35:27.165 What's the matrix of second derivatives? 00:35:29.690 --> 00:35:31.970 That's one second derivative now, 00:35:31.970 --> 00:35:40.020 but if I'm in two dimensions, I have a two by two matrix, 00:35:40.020 --> 00:35:46.480 it's the second x derivative, the second x derivative goes 00:35:46.480 --> 00:35:49.070 there -- 00:35:49.070 --> 00:35:56.360 shall I write it -- fxx, if you like, fxx, 00:35:56.360 --> 00:36:00.170 that means the second derivative of f in the x direction. 00:36:00.170 --> 00:36:04.500 fyy, second derivative in the y direction. 00:36:04.500 --> 00:36:07.970 Those are the pure derivatives, second derivatives. 00:36:07.970 --> 00:36:10.950 They have to be positive. 00:36:10.950 --> 00:36:13.430 For a minimum. 00:36:13.430 --> 00:36:15.590 This number has to be positive for a minimum. 00:36:15.590 --> 00:36:17.730 That number has to be positive for a minimum. 00:36:17.730 --> 00:36:20.410 But, that's not enough. 00:36:20.410 --> 00:36:23.680 Those numbers have to somehow be big enough 00:36:23.680 --> 00:36:30.090 to overcome this cross-derivative, 00:36:30.090 --> 00:36:31.780 Why is the matrix symmetric? 00:36:31.780 --> 00:36:35.730 Because the second derivative f with respect to x and y is 00:36:35.730 --> 00:36:37.950 equal to -- 00:36:37.950 --> 00:36:42.170 I can, that's the beautiful fact about second derivatives, is 00:36:42.170 --> 00:36:46.480 I can do those in either order and I get the same thing. 00:36:46.480 --> 00:36:54.410 So this is the same as that, and so, that's 00:36:54.410 --> 00:36:57.300 the matrix of second derivatives. 00:36:57.300 --> 00:37:01.280 And the test is, it has to be positive definite. 00:37:01.280 --> 00:37:05.960 You might remember, from, tucked in somewhere 00:37:05.960 --> 00:37:08.990 near the end of eighteen o' two or at least in the book, 00:37:08.990 --> 00:37:13.590 was the condition for a minimum, For a function 00:37:13.590 --> 00:37:15.470 of two variables. 00:37:15.470 --> 00:37:17.180 Let's -- when do you have a minimum? 00:37:17.180 --> 00:37:19.740 For a function of two variables, believe me, 00:37:19.740 --> 00:37:24.200 that's what Calculus is for. 00:37:24.200 --> 00:37:29.270 The condition is first derivatives have to be zero. 00:37:29.270 --> 00:37:32.630 And the matrix of second derivatives 00:37:32.630 --> 00:37:35.280 has to be positive definite. 00:37:35.280 --> 00:37:39.650 So you maybe remember there was an fxx times an fyy that 00:37:39.650 --> 00:37:42.610 had to be bigger than an an fxy squared, 00:37:42.610 --> 00:37:46.630 that's just our determinant, two by two. 00:37:46.630 --> 00:37:51.480 But now, we now know the answer for three by three, 00:37:51.480 --> 00:37:57.770 m by m, because we can do elimination by m by m matrices, 00:37:57.770 --> 00:38:01.660 we can connect eigenvalues of m by m matrices, 00:38:01.660 --> 00:38:06.070 we can do sum of squares, sum of m squares instead of only two 00:38:06.070 --> 00:38:11.470 squares; and so let's take a, let 00:38:11.470 --> 00:38:14.000 me go over here to do a three by three example. 00:38:17.110 --> 00:38:18.510 So, three by three example. 00:38:23.445 --> 00:38:23.945 OK. 00:38:27.060 --> 00:38:29.113 Oh, let me -- 00:38:32.980 --> 00:38:34.670 shall I use my favorite matrix? 00:38:37.210 --> 00:38:39.080 You've seen this matrix before. 00:38:45.340 --> 00:38:49.680 Yes, let's use the good matrix, four by one, oops, open. 00:38:56.130 --> 00:38:57.950 Is that matrix positive definite? 00:39:01.300 --> 00:39:04.440 What's -- so I'm going to ask questions about this matrix, 00:39:04.440 --> 00:39:08.260 is it positive definite, first of all? 00:39:08.260 --> 00:39:10.970 What's the function associated with that matrix, 00:39:10.970 --> 00:39:12.780 what's the x transpose Ax? 00:39:16.260 --> 00:39:18.860 Is -- do we have a minimum for that function, 00:39:18.860 --> 00:39:19.865 at zero? 00:39:22.460 --> 00:39:25.380 And then even what's the geometry? 00:39:25.380 --> 00:39:26.060 OK. 00:39:26.060 --> 00:39:29.200 First of all, is the matrix positive definite, 00:39:29.200 --> 00:39:33.460 now I've given you the numbers there so you can take 00:39:33.460 --> 00:39:35.370 the determinants, maybe that's the quickest, 00:39:35.370 --> 00:39:36.910 is that what you would do mentally, 00:39:36.910 --> 00:39:39.660 if I give you all a matrix on a quiz and say 00:39:39.660 --> 00:39:43.010 is it positive definite or not? 00:39:43.010 --> 00:39:47.240 I would take that determinant and I'd give the answer two. 00:39:47.240 --> 00:39:49.130 I would take that determinant and I 00:39:49.130 --> 00:39:55.380 would give the answer for that two by two determinant, 00:39:55.380 --> 00:39:57.400 I'd give the answer three, and anybody 00:39:57.400 --> 00:40:04.220 remember the answer for the three by three determinant? 00:40:04.220 --> 00:40:08.220 It was four, you remember for these special matrices, 00:40:08.220 --> 00:40:13.190 when we do determinants, they went up two, three, four, five, 00:40:13.190 --> 00:40:15.950 six, they just went up linearly. 00:40:15.950 --> 00:40:22.940 So that matrix has -- the determinants are two, three, 00:40:22.940 --> 00:40:24.615 and four. 00:40:24.615 --> 00:40:25.115 Pivots. 00:40:27.630 --> 00:40:31.160 What are the pivots for that matrix? 00:40:31.160 --> 00:40:34.470 I'll tell you, they're -- the first pivot is two, 00:40:34.470 --> 00:40:37.520 the next pivot is three over two, 00:40:37.520 --> 00:40:39.730 the next pivot is four over three. 00:40:42.590 --> 00:40:45.170 Because, the product of the pivots 00:40:45.170 --> 00:40:47.550 has to give me those determinants. 00:40:47.550 --> 00:40:50.370 The product of these two pivots gives me that determinant; 00:40:50.370 --> 00:40:53.574 the product of all the pivots gives me that determinant. 00:40:53.574 --> 00:40:54.615 What are the eigenvalues? 00:41:03.020 --> 00:41:05.717 Oh, I don't know. 00:41:08.600 --> 00:41:12.860 The eigenvalues I've got, what do I have a cubic equation -- 00:41:12.860 --> 00:41:14.320 a degree three equation? 00:41:17.230 --> 00:41:19.350 There are three eigenvalues to find. 00:41:23.350 --> 00:41:25.200 If I believe what I've said today, 00:41:25.200 --> 00:41:27.800 what do I know about these eigenvalues, 00:41:27.800 --> 00:41:30.710 even though I don't know the exact numbers. 00:41:30.710 --> 00:41:34.535 I -- I remember the numbers. 00:41:37.100 --> 00:41:39.740 Because these matrices are so important 00:41:39.740 --> 00:41:43.750 that people figure them. 00:41:43.750 --> 00:41:47.440 But -- what do you believe to be true about these three 00:41:47.440 --> 00:41:52.950 eigenvalues -- you believe that they are all positive. 00:41:52.950 --> 00:41:54.270 They're all positive. 00:41:54.270 --> 00:42:00.910 I think that they are two minus square root of two, two, 00:42:00.910 --> 00:42:02.630 and two plus the square root of two. 00:42:02.630 --> 00:42:03.720 I think. 00:42:03.720 --> 00:42:05.490 Let me just -- 00:42:05.490 --> 00:42:08.280 I can't write those numbers down without checking 00:42:08.280 --> 00:42:10.870 the simple checks, what the first simple check is 00:42:10.870 --> 00:42:15.110 the trace, so if I add those numbers I get six 00:42:15.110 --> 00:42:18.970 and if I add those numbers I get six. 00:42:18.970 --> 00:42:24.150 The other simple test is the determinant, if I -- 00:42:24.150 --> 00:42:26.930 can you do this, can you multiply those numbers 00:42:26.930 --> 00:42:29.240 together? 00:42:29.240 --> 00:42:32.610 I guess we can multiply by two out there. 00:42:32.610 --> 00:42:35.010 What's two minus square root of two times two 00:42:35.010 --> 00:42:39.530 plus square root of two, that'll be four minus two, 00:42:39.530 --> 00:42:41.610 that'll be two, yeah, two times two, 00:42:41.610 --> 00:42:46.280 that's got the determinant, right, so it's got, 00:42:46.280 --> 00:42:49.810 it's got a chance of being correct and I think it is. 00:42:49.810 --> 00:42:51.990 Now, what's the x transpose Ax? 00:42:51.990 --> 00:42:54.610 I better give myself enough room for that. 00:42:54.610 --> 00:42:59.180 x transpose Ax for this guy. 00:42:59.180 --> 00:43:07.290 It's two x1 squareds, and two x2 squareds, and two x3 squareds. 00:43:07.290 --> 00:43:10.800 Those come from the diagonal, those were easy. 00:43:10.800 --> 00:43:13.490 Now off the diagonal there's a minus and a minus, 00:43:13.490 --> 00:43:20.010 they come together there'll be a minus two minus two whats? 00:43:20.010 --> 00:43:26.390 Are coming from this one two and two one position, is the x1 x2. 00:43:26.390 --> 00:43:30.900 I'm doing mentally a multiplication 00:43:30.900 --> 00:43:34.300 of this matrix times a row vector 00:43:34.300 --> 00:43:37.680 on the left times a column vector on the right, 00:43:37.680 --> 00:43:41.730 and I know that these numbers show up in the answer. 00:43:41.730 --> 00:43:45.060 The diagonal is the perfect square, 00:43:45.060 --> 00:43:49.110 this off diagonal is a minus two x1 x2, 00:43:49.110 --> 00:43:55.280 and there are no x1 x3-s, and there're minus two x2 x3-s. 00:43:55.280 --> 00:43:58.860 And I believe that that expression is always positive. 00:44:01.670 --> 00:44:06.010 I believe that that curve, that graph, really, 00:44:06.010 --> 00:44:09.720 of that function, this is my function f, 00:44:09.720 --> 00:44:14.770 and I'm in more dimensions now than I can draw, it -- 00:44:14.770 --> 00:44:20.080 but the graph of that function goes upwards. 00:44:20.080 --> 00:44:22.320 It's a bowl. 00:44:22.320 --> 00:44:26.320 Or maybe the right word is -- 00:44:26.320 --> 00:44:34.210 just forgot, what's a long word for bowl? 00:44:34.210 --> 00:44:42.570 Hm, maybe paraboloid, I think, paraboloid comes in. 00:44:42.570 --> 00:44:46.150 I'll edit the tape and get that word in. 00:44:46.150 --> 00:44:54.410 Bowl, let's say, is, that, so that, and if I can -- 00:44:54.410 --> 00:44:56.700 I could complete the squares, I could write that 00:44:56.700 --> 00:44:59.740 as the sum of three squares, and those three squares 00:44:59.740 --> 00:45:02.780 would get multiplied by the three pivots. 00:45:02.780 --> 00:45:05.440 And the pivots are all positive. 00:45:05.440 --> 00:45:08.760 So I would have positive pivots times squares, 00:45:08.760 --> 00:45:11.630 the net result would be a positive function 00:45:11.630 --> 00:45:14.520 and a bowl which goes upwards. 00:45:14.520 --> 00:45:19.230 And then, finally, if I cut -- if I slice through this bowl, 00:45:19.230 --> 00:45:23.730 if I -- now I'm asking you to stretch your visualization 00:45:23.730 --> 00:45:26.680 here, because I'm in four dimensions, 00:45:26.680 --> 00:45:33.610 I've got x1 x2 x3 in the base, and this function is z, or f, 00:45:33.610 --> 00:45:35.000 or something. 00:45:35.000 --> 00:45:38.320 And its graph is going up. 00:45:38.320 --> 00:45:40.530 But I'm in four dimensions, because I've got three 00:45:40.530 --> 00:45:43.410 in the base and then the upward direction, 00:45:43.410 --> 00:45:49.640 but now if I cut through this four-dimensional picture, 00:45:49.640 --> 00:45:55.720 at level one, so, suppose I cut through this thing 00:45:55.720 --> 00:45:58.100 at height one. 00:45:58.100 --> 00:46:01.470 So I take all the points that are at height one. 00:46:04.980 --> 00:46:07.770 That gives me -- 00:46:07.770 --> 00:46:14.180 it gave me an ellipse over there, in that two by two case, 00:46:14.180 --> 00:46:19.350 in this case, this will be the equation of an ellipsoid, 00:46:19.350 --> 00:46:20.830 a football in other words. 00:46:23.770 --> 00:46:25.170 Well, not quite a football. 00:46:25.170 --> 00:46:26.450 A lopsided football. 00:46:26.450 --> 00:46:30.960 What would be, can I try to describe to you what 00:46:30.960 --> 00:46:35.590 the ellipsoid will look like, this ellipsoid, 00:46:35.590 --> 00:46:40.440 I'm sorry that the, that I've ended the matrix right -- 00:46:40.440 --> 00:46:46.230 at the point, let's -- let me be sure you've seen the equation. 00:46:46.230 --> 00:46:51.130 Two x1 squared, two x2 squared, two x3 squared, minus two 00:46:51.130 --> 00:46:57.000 of the cross parts, equal what? 00:46:57.000 --> 00:47:00.720 That is the equation of a football, so what 00:47:00.720 --> 00:47:03.780 do I mean by a football or an ellipsoid? 00:47:03.780 --> 00:47:12.520 I mean that, well, I'll draw a few. 00:47:12.520 --> 00:47:21.750 It's like that, it's got a center, 00:47:21.750 --> 00:47:31.990 and it's got it's got three principal directions. 00:47:31.990 --> 00:47:32.830 This ellipsoid. 00:47:32.830 --> 00:47:35.660 So -- you see what I'm saying, if we have a sphere then all 00:47:35.660 --> 00:47:36.890 directions would be the same. 00:47:39.530 --> 00:47:44.750 If we had a true football, or it's closer to a rugby ball, 00:47:44.750 --> 00:47:48.110 really, because it's more curved than a football, 00:47:48.110 --> 00:47:52.270 it would have one long direction and the other two 00:47:52.270 --> 00:47:54.020 would be equal. 00:47:54.020 --> 00:47:56.240 That would be like having a matrix 00:47:56.240 --> 00:47:59.700 that had one eigenvalue repeated. 00:47:59.700 --> 00:48:02.080 And then one other different. 00:48:02.080 --> 00:48:04.940 So this sphere comes from, like, the identity matrix, 00:48:04.940 --> 00:48:07.600 all eigenvalues the same. 00:48:07.600 --> 00:48:12.370 Our rugby ball comes from a case where -- 00:48:12.370 --> 00:48:17.070 three, the three, two of the three eigenvalues are the same. 00:48:17.070 --> 00:48:20.000 But we know how the case where -- the typical case, 00:48:20.000 --> 00:48:23.340 where the three eigenvalues were all different. 00:48:23.340 --> 00:48:25.551 So this will have -- 00:48:28.440 --> 00:48:31.830 How do I say it, if I look at this ellipsoid correctly, 00:48:31.830 --> 00:48:37.200 it'll have a major axis, it'll have a middle axis, 00:48:37.200 --> 00:48:41.520 and it'll have a minor axis. 00:48:41.520 --> 00:48:44.940 And those three axes will be in the direction 00:48:44.940 --> 00:48:47.450 of the eigenvectors. 00:48:47.450 --> 00:48:50.760 And the lengths of those axes will be 00:48:50.760 --> 00:48:52.496 determined by the eigenvalues. 00:48:55.260 --> 00:48:56.770 I can get -- 00:48:56.770 --> 00:49:02.520 turn this all into linear algebra, because we have -- 00:49:02.520 --> 00:49:06.410 the right thing we know about eigenvectors and eigenvalues, 00:49:06.410 --> 00:49:08.120 for that matrix is what? 00:49:08.120 --> 00:49:11.970 Of -- let me just tell you that, repeat the main linear algebra 00:49:11.970 --> 00:49:13.070 point. 00:49:13.070 --> 00:49:18.970 How could we turn what I said into algebra; 00:49:18.970 --> 00:49:25.050 we would write this A as Q, the eigenvector matrix, 00:49:25.050 --> 00:49:30.370 times lambda, the eigenvalue matrix times Q transposed. 00:49:30.370 --> 00:49:33.010 The principal axis theorem, we'll call it, 00:49:33.010 --> 00:49:33.900 now. 00:49:33.900 --> 00:49:37.980 The eigenvectors tell us the directions of the principal 00:49:37.980 --> 00:49:39.110 axes. 00:49:39.110 --> 00:49:43.240 The eigenvalues tell us the lengths of those axes, actually 00:49:43.240 --> 00:49:45.160 the lengths, or the half-lengths, 00:49:45.160 --> 00:49:48.970 or one over the eigenvalues, it turns out. 00:49:48.970 --> 00:49:53.260 And that is the matrix factorization 00:49:53.260 --> 00:49:55.490 which is the most important matrix 00:49:55.490 --> 00:50:00.540 factorization in our eigenvalue material so far. 00:50:00.540 --> 00:50:04.320 That's diagonalization for a symmetric matrix, 00:50:04.320 --> 00:50:09.350 so instead of the inverse I can write the transposed. 00:50:09.350 --> 00:50:09.930 OK. 00:50:09.930 --> 00:50:14.140 I've -- so what I've tried today is to tell you the -- 00:50:14.140 --> 00:50:20.510 what's going on with positive definite matrices. 00:50:20.510 --> 00:50:23.020 Ah, you see all how all these pieces are there 00:50:23.020 --> 00:50:25.721 and linear algebra connects them. 00:50:25.721 --> 00:50:26.220 OK. 00:50:28.760 --> 00:50:30.680 See you on Friday.