WEBVTT 00:00:11.825 --> 00:00:12.325 OK. 00:00:14.830 --> 00:00:15.900 cameras are rolling. 00:00:15.900 --> 00:00:21.840 This is lecture fourteen, starting a new chapter. 00:00:21.840 --> 00:00:24.660 Chapter about orthogonality. 00:00:24.660 --> 00:00:29.480 What it means for vectors to be orthogonal. 00:00:29.480 --> 00:00:33.070 What it means for subspaces to be orthogonal. 00:00:33.070 --> 00:00:37.370 What it means for bases to be orthogonal. 00:00:37.370 --> 00:00:44.460 So ninety degrees, this is a ninety-degree chapter. 00:00:44.460 --> 00:00:47.800 So what does it mean -- 00:00:47.800 --> 00:00:50.890 let me jump to subspaces. 00:00:50.890 --> 00:00:54.830 Because I've drawn here the big picture. 00:00:54.830 --> 00:00:59.910 This is the 18.06 picture here. 00:00:59.910 --> 00:01:04.800 And hold it down, guys. 00:01:04.800 --> 00:01:09.280 So this is the picture and we know a lot about that picture 00:01:09.280 --> 00:01:10.550 already. 00:01:10.550 --> 00:01:14.570 We know the dimension of every subspace. 00:01:14.570 --> 00:01:22.040 We know that these dimensions are r and n-r. 00:01:22.040 --> 00:01:26.320 We know that these dimensions are r and m-r. 00:01:29.680 --> 00:01:33.780 What I want to show now is what this figure is saying, 00:01:33.780 --> 00:01:38.140 that the angle -- 00:01:38.140 --> 00:01:43.520 the figure is just my attempt to draw what I'm now going to say, 00:01:43.520 --> 00:01:50.510 that the angle between these subspaces is ninety degrees. 00:01:50.510 --> 00:01:53.890 And the angle between these subspaces is ninety degrees. 00:01:53.890 --> 00:01:55.670 Now I have to say what does that mean? 00:01:55.670 --> 00:02:01.080 What does it mean for subspaces to be orthogonal? 00:02:01.080 --> 00:02:06.540 But I hope you appreciate the beauty of this picture, 00:02:06.540 --> 00:02:11.340 that that those subspaces are going 00:02:11.340 --> 00:02:14.170 to come out to be orthogonal. 00:02:14.170 --> 00:02:18.140 Those two and also those two. 00:02:18.140 --> 00:02:24.870 So that's like one point, one important point 00:02:24.870 --> 00:02:28.960 to step forward in understanding those subspaces. 00:02:28.960 --> 00:02:31.140 We knew what each subspace was like, 00:02:31.140 --> 00:02:33.500 we could compute bases for them. 00:02:33.500 --> 00:02:35.440 Now we know more. 00:02:35.440 --> 00:02:38.240 Or we will in a few minutes. 00:02:38.240 --> 00:02:38.750 OK. 00:02:38.750 --> 00:02:43.110 I have to say first of all what does it mean for two vectors 00:02:43.110 --> 00:02:44.920 to be orthogonal? 00:02:44.920 --> 00:02:46.470 So let me start with that. 00:02:49.560 --> 00:02:50.560 Orthogonal vectors. 00:02:57.010 --> 00:03:00.610 The word orthogonal is -- is just another word 00:03:00.610 --> 00:03:02.620 for perpendicular. 00:03:02.620 --> 00:03:05.510 It means that in n-dimensional space 00:03:05.510 --> 00:03:10.290 the angle between those vectors is ninety degrees. 00:03:10.290 --> 00:03:13.480 It means that they form a right triangle. 00:03:13.480 --> 00:03:20.940 It even means that the going way back to the Greeks that this 00:03:20.940 --> 00:03:29.010 angle that this triangle a vector x, a vector x, 00:03:29.010 --> 00:03:36.560 and a vector x+y -- of course that'll be the hypotenuse, 00:03:36.560 --> 00:03:42.750 so what was it the Greeks figured out and it's neat. 00:03:42.750 --> 00:03:45.860 It's the fact that the -- 00:03:45.860 --> 00:03:50.980 so these are orthogonal, this is a right angle, if -- 00:03:50.980 --> 00:03:57.660 so let me put the great name down, Pythagoras, 00:03:57.660 --> 00:04:00.540 I'm looking for -- what I looking for? 00:04:00.540 --> 00:04:05.980 I'm looking for the condition if you give me two vectors, how do 00:04:05.980 --> 00:04:08.540 I know if they're orthogonal? 00:04:08.540 --> 00:04:11.490 How can I tell two perpendicular vectors? 00:04:11.490 --> 00:04:13.830 And actually you probably know the answer. 00:04:13.830 --> 00:04:15.770 Let me write the answer down. 00:04:15.770 --> 00:04:20.550 Orthogonal vectors, what's the test for orthogonality? 00:04:20.550 --> 00:04:29.430 I take the dot product which I tend to write as x transpose y, 00:04:29.430 --> 00:04:33.060 because that's a row times a column, 00:04:33.060 --> 00:04:38.390 and that matrix multiplication just gives me the right thing, 00:04:38.390 --> 00:04:45.340 that x1y1+x2y2 and so on, so these vectors are orthogonal 00:04:45.340 --> 00:04:52.560 if this result x transpose y is zero. 00:04:52.560 --> 00:04:54.130 That's the test. 00:04:54.130 --> 00:04:55.340 OK. 00:04:55.340 --> 00:05:00.850 Can I connect that to other things? 00:05:00.850 --> 00:05:06.480 I mean -- it's just beautiful that here we have we're in n 00:05:06.480 --> 00:05:09.390 dimensions, we've got a couple of vectors, 00:05:09.390 --> 00:05:11.620 we want to know the angle between them, 00:05:11.620 --> 00:05:16.750 and the right thing to look at is the simplest thing that you 00:05:16.750 --> 00:05:18.930 could imagine, the dot product. 00:05:18.930 --> 00:05:19.600 OK. 00:05:19.600 --> 00:05:20.890 Now why? 00:05:20.890 --> 00:05:24.430 So I'm answering the question now why -- 00:05:24.430 --> 00:05:33.500 let's add some justification to this fact, that's the test. 00:05:33.500 --> 00:05:38.050 OK, so Pythagoras would say we've got a right triangle, 00:05:38.050 --> 00:05:41.400 if that length squared plus that length squared equals 00:05:41.400 --> 00:05:43.170 that length squared. 00:05:43.170 --> 00:05:51.800 OK, can I write it as x squared plus y squared equals 00:05:51.800 --> 00:05:54.890 x plus y squared? 00:05:54.890 --> 00:05:59.120 Now don't, please don't think that this is always true. 00:05:59.120 --> 00:06:03.360 This is only going to be true in this -- 00:06:03.360 --> 00:06:07.600 it's going to be equivalent to orthogonality. 00:06:07.600 --> 00:06:11.180 For other triangles of course, it's not true. 00:06:11.180 --> 00:06:12.540 For other triangles it's not. 00:06:12.540 --> 00:06:16.860 But for a right triangle somehow that fact 00:06:16.860 --> 00:06:18.810 should connect to that fact. 00:06:18.810 --> 00:06:20.240 Can we just make that connection? 00:06:23.580 --> 00:06:28.090 What's the connection between this test for orthogonality 00:06:28.090 --> 00:06:31.040 and this statement of orthogonality? 00:06:31.040 --> 00:06:35.520 Well, I guess I have to say what is the length squared? 00:06:35.520 --> 00:06:42.370 So let's continue on the board underneath with that equation. 00:06:42.370 --> 00:06:47.800 Give me another way to express the length squared of a vector. 00:06:47.800 --> 00:06:49.960 And let me just give you a vector. 00:06:49.960 --> 00:06:53.420 The vector one, two, three. 00:06:53.420 --> 00:06:55.730 That's in three dimensions. 00:06:55.730 --> 00:07:00.720 What is the length squared of the vector x equal one, two, 00:07:00.720 --> 00:07:03.410 three? 00:07:03.410 --> 00:07:06.820 So how do you find the length squared? 00:07:06.820 --> 00:07:10.710 Well, really you just, you want the length of that vector that 00:07:10.710 --> 00:07:15.220 goes along one -- up two, and out three -- 00:07:15.220 --> 00:07:19.910 and we'll come back to that right triangle stuff. 00:07:19.910 --> 00:07:25.928 The length squared is exactly x transpose x. 00:07:28.870 --> 00:07:31.230 Whenever I see x transpose x, I know 00:07:31.230 --> 00:07:35.040 I've got a number that's positive. 00:07:35.040 --> 00:07:37.750 It's a length squared unless it -- 00:07:37.750 --> 00:07:40.440 unless x happens to be the zero vector, 00:07:40.440 --> 00:07:44.500 that's the one case where the length is zero. 00:07:44.500 --> 00:07:49.770 So right -- this is just x1 squared plus x2 squared plus 00:07:49.770 --> 00:07:51.920 so on, plus xn squared. 00:07:51.920 --> 00:07:55.830 So one -- in the example I gave you what was the length squared 00:07:55.830 --> 00:07:59.230 of that vector one, two, three? 00:07:59.230 --> 00:08:02.070 So you square those, you get one, four and nine, 00:08:02.070 --> 00:08:04.310 you add, you get fourteen. 00:08:04.310 --> 00:08:08.590 So the vector one, two, three has length fourteen. 00:08:08.590 --> 00:08:13.000 So let me just put down a vector here. 00:08:13.000 --> 00:08:15.995 Let x be the vector one, two, three. 00:08:19.220 --> 00:08:22.060 Let me cook up a vector that's orthogonal to it. 00:08:25.550 --> 00:08:27.590 So what's the vector that's orthogonal? 00:08:27.590 --> 00:08:33.270 So right down here, x squared is one plus four plus nine, 00:08:33.270 --> 00:08:38.480 fourteen, let me cook up a vector that's orthogonal to it, 00:08:38.480 --> 00:08:47.170 we'll get right that that -- those two vectors are 00:08:47.170 --> 00:08:55.320 orthogonal, the length of y squared is five, 00:08:55.320 --> 00:09:02.520 and x plus y is one and two making three, 00:09:02.520 --> 00:09:06.290 two and minus one making one, three and zero making three, 00:09:06.290 --> 00:09:13.010 and the length of this squared is nine plus one plus nine, 00:09:13.010 --> 00:09:15.700 nineteen. 00:09:15.700 --> 00:09:20.810 And sure enough, I haven't proved anything. 00:09:20.810 --> 00:09:28.900 I've just like checked to see that my test x transpose 00:09:28.900 --> 00:09:31.530 y equals zero, which is true, right? 00:09:31.530 --> 00:09:35.370 Everybody sees that x transpose y is zero here? 00:09:35.370 --> 00:09:37.740 That's maybe the main point. 00:09:37.740 --> 00:09:41.910 That you should get really quick at doing x transpose y, 00:09:41.910 --> 00:09:45.980 so it's just this plus this plus this and that's zero. 00:09:45.980 --> 00:09:51.730 And sure enough, that clicks with fourteen plus five 00:09:51.730 --> 00:09:53.190 agreeing with nineteen. 00:09:53.190 --> 00:09:56.780 Now let me just do that in letters. 00:09:56.780 --> 00:10:01.110 So that's y transpose y. 00:10:01.110 --> 00:10:08.150 And this is x plus y transpose x plus y. 00:10:08.150 --> 00:10:09.290 OK. 00:10:09.290 --> 00:10:12.530 So I'm looking, again, this isn't always true. 00:10:12.530 --> 00:10:14.110 I repeat. 00:10:14.110 --> 00:10:18.020 This is going to be true when we have a right angle. 00:10:18.020 --> 00:10:20.210 And let's just -- well, of course, 00:10:20.210 --> 00:10:22.850 I'm just going to simplify this stuff here. 00:10:22.850 --> 00:10:25.960 There's an x transpose x there. 00:10:25.960 --> 00:10:29.850 And there's a y transpose y there. 00:10:29.850 --> 00:10:34.450 And there's an x transpose y. 00:10:34.450 --> 00:10:36.610 And there's a y transpose x. 00:10:41.700 --> 00:10:46.990 I knew I could do that simplification because I'm just 00:10:46.990 --> 00:10:50.150 doing matrix multiplication and I've just followed the rules. 00:10:50.150 --> 00:10:50.880 OK. 00:10:50.880 --> 00:10:53.650 So x transpose x-s cancel. 00:10:53.650 --> 00:10:55.910 Y transpose y-s cancel. 00:10:55.910 --> 00:10:57.950 And what about these guys? 00:10:57.950 --> 00:11:03.400 What can you tell me about the inner product of x with y 00:11:03.400 --> 00:11:06.890 and the inner product of y with x? 00:11:06.890 --> 00:11:09.620 Is there a difference? 00:11:09.620 --> 00:11:15.150 I think if we -- while we're doing real vectors, 00:11:15.150 --> 00:11:18.310 which is all we're doing now, there isn't a difference, 00:11:18.310 --> 00:11:19.660 there's no difference. 00:11:19.660 --> 00:11:23.690 If I take x transpose y, that'll give me zero, 00:11:23.690 --> 00:11:28.840 if I took y transpose x I would have the same x1y1 and x2y2 00:11:28.840 --> 00:11:35.280 and x3y3, it would be the same, so this is -- 00:11:35.280 --> 00:11:37.820 this is the same as that, this is really 00:11:37.820 --> 00:11:40.070 I'll knock that guy out and say two of these. 00:11:43.700 --> 00:11:47.360 So actually that's the -- 00:11:47.360 --> 00:11:51.960 this equation boiled down to this thing being zero. 00:11:51.960 --> 00:11:52.670 Right? 00:11:52.670 --> 00:11:56.010 Everything else canceled and this equation 00:11:56.010 --> 00:11:57.250 boiled down to that one. 00:11:57.250 --> 00:12:00.450 So that's really all I wanted. 00:12:00.450 --> 00:12:08.170 I just wanted to check that Pythagoras for a right triangle 00:12:08.170 --> 00:12:13.250 led me to this -- of course I cancel the two now. 00:12:13.250 --> 00:12:14.680 No problem. 00:12:14.680 --> 00:12:18.880 To x transpose y equals zero as the test. 00:12:18.880 --> 00:12:19.610 Fair enough. 00:12:19.610 --> 00:12:21.190 OK. 00:12:21.190 --> 00:12:23.150 You knew it was coming. 00:12:23.150 --> 00:12:27.010 The dot product of orthogonal vectors is zero. 00:12:27.010 --> 00:12:31.010 It's just -- I just want to say that's really neat. 00:12:31.010 --> 00:12:32.760 That it comes out so well. 00:12:32.760 --> 00:12:34.830 All right. 00:12:34.830 --> 00:12:38.760 Now what about -- so now I know if two -- 00:12:38.760 --> 00:12:40.700 when two vectors are orthogonal. 00:12:40.700 --> 00:12:45.860 By the way, what about if one of these guys is the zero vector? 00:12:45.860 --> 00:12:51.660 Suppose x is the zero vector, and y is whatever. 00:12:51.660 --> 00:12:55.010 Are they orthogonal? 00:12:55.010 --> 00:12:57.190 Sure. 00:12:57.190 --> 00:12:59.310 In math the one thing about math is you're 00:12:59.310 --> 00:13:01.910 supposed to follow the rules. 00:13:01.910 --> 00:13:05.450 So you're supposed to -- if x is the zero vector, 00:13:05.450 --> 00:13:09.330 you're supposed to take the zero vector dotted with y 00:13:09.330 --> 00:13:12.180 and of course you always get zero. 00:13:12.180 --> 00:13:17.340 So just so we're all sure, the zero vector 00:13:17.340 --> 00:13:20.730 is orthogonal to everybody. 00:13:20.730 --> 00:13:24.770 But what I want to -- 00:13:24.770 --> 00:13:32.240 what I now want to think about is subspaces. 00:13:32.240 --> 00:13:37.530 What does it mean for me to say that some subspace is 00:13:37.530 --> 00:13:39.320 orthogonal to some other subspace? 00:13:39.320 --> 00:13:41.110 So OK. 00:13:41.110 --> 00:13:42.590 Now I've got to write this down. 00:13:42.590 --> 00:13:47.960 So because we're defining definition of subspace 00:13:47.960 --> 00:14:02.390 S is orthogonal so to subspace let's say T, 00:14:02.390 --> 00:14:04.130 so I've got a couple of subspaces. 00:14:07.350 --> 00:14:11.480 And what should it mean for those guys to be orthogonal? 00:14:11.480 --> 00:14:14.780 It's just sort of what's the natural extension 00:14:14.780 --> 00:14:19.990 from orthogonal vectors to orthogonal subspaces? 00:14:19.990 --> 00:14:26.820 Well, and in particular, let's think 00:14:26.820 --> 00:14:34.410 of some orthogonal subspaces, like this wall. 00:14:34.410 --> 00:14:37.300 Let's say in three dimensions. 00:14:37.300 --> 00:14:41.320 So the blackboard extended to infinity, right, is a -- 00:14:41.320 --> 00:14:44.950 is a subspace, a plane, a two-dimensional subspace. 00:14:47.680 --> 00:14:51.310 It's a little bumpy but anyway, it's a -- 00:14:51.310 --> 00:14:57.001 think of it as a subspace, let me take the floor as another 00:14:57.001 --> 00:14:57.500 subspace. 00:15:01.850 --> 00:15:07.840 Again, it's not a great subspace, 00:15:07.840 --> 00:15:15.400 MIT only built it like so-so, but I'll 00:15:15.400 --> 00:15:18.840 put the origin right here. 00:15:18.840 --> 00:15:23.040 So the origin of the world is right there. 00:15:23.040 --> 00:15:23.540 OK. 00:15:26.730 --> 00:15:30.340 Thereby giving linear algebra its proper importance in this. 00:15:30.340 --> 00:15:31.190 OK. 00:15:31.190 --> 00:15:34.290 So there is one subspace, there's another one. 00:15:34.290 --> 00:15:35.940 The floor. 00:15:35.940 --> 00:15:42.200 And are they orthogonal? 00:15:42.200 --> 00:15:44.130 What does it mean for two subspaces 00:15:44.130 --> 00:15:46.460 to be orthogonal and in that special case 00:15:46.460 --> 00:15:47.650 are they orthogonal? 00:15:47.650 --> 00:15:48.290 All right. 00:15:48.290 --> 00:15:50.950 Let's finish this sentence. 00:15:50.950 --> 00:15:58.501 What does it mean means we have to know 00:15:58.501 --> 00:15:59.500 what we're talking about 00:15:59.500 --> 00:16:00.040 here. 00:16:00.040 --> 00:16:05.310 So what would be a reasonable idea of orthogonal? 00:16:05.310 --> 00:16:08.740 Well, let me put the right thing up. 00:16:08.740 --> 00:16:17.930 It means that every vector in S, every vector in S, 00:16:17.930 --> 00:16:23.565 is orthogonal to -- 00:16:27.290 --> 00:16:30.080 what I going to say? 00:16:30.080 --> 00:16:38.000 Every vector in T. 00:16:38.000 --> 00:16:42.990 That's a reasonable and it's a good 00:16:42.990 --> 00:16:45.560 and it's the right definition for two subspaces 00:16:45.560 --> 00:16:47.010 to be orthogonal. 00:16:47.010 --> 00:16:50.740 But I just want you to see, hey, what does that mean? 00:16:50.740 --> 00:16:53.620 So answer the question about the -- 00:16:53.620 --> 00:16:57.440 the blackboard and the floor. 00:16:57.440 --> 00:17:01.440 Are those two subspaces, they're two-dimensional, right, 00:17:01.440 --> 00:17:03.830 and we're in R^3. 00:17:03.830 --> 00:17:09.665 It's like a xz plane or something and a xy plane. 00:17:15.089 --> 00:17:17.109 Are they orthogonal? 00:17:17.109 --> 00:17:19.560 Is every vector in the blackboard orthogonal 00:17:19.560 --> 00:17:23.349 to every vector in the floor, starting from the origin 00:17:23.349 --> 00:17:24.140 right there? 00:17:27.640 --> 00:17:28.900 Yes or no? 00:17:28.900 --> 00:17:31.010 I could take a vote. 00:17:31.010 --> 00:17:35.040 Well we get some yeses and some noes. 00:17:35.040 --> 00:17:36.170 No is the answer. 00:17:36.170 --> 00:17:37.810 They're not. 00:17:37.810 --> 00:17:41.340 You can tell me a vector in the blackboard 00:17:41.340 --> 00:17:43.995 and a vector in the floor that are not orthogonal. 00:17:49.360 --> 00:17:54.960 Well you can tell me quite a few, I guess. 00:17:54.960 --> 00:17:59.140 Maybe like I could take some forty-five-degree guy 00:17:59.140 --> 00:18:05.730 in the blackboard, and something in the floor, 00:18:05.730 --> 00:18:08.580 they're not at ninety degrees, right? 00:18:08.580 --> 00:18:10.780 In fact, even more, you could tell me 00:18:10.780 --> 00:18:17.140 a vector that's in both the blackboard plane and the floor 00:18:17.140 --> 00:18:20.550 plane, so it's certainly not orthogonal to itself. 00:18:20.550 --> 00:18:25.600 So for sure, those two planes aren't orthogonal. 00:18:25.600 --> 00:18:26.950 What would that be? 00:18:26.950 --> 00:18:29.350 So what's a vector that's -- 00:18:29.350 --> 00:18:32.320 in both of those planes? 00:18:32.320 --> 00:18:35.170 It's this guy running along the crack 00:18:35.170 --> 00:18:39.540 there, in the intersection, the intersection. 00:18:39.540 --> 00:18:41.360 A vector, you know -- 00:18:41.360 --> 00:18:45.460 if two subspaces meet at some vector, well then for sure 00:18:45.460 --> 00:18:51.170 they're not orthogonal, because that vector is in one 00:18:51.170 --> 00:18:54.240 and it's in the other, and it's not orthogonal to itself 00:18:54.240 --> 00:18:55.770 unless it's zero. 00:18:55.770 --> 00:19:01.000 So the only I mean so orthogonal is 00:19:01.000 --> 00:19:04.220 for me to say these two subspaces are orthogonal first 00:19:04.220 --> 00:19:09.080 of all I'm certainly saying that they don't intersect 00:19:09.080 --> 00:19:14.550 in any nonzero vector. 00:19:14.550 --> 00:19:18.430 But also I mean more than that just not intersecting 00:19:18.430 --> 00:19:18.990 isn't good 00:19:18.990 --> 00:19:19.490 enough. 00:19:19.490 --> 00:19:26.150 So give me an example, oh, let's say in the plane, oh well, 00:19:26.150 --> 00:19:30.090 when do we have orthogonal subspaces in the plane? 00:19:30.090 --> 00:19:32.100 Yeah, tell me in the plane, so we don't -- 00:19:32.100 --> 00:19:34.960 there aren't that many different subspaces in the plane. 00:19:34.960 --> 00:19:38.385 What what have we got in the plane as possible subspaces? 00:19:40.900 --> 00:19:44.420 The zero vector, real small. 00:19:44.420 --> 00:19:47.330 A line through the origin. 00:19:47.330 --> 00:19:49.570 Or the whole plane. 00:19:49.570 --> 00:19:50.420 OK. 00:19:50.420 --> 00:19:55.769 Now so when is a line through the origin orthogonal 00:19:55.769 --> 00:19:56.560 to the whole plane? 00:19:59.870 --> 00:20:02.070 Never, right, never. 00:20:02.070 --> 00:20:05.620 When is a line through the origin orthogonal to the zero 00:20:05.620 --> 00:20:06.120 subspace? 00:20:09.310 --> 00:20:10.180 Always. 00:20:10.180 --> 00:20:10.910 Right. 00:20:10.910 --> 00:20:13.480 When is a line through the origin orthogonal 00:20:13.480 --> 00:20:15.920 to a different line through the origin? 00:20:15.920 --> 00:20:19.880 Well, that's the case that we all have a clear picture 00:20:19.880 --> 00:20:21.070 of, they -- 00:20:21.070 --> 00:20:23.610 the two lines have to meet at ninety degrees. 00:20:23.610 --> 00:20:28.340 They have only the -- so that's like this simple case 00:20:28.340 --> 00:20:29.090 I'm talking about. 00:20:29.090 --> 00:20:31.970 There's one subspace, there's the other subspace. 00:20:31.970 --> 00:20:33.770 They only meet at zero. 00:20:33.770 --> 00:20:35.570 And they're orthogonal. 00:20:35.570 --> 00:20:36.340 OK. 00:20:36.340 --> 00:20:37.930 Now. 00:20:37.930 --> 00:20:43.540 So we now know what it means for two subspaces to be orthogonal. 00:20:43.540 --> 00:20:47.240 And now I want to say that this is true for the row 00:20:47.240 --> 00:20:49.080 space and the null space. 00:20:49.080 --> 00:20:49.740 OK. 00:20:49.740 --> 00:20:53.880 So that's the neat fact. 00:20:53.880 --> 00:21:05.720 So row space is orthogonal to the null space. 00:21:05.720 --> 00:21:07.435 Now how did I come up with that? 00:21:12.540 --> 00:21:16.500 But you see the rank it's great, that means that these -- 00:21:16.500 --> 00:21:19.020 that these subspaces are just the right things, 00:21:19.020 --> 00:21:23.220 they're just cutting the whole space up 00:21:23.220 --> 00:21:27.560 into two perpendicular subspaces. 00:21:27.560 --> 00:21:28.060 OK. 00:21:28.060 --> 00:21:28.560 So why? 00:21:33.710 --> 00:21:38.040 Well, what have I got to work with? 00:21:38.040 --> 00:21:41.230 All I know is the null space. 00:21:41.230 --> 00:21:46.360 The null space has vectors that solve Ax equals zero. 00:21:46.360 --> 00:21:49.780 So this is a guy x. 00:21:49.780 --> 00:21:53.940 x is in the null space. 00:21:53.940 --> 00:21:57.350 Then Ax is zero. 00:21:57.350 --> 00:22:03.250 So why is it orthogonal to the rows of A? 00:22:03.250 --> 00:22:05.340 If I write down Ax equals zero, which 00:22:05.340 --> 00:22:07.720 is all I know about the null space, 00:22:07.720 --> 00:22:13.130 then I guess I want you to see that that's telling me, 00:22:13.130 --> 00:22:16.430 just that equation right there is telling me 00:22:16.430 --> 00:22:19.670 that the rows of A, let me write it out. 00:22:19.670 --> 00:22:24.000 There's row one of A. 00:22:24.000 --> 00:22:24.540 Row two. 00:22:27.290 --> 00:22:32.300 Row m of A. that's A. 00:22:32.300 --> 00:22:34.530 And it's multiplying X. 00:22:34.530 --> 00:22:36.560 And it's producing zero. 00:22:36.560 --> 00:22:37.060 OK. 00:22:41.550 --> 00:22:47.200 Written out that way you'll see it. 00:22:47.200 --> 00:22:50.200 So I'm saying that a vector in the row space 00:22:50.200 --> 00:22:54.250 is perpendicular to this guy X in the null space. 00:22:54.250 --> 00:22:57.310 And you see why? 00:22:57.310 --> 00:22:59.550 Because this equation is telling you 00:22:59.550 --> 00:23:06.330 that row one of A multiplying that's a dot product, right? 00:23:06.330 --> 00:23:11.480 Row one of A dot product with this x is producing this zero. 00:23:11.480 --> 00:23:15.970 So x is orthogonal to the first row. 00:23:15.970 --> 00:23:17.670 And to the second row. 00:23:17.670 --> 00:23:20.630 Row two of A, x is giving that zero. 00:23:20.630 --> 00:23:23.590 Row m of A times x is giving that zero. 00:23:23.590 --> 00:23:25.020 So x is -- 00:23:25.020 --> 00:23:27.380 the equation is telling me that x 00:23:27.380 --> 00:23:31.309 is orthogonal to all the rows. 00:23:31.309 --> 00:23:32.600 Right, it's just sitting there. 00:23:32.600 --> 00:23:36.380 That's all we -- it had to be sitting there because we 00:23:36.380 --> 00:23:40.000 didn't know anything more about the null space than this. 00:23:40.000 --> 00:23:46.120 And now I guess to be totally complete here 00:23:46.120 --> 00:23:49.350 I'd now check that x is orthogonal 00:23:49.350 --> 00:23:51.720 to each separate row. 00:23:51.720 --> 00:23:55.710 But what else strictly speaking do I have to do? 00:23:58.670 --> 00:24:02.340 To show that those subspaces are orthogonal, 00:24:02.340 --> 00:24:05.580 I have to take this x in the null space and show that 00:24:05.580 --> 00:24:10.380 it's orthogonal to every vector in the row space, 00:24:10.380 --> 00:24:12.740 every vector in the row space, so what -- 00:24:12.740 --> 00:24:15.330 what else is in the row space? 00:24:15.330 --> 00:24:18.970 This row is in the row space, that row is in the row space, 00:24:18.970 --> 00:24:22.870 they're all there, but it's not the whole row space, 00:24:22.870 --> 00:24:24.990 right, we just have to like remember, what does it 00:24:24.990 --> 00:24:29.610 mean, what does that word space telling us? 00:24:29.610 --> 00:24:34.470 And what else is in the row space? 00:24:34.470 --> 00:24:37.810 Besides the rows? 00:24:37.810 --> 00:24:41.990 All their combinations. 00:24:41.990 --> 00:24:44.680 So I really have to check that sure enough 00:24:44.680 --> 00:24:46.770 if x is perpendicular to row one, 00:24:46.770 --> 00:24:49.720 row two, all the different separate rows, 00:24:49.720 --> 00:24:54.110 then also x is perpendicular to a combination of the rows. 00:24:54.110 --> 00:24:57.000 And that's just matrix multiplication again. 00:24:57.000 --> 00:25:03.300 You know, I have row one transpose x is zero, 00:25:03.300 --> 00:25:11.000 so on, row two transpose x is zero, 00:25:11.000 --> 00:25:16.800 so I'm entitled to multiply that by some c1, this by some c2, 00:25:16.800 --> 00:25:20.700 I still have zeroes, I'm entitled to add, 00:25:20.700 --> 00:25:24.760 so I have c1 row one so -- so all this when I put that 00:25:24.760 --> 00:25:33.290 together that's big parentheses c1 row one plus c2 row two 00:25:33.290 --> 00:25:34.780 and so on. 00:25:34.780 --> 00:25:38.220 Transpose x is zero. 00:25:38.220 --> 00:25:38.720 Right? 00:25:38.720 --> 00:25:41.380 I just added the zeroes and got zero, 00:25:41.380 --> 00:25:43.760 and I just added these following the rule. 00:25:46.550 --> 00:25:48.490 No big deal. 00:25:48.490 --> 00:25:51.890 The whole point was right sitting in that. 00:25:51.890 --> 00:25:54.610 OK. 00:25:54.610 --> 00:26:02.480 So if I come back to this figure now I'm like a happier person. 00:26:02.480 --> 00:26:05.090 Because I have this -- 00:26:05.090 --> 00:26:10.730 I now see how those subspaces are oriented. 00:26:10.730 --> 00:26:14.090 And these subspaces are also oriented. 00:26:14.090 --> 00:26:20.200 Well, actually why is that orthogonality? 00:26:20.200 --> 00:26:23.660 Well, it's the same statement for A transpose 00:26:23.660 --> 00:26:25.330 that that one was for A. 00:26:25.330 --> 00:26:27.670 So I won't take time to prove it again 00:26:27.670 --> 00:26:32.500 because we've checked it for every matrix 00:26:32.500 --> 00:26:35.960 and A transpose is just as good a matrix as A. 00:26:35.960 --> 00:26:39.520 So we're orthogonal over there. 00:26:39.520 --> 00:26:46.180 So we really have carved up this -- 00:26:46.180 --> 00:26:50.070 this was like carving up m-dimensional space 00:26:50.070 --> 00:26:56.070 into two subspaces and this one was carving up 00:26:56.070 --> 00:27:01.810 n-dimensional space into two subspaces. 00:27:01.810 --> 00:27:06.500 And well, one more thing here. 00:27:06.500 --> 00:27:07.550 One more important thing. 00:27:11.110 --> 00:27:13.330 Let me move into three dimensions. 00:27:15.990 --> 00:27:22.780 Tell me a couple of orthogonal subspaces in three dimensions 00:27:22.780 --> 00:27:27.450 that somehow don't carve up the whole space, 00:27:27.450 --> 00:27:30.330 there's stuff left there. 00:27:30.330 --> 00:27:34.830 I'm thinking of a couple of orthogonal lines. 00:27:34.830 --> 00:27:38.550 If I -- suppose I'm in three dimensions, R^3. 00:27:38.550 --> 00:27:43.510 And I have one line, one one-dimensional subspace, 00:27:43.510 --> 00:27:46.030 and a perpendicular one. 00:27:46.030 --> 00:27:51.170 Could those be the row space and the null space? 00:27:51.170 --> 00:27:54.590 Could those be the row space and the null space? 00:27:54.590 --> 00:28:00.520 Could I be in three dimensions and have 00:28:00.520 --> 00:28:05.930 a row space that's a line and a null space that's a line? 00:28:05.930 --> 00:28:07.270 No. 00:28:07.270 --> 00:28:10.180 Why not? 00:28:10.180 --> 00:28:11.840 Because the dimensions aren't right. 00:28:11.840 --> 00:28:12.340 Right? 00:28:12.340 --> 00:28:14.080 The dimensions are no good. 00:28:14.080 --> 00:28:19.050 The dimensions here, r and n-r, they add up to three, 00:28:19.050 --> 00:28:21.490 they add up to n. 00:28:21.490 --> 00:28:23.220 If I take -- 00:28:23.220 --> 00:28:26.800 just follow that example -- 00:28:26.800 --> 00:28:30.910 if the row space is one-dimensional, 00:28:30.910 --> 00:28:36.030 suppose A is what's a good in R^3, 00:28:36.030 --> 00:28:39.560 I want a one-dimensional row space, let me take one, two, 00:28:39.560 --> 00:28:43.470 five, two, four, ten. 00:28:43.470 --> 00:28:45.330 What's the dimension of that row space? 00:28:48.590 --> 00:28:50.220 One. 00:28:50.220 --> 00:28:52.260 What's the dimension of the null space? 00:28:56.160 --> 00:28:59.130 Tell what's the null space look like in that case? 00:28:59.130 --> 00:29:01.670 The row space is a line, right? 00:29:01.670 --> 00:29:06.440 One-dimensional, it's just a line through one, two, five. 00:29:06.440 --> 00:29:08.370 Geometrically what's the row space look like? 00:29:13.950 --> 00:29:15.660 What's its dimension? 00:29:15.660 --> 00:29:20.920 So here r here n is three, the rank 00:29:20.920 --> 00:29:26.610 is one, so the dimension of the null space, 00:29:26.610 --> 00:29:32.180 so I'm looking at this x, x1, x2, x3. 00:29:32.180 --> 00:29:33.680 To give zero. 00:29:33.680 --> 00:29:43.300 So the dimension of the null space is we all know is two. 00:29:43.300 --> 00:29:43.800 Right. 00:29:43.800 --> 00:29:45.440 It's a plane. 00:29:45.440 --> 00:29:49.770 And now actually we know, we see better, what plane is it? 00:29:49.770 --> 00:29:52.660 What plane is it? 00:29:52.660 --> 00:29:57.500 It's the plane that's perpendicular to one, 00:29:57.500 --> 00:29:59.950 two, five. 00:29:59.950 --> 00:30:00.450 Right? 00:30:00.450 --> 00:30:01.350 We now see. 00:30:01.350 --> 00:30:04.710 In fact the two, four, ten didn't actually 00:30:04.710 --> 00:30:07.000 have any effect at all. 00:30:07.000 --> 00:30:09.750 I could have just ignored that. 00:30:09.750 --> 00:30:14.340 That didn't change the row space or the null space. 00:30:14.340 --> 00:30:17.051 I'll just make that one equation. 00:30:17.051 --> 00:30:17.550 Yeah. 00:30:17.550 --> 00:30:18.000 OK. 00:30:18.000 --> 00:30:18.499 Sure. 00:30:18.499 --> 00:30:21.210 That's the easiest to deal with. 00:30:21.210 --> 00:30:22.130 One equation. 00:30:22.130 --> 00:30:24.440 Three unknowns. 00:30:24.440 --> 00:30:32.670 And I want to ask -- 00:30:32.670 --> 00:30:37.180 what would the equation give me the null space, 00:30:37.180 --> 00:30:41.380 and you would have said back in September 00:30:41.380 --> 00:30:43.410 you would have said it gives you a plane, 00:30:43.410 --> 00:30:46.530 and we're completely right. 00:30:46.530 --> 00:30:49.530 And the plane it gives you, the normal vector, 00:30:49.530 --> 00:30:53.610 you remember in calculus, there was this dumb normal vector 00:30:53.610 --> 00:30:54.760 called N. 00:30:54.760 --> 00:30:55.496 Well there it is. 00:30:55.496 --> 00:30:56.120 One, two, five. 00:30:56.120 --> 00:30:56.620 OK. 00:30:56.620 --> 00:31:08.820 What is the what's the point I want to make here? 00:31:08.820 --> 00:31:10.010 I want to make -- 00:31:10.010 --> 00:31:14.300 I want to emphasize that not only are the -- 00:31:14.300 --> 00:31:15.410 let me write it in words. 00:31:19.930 --> 00:31:33.900 So I want to write the null space and the row space are 00:31:33.900 --> 00:31:40.290 orthogonal, that's this neat fact, which we've -- 00:31:40.290 --> 00:31:43.240 we've just checked from Ax equals zero, 00:31:43.240 --> 00:31:48.750 but now I want to say more because there's a little more 00:31:48.750 --> 00:31:51.190 that's true. 00:31:51.190 --> 00:31:54.940 Their dimensions add to the whole space. 00:31:54.940 --> 00:31:57.660 So that's like a little extra information. 00:31:57.660 --> 00:31:59.860 That it's not like I could have -- 00:31:59.860 --> 00:32:03.030 I couldn't have a line and a line in three dimensions. 00:32:03.030 --> 00:32:07.130 Those don't add up one and one don't add to three. 00:32:07.130 --> 00:32:17.820 So I used the word orthogonal complements in R^n. 00:32:17.820 --> 00:32:20.000 And the idea of this word complement 00:32:20.000 --> 00:32:28.720 is that the orthogonal complement of a row space 00:32:28.720 --> 00:32:32.950 contains not just some vectors that are orthogonal to it, 00:32:32.950 --> 00:32:34.400 but all. 00:32:34.400 --> 00:32:36.050 So what does that mean? 00:32:36.050 --> 00:32:42.260 That means that the null space contains all, not just 00:32:42.260 --> 00:32:51.270 some but all, vectors that are perpendicular to the row space. 00:32:51.270 --> 00:32:52.260 OK. 00:32:52.260 --> 00:33:04.190 Really what I've done in this half of the lecture is just 00:33:04.190 --> 00:33:09.100 notice some of the nice geometry that -- 00:33:09.100 --> 00:33:12.030 that we didn't pick up before because we didn't discuss 00:33:12.030 --> 00:33:15.010 perpendicular vectors before. 00:33:15.010 --> 00:33:16.800 But it was all sitting there. 00:33:16.800 --> 00:33:18.440 And now we picked it up. 00:33:18.440 --> 00:33:21.370 That these vectors are orthogonal complements. 00:33:21.370 --> 00:33:23.740 And I guess I even call this part 00:33:23.740 --> 00:33:26.730 one of the fundamental theorem of linear algebra. 00:33:26.730 --> 00:33:32.330 The fundamental theorem of linear algebra 00:33:32.330 --> 00:33:37.230 is about these four subspaces, so part one 00:33:37.230 --> 00:33:41.660 is about their dimension, maybe I should call it part two now. 00:33:41.660 --> 00:33:44.600 Their dimensions we got. 00:33:44.600 --> 00:33:49.010 Now we're getting their orthogonality, that's part two. 00:33:49.010 --> 00:33:54.640 And part three will be about bases for them. 00:33:54.640 --> 00:33:57.290 Orthogonal bases. 00:33:57.290 --> 00:34:00.520 So that's coming up. 00:34:00.520 --> 00:34:01.570 OK. 00:34:01.570 --> 00:34:10.870 So I'm happy with that geometry right now. 00:34:10.870 --> 00:34:11.980 OK. 00:34:11.980 --> 00:34:12.949 OK. 00:34:12.949 --> 00:34:16.239 Now what's my next goal in this chapter? 00:34:16.239 --> 00:34:19.087 Here's the main problem of the chapter. 00:34:19.087 --> 00:34:21.420 The main problem of the chapter is -- so this is coming. 00:34:21.420 --> 00:34:22.378 It's coming attraction. 00:34:22.378 --> 00:34:37.160 This is the very last chapter that's about Ax=b. 00:34:44.030 --> 00:34:48.690 I would like to solve that system of equations 00:34:48.690 --> 00:34:50.710 when there is no solution. 00:34:54.560 --> 00:34:56.880 You may say what a ridiculous thing to do. 00:34:56.880 --> 00:35:00.670 But I have to say it's done all the time. 00:35:00.670 --> 00:35:02.440 In fact it has to be done. 00:35:02.440 --> 00:35:07.510 You get -- so the problem is solve -- 00:35:07.510 --> 00:35:20.626 the best possible solve I'll put quote Ax=b when there is no 00:35:20.626 --> 00:35:21.125 solution. 00:35:24.470 --> 00:35:25.930 And of course what does that mean? 00:35:25.930 --> 00:35:29.550 b isn't in the column space. 00:35:29.550 --> 00:35:35.720 And it's quite typical if this matrix A is rectangular if I -- 00:35:35.720 --> 00:35:40.370 maybe I have m equations and that's bigger than the number 00:35:40.370 --> 00:35:47.840 of unknowns, then for sure the rank is not m, 00:35:47.840 --> 00:35:51.910 the rank couldn't be m now, so there'll be a lot of right-hand 00:35:51.910 --> 00:36:00.300 sides with no solution, but here's an example. 00:36:00.300 --> 00:36:02.760 Some satellite is buzzing along. 00:36:02.760 --> 00:36:06.196 You measure its position. 00:36:06.196 --> 00:36:07.570 You make a thousand measurements. 00:36:10.190 --> 00:36:13.940 So that gives you a thousand equations for the -- 00:36:13.940 --> 00:36:18.340 for the parameters that -- that give the position. 00:36:18.340 --> 00:36:20.350 But there aren't a thousand parameters, 00:36:20.350 --> 00:36:23.310 there's just maybe six or something. 00:36:23.310 --> 00:36:26.930 Or you're measuring the -- you're doing questionnaires. 00:36:29.690 --> 00:36:36.210 You're measuring resistances. 00:36:36.210 --> 00:36:37.650 You're taking pulses. 00:36:37.650 --> 00:36:41.120 You're measuring somebody's pulse rate. 00:36:41.120 --> 00:36:42.190 There's just one unknown. 00:36:42.190 --> 00:36:42.690 OK. 00:36:42.690 --> 00:36:44.890 The pulse rate. 00:36:44.890 --> 00:36:46.950 So you measure it once, OK, fine, 00:36:46.950 --> 00:36:49.510 but if you really want to know it, 00:36:49.510 --> 00:36:54.800 you measure it multiple times, but then the measurements have 00:36:54.800 --> 00:36:57.120 noise in them, so there's -- 00:36:57.120 --> 00:36:59.820 the problem is that in many many problems 00:36:59.820 --> 00:37:03.740 we've got too many equations and they've got 00:37:03.740 --> 00:37:05.240 noise in the right-hand side. 00:37:05.240 --> 00:37:11.780 So Ax=b I can't expect to solve it exactly right, 00:37:11.780 --> 00:37:13.230 because I don't even know what -- 00:37:13.230 --> 00:37:18.680 there's a measurement mistake in b. 00:37:18.680 --> 00:37:21.790 But there's information too. 00:37:21.790 --> 00:37:25.120 There's a lot of information about x in there. 00:37:25.120 --> 00:37:30.070 And what I want to do is like separate the noise, the junk, 00:37:30.070 --> 00:37:33.860 from the information. 00:37:33.860 --> 00:37:38.680 And so this is a straightforward linear algebra problem. 00:37:38.680 --> 00:37:41.600 How do I solve, what's the best solution? 00:37:41.600 --> 00:37:43.600 OK. 00:37:43.600 --> 00:37:45.070 Now. 00:37:45.070 --> 00:37:52.190 I want to say so that's like describes the problem 00:37:52.190 --> 00:37:55.010 in an algebraic way. 00:37:55.010 --> 00:37:58.820 I got some equations, I'm looking for the best solution. 00:37:58.820 --> 00:38:02.020 Well, one way to find it is -- one way to start, 00:38:02.020 --> 00:38:09.580 one way to find a solution is throw away equations until 00:38:09.580 --> 00:38:12.340 you've got a nice, square invertible system and solve 00:38:12.340 --> 00:38:14.760 that. 00:38:14.760 --> 00:38:18.620 That's not satisfactory. 00:38:18.620 --> 00:38:21.410 There's no reason in these measurements 00:38:21.410 --> 00:38:23.300 to say these measurements are perfect 00:38:23.300 --> 00:38:25.970 and these measurements are useless. 00:38:25.970 --> 00:38:27.990 We want to use all the measurements 00:38:27.990 --> 00:38:32.140 to get the best information, to get the maximum information. 00:38:32.140 --> 00:38:33.170 But how? 00:38:33.170 --> 00:38:34.390 OK. 00:38:34.390 --> 00:38:40.020 Let me anticipate a matrix that's going to show up. 00:38:40.020 --> 00:38:43.720 This A is typically rectangular. 00:38:43.720 --> 00:38:47.390 But a matrix that shows up whenever you have -- 00:38:47.390 --> 00:38:52.260 and we chapter three was all about rectangular matrices. 00:38:52.260 --> 00:38:54.840 And we know when this is solvable, 00:38:54.840 --> 00:38:58.020 you could do elimination on it, right? 00:38:58.020 --> 00:39:00.600 But I'm thinking hey, you do elimination 00:39:00.600 --> 00:39:03.299 and you get equation zero equal other non-zeroes. 00:39:03.299 --> 00:39:05.590 I'm thinking we really -- elimination is going to fail. 00:39:05.590 --> 00:39:05.630 So that's our question. 00:39:05.630 --> 00:39:07.046 Elimination will get us down to -- 00:39:16.320 --> 00:39:18.700 will tell us if there is a solution or not. 00:39:18.700 --> 00:39:22.600 But I'm now thinking not. 00:39:22.600 --> 00:39:24.130 So what are we going to do? 00:39:24.130 --> 00:39:24.720 OK. 00:39:24.720 --> 00:39:25.320 All right. 00:39:25.320 --> 00:39:30.810 I want to tell you to jump ahead to the matrix that 00:39:30.810 --> 00:39:32.310 will play a key role. 00:39:32.310 --> 00:39:35.780 So this is the matrix that you want to understand 00:39:35.780 --> 00:39:38.950 for this chapter four. 00:39:38.950 --> 00:39:47.740 And it's the matrix A transpose A. 00:39:47.740 --> 00:39:53.130 What's -- tell me some things about that matrix. 00:39:53.130 --> 00:39:58.640 So A is this m by n matrix, rectangular, but now 00:39:58.640 --> 00:40:03.100 I'm saying that the good matrix that shows up in the end 00:40:03.100 --> 00:40:05.130 is A transpose A. 00:40:05.130 --> 00:40:09.120 So tell me something about that. 00:40:09.120 --> 00:40:14.010 Is it -- what's the first thing you know about A transpose A. 00:40:14.010 --> 00:40:15.390 It's square. 00:40:15.390 --> 00:40:16.780 Right? 00:40:16.780 --> 00:40:21.740 Square because this is m by n and this is n by m. 00:40:21.740 --> 00:40:24.090 So this is the result is n by n. 00:40:24.090 --> 00:40:25.240 Good. 00:40:25.240 --> 00:40:26.230 Square. 00:40:26.230 --> 00:40:27.990 What else? 00:40:27.990 --> 00:40:28.970 It's symmetric. 00:40:28.970 --> 00:40:29.535 Good. 00:40:29.535 --> 00:40:30.160 It's symmetric. 00:40:35.920 --> 00:40:39.650 Because you remember how to do that. 00:40:39.650 --> 00:40:44.270 If we transpose that matrix let's transpose it, 00:40:44.270 --> 00:40:48.510 A transpose A, if I transpose it, 00:40:48.510 --> 00:40:55.230 then that comes first transposed, this comes second, 00:40:55.230 --> 00:41:01.990 transposed, and then transposing twice is leaves it -- 00:41:01.990 --> 00:41:05.100 brings it back to the same so it's symmetric. 00:41:05.100 --> 00:41:06.620 Good. 00:41:06.620 --> 00:41:11.340 Now we now know how to ask more about a matrix. 00:41:14.050 --> 00:41:20.110 I'm interested in is it invertible? 00:41:20.110 --> 00:41:23.810 If not, what's its null space? 00:41:23.810 --> 00:41:26.940 So I want to know about -- because you're going to see, 00:41:26.940 --> 00:41:32.260 well, let me -- let me even, well I shouldn't do this, 00:41:32.260 --> 00:41:33.350 but I will. 00:41:33.350 --> 00:41:37.460 Let me tell you what equation to solve 00:41:37.460 --> 00:41:42.300 when you can't solve that one. 00:41:42.300 --> 00:41:47.450 The good equation comes from multiplying both sides 00:41:47.450 --> 00:41:53.150 by A transpose, so the good equation that you get to 00:41:53.150 --> 00:41:54.820 is this one. 00:41:54.820 --> 00:42:00.790 A transpose Ax equals A transpose b. 00:42:04.960 --> 00:42:08.700 That will be the central equation in the chapter. 00:42:08.700 --> 00:42:10.950 So I think why not tell it to you. 00:42:10.950 --> 00:42:13.290 Why not admit it right away. 00:42:13.290 --> 00:42:14.440 OK. 00:42:14.440 --> 00:42:15.260 I have to -- 00:42:15.260 --> 00:42:17.960 I should really give x. 00:42:17.960 --> 00:42:27.750 I want to sort of indicate that this x isn't I mean this x was 00:42:27.750 --> 00:42:30.580 the solution to that equation if it existed, 00:42:30.580 --> 00:42:33.440 but probably didn't. 00:42:33.440 --> 00:42:39.020 Now let me give this a different name, x hat. 00:42:39.020 --> 00:42:45.740 Because I'm hoping this one will have a solution. 00:42:45.740 --> 00:42:49.680 And I'm saying that it's my best solution. 00:42:49.680 --> 00:42:52.400 I'll have to say what does best mean. 00:42:52.400 --> 00:42:55.720 But that's going to be my -- my plan. 00:42:55.720 --> 00:42:59.650 I'm going to say that the best solution solves this equation. 00:42:59.650 --> 00:43:03.770 So you see right away why I'm so interested in this matrix 00:43:03.770 --> 00:43:05.620 A transpose A. 00:43:05.620 --> 00:43:07.130 And in its invertibility. 00:43:07.130 --> 00:43:07.630 OK. 00:43:10.210 --> 00:43:11.720 Now, when is it invertible? 00:43:14.290 --> 00:43:14.900 OK. 00:43:14.900 --> 00:43:22.040 Let me take a case, let me just do an example and then -- 00:43:22.040 --> 00:43:25.910 I'll just pick a matrix here. 00:43:25.910 --> 00:43:28.510 Just so we see what A transpose A looks like. 00:43:28.510 --> 00:43:35.010 So let me take a matrix A one, one, one, one, two, five. 00:43:35.010 --> 00:43:37.190 Just to invent a matrix. 00:43:37.190 --> 00:43:40.600 So there's a matrix A. 00:43:40.600 --> 00:43:48.390 Notice that it has M equal three rows and N equal two columns. 00:43:48.390 --> 00:43:50.950 Its rank is -- 00:43:50.950 --> 00:43:55.200 the rank of that matrix is two. 00:43:55.200 --> 00:43:58.460 Right, yeah, the columns are independent. 00:43:58.460 --> 00:44:01.200 Does Ax equal b? 00:44:01.200 --> 00:44:09.086 If I look at Ax=b, so x is just x1 x2, and b is b1 b2 b3. 00:44:12.700 --> 00:44:15.300 Do I expect to solve Ax=b? 00:44:15.300 --> 00:44:17.760 What's -- no way, right? 00:44:17.760 --> 00:44:21.440 I mean linear algebra's great, but solving 00:44:21.440 --> 00:44:24.040 three equations with only two unknowns usually 00:44:24.040 --> 00:44:26.020 we can't do it. 00:44:26.020 --> 00:44:30.050 We can only solve it if this vector is b is what? 00:44:33.230 --> 00:44:37.780 I can solve that equation if that vector b1 b2 b3 00:44:37.780 --> 00:44:41.700 is in the column space. 00:44:41.700 --> 00:44:44.650 If it's a combination of those columns then fine. 00:44:44.650 --> 00:44:47.030 But usually it won't be. 00:44:47.030 --> 00:44:49.580 The combinations just fill up a plane 00:44:49.580 --> 00:44:52.780 and most vectors aren't on that plane. 00:44:52.780 --> 00:44:56.630 So what I'm saying is that I'm going to work 00:44:56.630 --> 00:44:59.990 with the matrix A transpose A. 00:44:59.990 --> 00:45:03.600 And I just want to figure out in this example what 00:45:03.600 --> 00:45:07.780 A transpose A is. 00:45:07.780 --> 00:45:09.390 So it's two by two. 00:45:09.390 --> 00:45:13.320 The first entry is a three, the next entry is an eight, 00:45:13.320 --> 00:45:15.160 this entry is -- 00:45:18.040 --> 00:45:20.910 what's that entry? 00:45:20.910 --> 00:45:23.110 Eight, for sure. 00:45:23.110 --> 00:45:25.270 We knew it had to be, and this entry 00:45:25.270 --> 00:45:32.630 is, what's that now, getting out my trusty calculator, thirty, 00:45:32.630 --> 00:45:35.880 is that right? 00:45:35.880 --> 00:45:37.580 And is that matrix invertible? 00:45:37.580 --> 00:45:38.080 Thirty. 00:45:38.080 --> 00:45:40.960 There's an A transpose A. 00:45:40.960 --> 00:45:42.620 And it is invertible, right? 00:45:42.620 --> 00:45:46.040 Three, eight is not a multiple of eight, thirty -- 00:45:46.040 --> 00:45:48.050 and it's invertible. 00:45:48.050 --> 00:45:51.420 And that's the normal, that's what I expect. 00:45:51.420 --> 00:45:56.730 So this is I want to show. 00:45:56.730 --> 00:45:59.740 So here's the final -- here's the key point. 00:45:59.740 --> 00:46:04.660 The null space of A transpose A -- 00:46:04.660 --> 00:46:06.645 it's not going to be always invertible. 00:46:09.650 --> 00:46:11.450 Tell me a matrix -- 00:46:11.450 --> 00:46:15.640 I have to say that I can't say A transpose A is always 00:46:15.640 --> 00:46:16.770 invertible. 00:46:16.770 --> 00:46:19.590 Because that's asking too much. 00:46:19.590 --> 00:46:22.860 I mean what could the matrix A be, for example, 00:46:22.860 --> 00:46:27.330 so that A transpose A was not invertible? 00:46:27.330 --> 00:46:29.110 Well, it even could be the zero matrix. 00:46:29.110 --> 00:46:32.250 I mean that's like extreme case. 00:46:32.250 --> 00:46:38.960 Suppose I make this rank -- 00:46:38.960 --> 00:46:46.090 suppose I change to that A. 00:46:46.090 --> 00:46:49.240 Now I figure out A transpose A again and I get -- 00:46:49.240 --> 00:46:49.840 what do I get? 00:46:53.890 --> 00:46:58.360 I get nine, I get nine of course and here I 00:46:58.360 --> 00:47:02.434 get what's that entry? 00:47:02.434 --> 00:47:02.975 Twenty-seven. 00:47:06.900 --> 00:47:09.100 And is that matrix invertible? 00:47:09.100 --> 00:47:09.600 No. 00:47:12.110 --> 00:47:13.470 And why do I -- 00:47:13.470 --> 00:47:16.590 I knew it wouldn't be invertible anyway. 00:47:16.590 --> 00:47:23.190 Because this matrix only has rank one. 00:47:23.190 --> 00:47:26.100 And if I have a product of matrices of rank one, 00:47:26.100 --> 00:47:30.340 the product is not going to have a rank bigger than one. 00:47:30.340 --> 00:47:33.810 So I'm not surprised that the answer only has rank one. 00:47:33.810 --> 00:47:36.730 And that's what I -- 00:47:36.730 --> 00:47:41.200 always happens, that the rank of A transpose A 00:47:41.200 --> 00:47:44.460 comes out to equal the rank of A. 00:47:44.460 --> 00:47:49.580 So, yes, so the null space of A transpose A 00:47:49.580 --> 00:47:57.580 equals the null space of A, the rank of A transpose A 00:47:57.580 --> 00:48:01.840 equals the rank of A. 00:48:01.840 --> 00:48:11.460 So let's -- as soon as I can why that's true. 00:48:11.460 --> 00:48:18.920 But let's draw from that what the fact that I want. 00:48:18.920 --> 00:48:24.810 This tells me that this square symmetric matrix is invertible 00:48:24.810 --> 00:48:26.650 if -- 00:48:26.650 --> 00:48:30.460 so here's my conclusion. 00:48:30.460 --> 00:48:40.400 A transpose A is invertible if exactly when -- 00:48:40.400 --> 00:48:46.390 exactly if this null space is only got the zero vector. 00:48:46.390 --> 00:48:51.700 Which means the columns of A are independent. 00:48:51.700 --> 00:48:54.342 So A transpose A is invertible exactly 00:48:54.342 --> 00:48:55.550 if A has independent columns. 00:48:55.550 --> 00:49:12.290 That's the fact that I need about A transpose A. 00:49:12.290 --> 00:49:17.160 And then you'll see next time how A transpose A 00:49:17.160 --> 00:49:18.780 enters everything. 00:49:18.780 --> 00:49:22.130 Next lecture is actually a crucial one. 00:49:22.130 --> 00:49:26.280 Here I'm preparing for it by getting us 00:49:26.280 --> 00:49:29.000 thinking about A transpose A. 00:49:29.000 --> 00:49:31.970 And its rank is the same as the rank of A, 00:49:31.970 --> 00:49:34.310 and we can decide when it's invertible. 00:49:34.310 --> 00:49:34.980 OK. 00:49:34.980 --> 00:49:35.938 So I'll see you Friday. 00:49:35.938 --> 00:49:37.140 Thanks.