WEBVTT 00:00:10.240 --> 00:00:17.470 OK, guys the -- we're almost ready to make this lecture 00:00:17.470 --> 00:00:19.310 immortal. 00:00:19.310 --> 00:00:21.340 OK. 00:00:21.340 --> 00:00:22.720 Are we on? 00:00:22.720 --> 00:00:23.490 All right. 00:00:23.490 --> 00:00:26.820 This is an important lecture. 00:00:26.820 --> 00:00:30.170 It's about projections. 00:00:30.170 --> 00:00:36.430 Let me start by just projecting a vector b down on a vector a. 00:00:36.430 --> 00:00:41.400 So just to so you see what the geometry looks like in when 00:00:41.400 --> 00:00:44.060 I'm in -- in just two dimensions, 00:00:44.060 --> 00:00:51.670 I'd like to find the point along this line so that line through 00:00:51.670 --> 00:00:55.080 a is a one-dimensional subspace, so I'm starting with one 00:00:55.080 --> 00:00:56.270 dimension. 00:00:56.270 --> 00:01:00.940 I'd like to find the point on that line closest to a. 00:01:00.940 --> 00:01:03.390 Can I just take that problem first and then I'll 00:01:03.390 --> 00:01:06.600 explain why I want to do it and why I want 00:01:06.600 --> 00:01:09.620 to project on other subspaces. 00:01:09.620 --> 00:01:14.780 So where's the point closest to b that's on that line? 00:01:14.780 --> 00:01:17.480 It's somewhere there. 00:01:17.480 --> 00:01:24.780 And let me connect that and -- and what's the whole point 00:01:24.780 --> 00:01:25.800 of my picture now? 00:01:25.800 --> 00:01:29.440 What's the -- where does orthogonality come into this 00:01:29.440 --> 00:01:30.350 picture? 00:01:30.350 --> 00:01:33.860 The whole point is that this best point, that's 00:01:33.860 --> 00:01:44.200 the projection, P, of b onto the line, where's orthogonality? 00:01:44.200 --> 00:01:46.820 It's the fact that that's a right angle. 00:01:46.820 --> 00:01:50.990 That this -- the error -- this is like how much I'm wrong 00:01:50.990 --> 00:01:51.940 by -- 00:01:51.940 --> 00:01:57.730 this is the difference between b and P, the whole point is -- 00:01:57.730 --> 00:02:01.090 that's perpendicular to a. 00:02:01.090 --> 00:02:03.660 That's got to give us the equation. 00:02:03.660 --> 00:02:07.370 That's got to tell us -- that's the one fact we know, 00:02:07.370 --> 00:02:11.090 that's got to tell us where that projection is. 00:02:11.090 --> 00:02:14.200 Let me also say, look -- 00:02:14.200 --> 00:02:16.970 I've drawn a triangle there. 00:02:16.970 --> 00:02:19.220 So if we were doing trigonometry we 00:02:19.220 --> 00:02:25.680 would do like we would have angles theta and distances that 00:02:25.680 --> 00:02:29.330 would involve sine theta and cos theta that 00:02:29.330 --> 00:02:33.120 leads to lousy formulas compared to linear algebra. 00:02:33.120 --> 00:02:40.750 The formula that we want comes out nicely and what's the -- 00:02:40.750 --> 00:02:42.260 what do we know? 00:02:42.260 --> 00:02:49.040 We know that P, this projection, is some multiple of a, right? 00:02:49.040 --> 00:02:50.690 It's on that line. 00:02:50.690 --> 00:02:54.550 So we know it's in that one-dimensional subspace, it's 00:02:54.550 --> 00:02:57.870 some multiple, let me call that multiple x, 00:02:57.870 --> 00:02:59.060 of a. 00:02:59.060 --> 00:03:03.530 So really it's that number x I'd like to find. 00:03:03.530 --> 00:03:07.500 So this is going to be simple in 1-D, 00:03:07.500 --> 00:03:11.130 so let's just carry it through, and then see 00:03:11.130 --> 00:03:13.890 how it goes in high dimensions. 00:03:13.890 --> 00:03:14.510 OK. 00:03:14.510 --> 00:03:17.120 The key fact is -- 00:03:17.120 --> 00:03:21.000 the key to everything is that perpendicular. 00:03:21.000 --> 00:03:31.150 The fact that a is perpendicular to a is perpendicular to e. 00:03:31.150 --> 00:03:34.470 Which is (b-ax), xa. 00:03:34.470 --> 00:03:36.510 I don't care what -- 00:03:36.510 --> 00:03:37.010 xa. 00:03:37.010 --> 00:03:41.950 That that equals zero. 00:03:41.950 --> 00:03:44.010 Do you see that as the central equation, 00:03:44.010 --> 00:03:50.100 that's saying that this a is perpendicular to this -- 00:03:50.100 --> 00:03:53.700 correction, that's going to tell us what x is. 00:03:53.700 --> 00:03:57.890 Let me just raise the board and simplify that and out will 00:03:57.890 --> 00:03:59.790 come x. 00:03:59.790 --> 00:04:00.290 OK. 00:04:00.290 --> 00:04:06.540 So if I simplify that, let's see, I'll move one to -- 00:04:06.540 --> 00:04:09.580 one term to one side, the other term will be on the other side, 00:04:09.580 --> 00:04:14.690 it looks to me like x times a transpose a 00:04:14.690 --> 00:04:18.560 is equal to a transpose b. 00:04:21.370 --> 00:04:22.210 Right? 00:04:22.210 --> 00:04:25.240 I have a transpose b as one f- one term, 00:04:25.240 --> 00:04:27.510 a transpose a as the other, so right 00:04:27.510 --> 00:04:30.590 away here's my a transpose a. 00:04:30.590 --> 00:04:32.900 But it's just a number now. 00:04:32.900 --> 00:04:35.400 And I divide by it. 00:04:35.400 --> 00:04:42.200 And I get the answer. x is a transpose b over a transpose a. 00:04:45.220 --> 00:04:48.320 And P, the projection I wanted, is -- 00:04:48.320 --> 00:04:50.530 that's the right multiple. 00:04:50.530 --> 00:04:52.720 That's got a cosine theta built in. 00:04:52.720 --> 00:04:55.560 But we don't need to look at angles. 00:04:55.560 --> 00:04:58.040 It's -- we've just got vectors here. 00:04:58.040 --> 00:05:02.140 And the projection is P is a times that x. 00:05:07.450 --> 00:05:08.349 Or x times that a. 00:05:08.349 --> 00:05:10.640 But I'm really going to -- eventually I'm going to want 00:05:10.640 --> 00:05:13.380 that x coming on the right-hand side. 00:05:13.380 --> 00:05:20.630 So do you see that I've got two of the three formulas already, 00:05:20.630 --> 00:05:24.750 right here, I've got the -- 00:05:24.750 --> 00:05:29.140 that's the equation -- that leads me to the answer, 00:05:29.140 --> 00:05:34.230 here's the answer for x, and here's the projection. 00:05:34.230 --> 00:05:34.870 OK. 00:05:34.870 --> 00:05:37.520 can I do add just one more thing to 00:05:37.520 --> 00:05:39.880 this one-dimensional problem? 00:05:39.880 --> 00:05:45.150 One more like lift it up into linear algebra, into matrices. 00:05:48.460 --> 00:05:50.260 Here's the last thing I want to do -- 00:05:50.260 --> 00:05:52.120 but don't forget those formulas. 00:05:52.120 --> 00:05:54.760 a transpose b over a transpose 00:05:54.760 --> 00:05:55.680 a. 00:05:55.680 --> 00:06:06.020 Actually let's look at that for a moment first. 00:06:06.020 --> 00:06:09.390 Suppose -- Let me take this next step. 00:06:09.390 --> 00:06:11.490 So P is a times x. 00:06:11.490 --> 00:06:13.170 So can I write that then? 00:06:13.170 --> 00:06:20.800 P is a times this neat number, a transpose b over a transpose a. 00:06:24.130 --> 00:06:26.690 That's our projection. 00:06:26.690 --> 00:06:30.060 Can I ask a couple of questions about it, just while we look, 00:06:30.060 --> 00:06:32.510 get that digest that formula. 00:06:32.510 --> 00:06:37.020 Suppose b is doubled. 00:06:37.020 --> 00:06:41.600 Suppose I change b to two b. 00:06:41.600 --> 00:06:42.935 What happens to the projection? 00:06:46.200 --> 00:06:48.950 So suppose I instead of that vector b that I drew 00:06:48.950 --> 00:06:51.920 on the board make it two b, twice as long -- 00:06:51.920 --> 00:06:55.540 what's the projection now? 00:06:55.540 --> 00:06:57.550 It's doubled too, right? 00:06:57.550 --> 00:07:01.060 It's going to be twice as far, if b goes twice as far, 00:07:01.060 --> 00:07:03.030 the projection will go twice as far. 00:07:03.030 --> 00:07:04.160 And you see it there. 00:07:04.160 --> 00:07:09.180 If I put in an extra factor two, then P's got that factor 00:07:09.180 --> 00:07:09.840 too. 00:07:09.840 --> 00:07:14.040 Now what about if I double a? 00:07:14.040 --> 00:07:19.850 What if I double the vector a that I'm projecting onto? 00:07:19.850 --> 00:07:23.220 What changes? 00:07:23.220 --> 00:07:25.320 The projection doesn't change at all. 00:07:25.320 --> 00:07:25.820 Right? 00:07:25.820 --> 00:07:28.890 Because I'm just -- the line didn't change. 00:07:28.890 --> 00:07:31.620 If I double a or I take minus a. 00:07:31.620 --> 00:07:33.430 It's still that same line. 00:07:33.430 --> 00:07:35.310 The projection's still in the same place. 00:07:35.310 --> 00:07:40.540 And of course if I double a I get a four up above, 00:07:40.540 --> 00:07:43.600 and I get a four -- an extra four below, they cancel out, 00:07:43.600 --> 00:07:45.580 and the projection is the same. 00:07:45.580 --> 00:07:46.350 OK. 00:07:46.350 --> 00:07:48.970 So really, this -- 00:07:48.970 --> 00:07:54.750 I want to look at this as the projection -- 00:07:54.750 --> 00:07:57.630 there's a matrix here. 00:07:57.630 --> 00:08:01.960 The projection is carried out by some matrix 00:08:01.960 --> 00:08:05.090 that I'm going to call the projection matrix. 00:08:05.090 --> 00:08:07.970 And in other words the projection 00:08:07.970 --> 00:08:12.240 is some matrix that acts on this guy b 00:08:12.240 --> 00:08:14.100 and produces the projection. 00:08:14.100 --> 00:08:21.270 The projection P is the projection matrix acting 00:08:21.270 --> 00:08:24.230 on whatever the input is. 00:08:24.230 --> 00:08:28.130 The input is b, the projection matrix is P. OK. 00:08:28.130 --> 00:08:30.850 Actually you can tell me right away what 00:08:30.850 --> 00:08:31.970 this projection matrix 00:08:31.970 --> 00:08:32.470 is. 00:08:32.470 --> 00:08:37.530 So this is a pretty interesting matrix. 00:08:37.530 --> 00:08:40.650 What matrix is multiplying b? 00:08:40.650 --> 00:08:42.690 I'm just -- just from my formula -- 00:08:42.690 --> 00:08:44.120 I see what P is. 00:08:44.120 --> 00:08:51.650 P, this projection matrix, is -- 00:08:51.650 --> 00:08:52.360 what is it? 00:08:52.360 --> 00:08:59.515 I see a a transpose above, and I see a transpose a below. 00:09:02.470 --> 00:09:04.560 And those don't cancel. 00:09:04.560 --> 00:09:06.620 That's not one. 00:09:06.620 --> 00:09:07.260 Right? 00:09:07.260 --> 00:09:08.230 That's a matrix. 00:09:08.230 --> 00:09:10.580 Because down here, the a transpose a, 00:09:10.580 --> 00:09:13.690 that's just a number, a transpose a, that's 00:09:13.690 --> 00:09:18.060 the length of a squared, and up above is a column times 00:09:18.060 --> 00:09:19.040 a row. 00:09:19.040 --> 00:09:22.050 Column times a row is a matrix. 00:09:22.050 --> 00:09:26.450 So this is a full-scale n by n matrix, if I -- 00:09:26.450 --> 00:09:28.880 if I'm in n dimensions. 00:09:28.880 --> 00:09:32.270 And it's kind of an interesting one. 00:09:32.270 --> 00:09:36.510 And it's the one which if I multiply by b then I get this, 00:09:36.510 --> 00:09:39.750 you see once again I'm putting parentheses 00:09:39.750 --> 00:09:41.140 in different places. 00:09:41.140 --> 00:09:46.480 I'm putting the parentheses right there. 00:09:46.480 --> 00:09:49.280 I'm saying OK, that's really the matrix 00:09:49.280 --> 00:09:55.990 that produces this projection. 00:09:55.990 --> 00:09:56.980 OK. 00:09:56.980 --> 00:10:00.280 Now, tell me -- 00:10:00.280 --> 00:10:02.910 all right, what are the properties of that matrix? 00:10:05.420 --> 00:10:08.900 I'm just using letters here, a and b, I could put in numbers, 00:10:08.900 --> 00:10:12.100 but I think it's -- for once, it's clearer with letters, 00:10:12.100 --> 00:10:16.790 because all formulas are simple, a transpose b over a transpose 00:10:16.790 --> 00:10:21.770 a -- that's the number that multiplies the a, and then I 00:10:21.770 --> 00:10:26.160 see wait a minute, there's a matrix here and what's the rank 00:10:26.160 --> 00:10:29.260 of that matrix, by the way? 00:10:29.260 --> 00:10:31.360 What's the rank of that matrix, yeah -- 00:10:31.360 --> 00:10:33.260 let me just ask you about that matrix. 00:10:33.260 --> 00:10:38.250 Which looks a little strange, a a transpose over this number. 00:10:38.250 --> 00:10:46.180 But well, I could ask you its column space. 00:10:46.180 --> 00:10:48.920 Yeah, let me ask you its column space. 00:10:48.920 --> 00:10:52.180 So what's the column space of a matrix? 00:10:52.180 --> 00:10:56.460 If you multiply that matrix by anything 00:10:56.460 --> 00:10:59.810 you always get in the column space, right? 00:10:59.810 --> 00:11:04.900 The column space of a matrix is when you multiply any vector 00:11:04.900 --> 00:11:09.900 by that matrix -- any vector b, by the matrix, 00:11:09.900 --> 00:11:13.360 you always land in the column space. 00:11:13.360 --> 00:11:16.320 That's what column spaces work. 00:11:16.320 --> 00:11:20.060 Now what space do we always land in? 00:11:20.060 --> 00:11:22.110 What's the column space of -- 00:11:22.110 --> 00:11:26.160 what's the result when I multiply this any vector 00:11:26.160 --> 00:11:29.550 b by my matrix? 00:11:29.550 --> 00:11:34.710 So I have P times b, where I? 00:11:34.710 --> 00:11:36.170 I'm on that line, right? 00:11:36.170 --> 00:11:40.610 The column space, so here are facts about this matrix. 00:11:40.610 --> 00:11:46.220 The column space of P, of this projection matrix, 00:11:46.220 --> 00:11:49.190 is the line through a. 00:11:52.620 --> 00:12:03.720 And the rank of this matrix is you can all say it at once one. 00:12:03.720 --> 00:12:04.520 Right. 00:12:04.520 --> 00:12:06.590 The rank is one. 00:12:06.590 --> 00:12:08.300 This is a rank one matrix. 00:12:08.300 --> 00:12:10.960 Actually it's exactly the form that we're 00:12:10.960 --> 00:12:15.130 familiar with a rank one matrix. 00:12:15.130 --> 00:12:20.840 A column times a row, that's a rank one matrix, that column 00:12:20.840 --> 00:12:27.400 is the basis for the column space. 00:12:27.400 --> 00:12:28.770 Just one dimension. 00:12:28.770 --> 00:12:29.550 OK. 00:12:29.550 --> 00:12:32.560 So I know that much about the matrix. 00:12:32.560 --> 00:12:35.490 But now there are two more facts about the matrix 00:12:35.490 --> 00:12:40.230 that I want to notice. 00:12:40.230 --> 00:12:42.980 First of all is the matrix symmetric? 00:12:42.980 --> 00:12:45.580 That's a natural question for matrices. 00:12:45.580 --> 00:12:48.380 And the answer is yes. 00:12:48.380 --> 00:12:52.620 If I take the transpose of this -- there's a number down there, 00:12:52.620 --> 00:12:57.300 the transpose of a a transpose is a a transpose. 00:12:57.300 --> 00:12:59.200 So P is symmetric. 00:12:59.200 --> 00:13:04.500 P transpose equals P. So this is a key property. 00:13:04.500 --> 00:13:07.730 That the projection matrix is symmetric. 00:13:07.730 --> 00:13:12.430 One more property now and this is the real one. 00:13:12.430 --> 00:13:18.570 What happens if I do the projection twice? 00:13:18.570 --> 00:13:21.740 So I'm looking for something, some information 00:13:21.740 --> 00:13:24.220 about P squared. 00:13:24.220 --> 00:13:27.530 But just give me in terms of that picture, in terms 00:13:27.530 --> 00:13:33.940 my picture, I take any vector b, I multiply it 00:13:33.940 --> 00:13:36.280 by my projection matrix, and that 00:13:36.280 --> 00:13:41.010 puts me there, so this is Pb. 00:13:41.010 --> 00:13:45.170 And now I project again. 00:13:45.170 --> 00:13:46.790 What happens now? 00:13:46.790 --> 00:13:49.620 What happens when I apply the projection matrix 00:13:49.620 --> 00:13:51.880 a second time? 00:13:51.880 --> 00:13:56.160 To this, so I'm applying it once brings me here 00:13:56.160 --> 00:14:01.420 and the second time brings me I stay put. 00:14:01.420 --> 00:14:02.170 Right? 00:14:02.170 --> 00:14:05.490 The projection for a point on this line the projection 00:14:05.490 --> 00:14:08.530 is right where it is. 00:14:08.530 --> 00:14:10.610 The projection is the same point. 00:14:10.610 --> 00:14:16.340 So that means that if I project twice, 00:14:16.340 --> 00:14:21.430 I get the same answer as I did in the first projection. 00:14:21.430 --> 00:14:26.510 So those are the two properties that 00:14:26.510 --> 00:14:30.630 tell me I'm looking at a projection matrix. 00:14:30.630 --> 00:14:35.070 It's symmetric and it's square is itself. 00:14:35.070 --> 00:14:37.500 Because if I project a second time, 00:14:37.500 --> 00:14:40.740 it's the same result as the first result. 00:14:40.740 --> 00:14:41.940 OK. 00:14:41.940 --> 00:14:49.120 So that's -- and then here's the exact formula when I know what 00:14:49.120 --> 00:14:53.450 I'm projecting onto, that line through a, then I know what P 00:14:53.450 --> 00:14:54.460 is. 00:14:54.460 --> 00:14:57.070 So do you see that I have all the pieces here 00:14:57.070 --> 00:14:59.990 for projection on a line? 00:14:59.990 --> 00:15:04.880 Now, and those -- please remember those. 00:15:04.880 --> 00:15:07.330 So there are three formulas to remember. 00:15:07.330 --> 00:15:11.650 The formula for x, the formula for P, which is just ax, 00:15:11.650 --> 00:15:17.680 and then the formula for capital P, which is the matrix. 00:15:17.680 --> 00:15:18.430 Good. 00:15:18.430 --> 00:15:18.970 Good. 00:15:18.970 --> 00:15:20.400 OK. 00:15:20.400 --> 00:15:26.780 Now I want to move to more dimensions. 00:15:26.780 --> 00:15:30.610 So we're going to have three formulas again but you'll have 00:15:30.610 --> 00:15:32.170 to -- 00:15:32.170 --> 00:15:35.830 they'll be a little different because we won't have a single 00:15:35.830 --> 00:15:40.990 line but -- a plane or three-dimensional 00:15:40.990 --> 00:15:44.530 or a n-dimensional subspace. 00:15:44.530 --> 00:15:45.230 OK. 00:15:45.230 --> 00:15:49.960 So now I'll move to the next question. 00:15:49.960 --> 00:16:04.020 Maybe -- let me say first why I want this projection, 00:16:04.020 --> 00:16:06.200 and then we'll figure out what it is, 00:16:06.200 --> 00:16:11.040 we'll go completely in parallel there, and then we'll use it. 00:16:11.040 --> 00:16:14.420 OK, why do I want this projection? 00:16:14.420 --> 00:16:17.083 Well, so why project? 00:16:21.600 --> 00:16:32.010 It's because I'm as I mentioned last time this new chapter 00:16:32.010 --> 00:16:47.890 deals with equations Ax=b may have no solution. 00:16:54.540 --> 00:16:57.380 So that's really my problem, that I'm 00:16:57.380 --> 00:17:00.240 given a bunch of equations probably too 00:17:00.240 --> 00:17:04.450 many equations, more equations than unknowns, 00:17:04.450 --> 00:17:08.119 and I can't solve them. 00:17:08.119 --> 00:17:09.079 OK. 00:17:09.079 --> 00:17:11.050 So what do I do? 00:17:11.050 --> 00:17:15.359 I solve the closest problem that I can solve. 00:17:15.359 --> 00:17:17.730 And what's the closest one? 00:17:17.730 --> 00:17:21.460 Well, ax will always be in the column space of a. 00:17:21.460 --> 00:17:22.970 That's my problem. 00:17:22.970 --> 00:17:27.190 My problem is ax has to be in the column space 00:17:27.190 --> 00:17:31.130 and b is probably not in the column space. 00:17:31.130 --> 00:17:36.620 So I change b to what? 00:17:36.620 --> 00:17:40.820 I choose the closest vector in the column space, 00:17:40.820 --> 00:17:50.350 so I'll solve Ax equal P instead. 00:17:50.350 --> 00:17:52.660 That one I can do. 00:17:52.660 --> 00:17:57.220 Where P is this is the projection 00:17:57.220 --> 00:18:04.460 of b onto the column space. 00:18:04.460 --> 00:18:07.010 That's why I want to be able to do this. 00:18:07.010 --> 00:18:10.460 Because I have to find a solution here, 00:18:10.460 --> 00:18:13.370 and I'm going to put a little hat there 00:18:13.370 --> 00:18:17.400 to indicate that it's not the x, it's 00:18:17.400 --> 00:18:21.660 not the x that doesn't exist, it's 00:18:21.660 --> 00:18:27.000 the x hat that's best possible. 00:18:27.000 --> 00:18:32.170 So I must be able to figure out what's 00:18:32.170 --> 00:18:35.310 the good projection there. 00:18:35.310 --> 00:18:38.600 What's the good right-hand side that is in the column space 00:18:38.600 --> 00:18:42.330 that's as close as possible to b and then I'm -- 00:18:42.330 --> 00:18:43.820 then I know what to do. 00:18:43.820 --> 00:18:44.620 OK. 00:18:44.620 --> 00:18:47.500 So now I've got that problem. 00:18:47.500 --> 00:18:50.190 So that's why I have the problem again 00:18:50.190 --> 00:18:54.050 but now let me say I'm in three dimensions, 00:18:54.050 --> 00:18:58.630 so I have a plane maybe for example, 00:18:58.630 --> 00:19:06.390 and I have a vector b that's not in the plane. 00:19:06.390 --> 00:19:10.840 And I want to project b down into the plane. 00:19:13.420 --> 00:19:15.510 OK. 00:19:15.510 --> 00:19:17.540 So there's my question. 00:19:17.540 --> 00:19:19.890 How do I project a vector and I'm -- 00:19:19.890 --> 00:19:22.670 what I'm looking for is a nice formula, 00:19:22.670 --> 00:19:28.120 and I'm counting on linear algebra to just come out right, 00:19:28.120 --> 00:19:34.940 a nice formula for the projection of b into the plane. 00:19:34.940 --> 00:19:36.310 The nearest point. 00:19:36.310 --> 00:19:41.990 So this again a right angle is going to be crucial. 00:19:41.990 --> 00:19:42.570 OK. 00:19:42.570 --> 00:19:46.630 Now so what's -- first of all I've got to say what is that 00:19:46.630 --> 00:19:48.100 plane. 00:19:48.100 --> 00:19:51.460 To get a formula I have to tell you what the plane is. 00:19:51.460 --> 00:19:53.990 How I going to tell you a plane? 00:19:53.990 --> 00:19:56.580 I'll tell you a basis for the plane, 00:19:56.580 --> 00:20:03.476 I'll tell you two vectors a one and a two that give you a basis 00:20:03.476 --> 00:20:05.100 for the plane, so that -- let us say -- 00:20:05.100 --> 00:20:12.510 say there's an a one and here's an a -- a vector a two. 00:20:12.510 --> 00:20:15.690 They don't have to be perpendicular. 00:20:15.690 --> 00:20:18.921 But they better be independent, because then that tells me the 00:20:18.921 --> 00:20:19.420 plane. 00:20:19.420 --> 00:20:24.880 The plane is the -- is the plane of a one and a two. 00:20:31.760 --> 00:20:34.320 And actually going back to my -- 00:20:34.320 --> 00:20:39.620 to this connection, this plane is a column space, 00:20:39.620 --> 00:20:47.990 it's the column space of what matrix? 00:20:47.990 --> 00:20:56.610 What matrix, so how do I connect the two questions? 00:20:56.610 --> 00:21:01.220 I'm thinking how do I project onto a plane 00:21:01.220 --> 00:21:06.030 and I want to get a matrix in here. 00:21:06.030 --> 00:21:09.760 Everything's cleaner if I write it in terms of a matrix. 00:21:09.760 --> 00:21:14.680 So what matrix has these -- has that column space? 00:21:14.680 --> 00:21:17.270 Well of course it's just the matrix 00:21:17.270 --> 00:21:20.240 that has a one in the first column 00:21:20.240 --> 00:21:23.790 and a two in the second column. 00:21:23.790 --> 00:21:27.820 Right, just just let's be sure we've got the question 00:21:27.820 --> 00:21:30.060 before we get to the answer. 00:21:30.060 --> 00:21:34.290 So I'm looking for again I'm given 00:21:34.290 --> 00:21:38.140 a matrix a with two columns. 00:21:38.140 --> 00:21:44.110 And really I'm ready once I get to two I'm ready for n. 00:21:44.110 --> 00:21:47.610 So it could be two columns, it could be n columns. 00:21:47.610 --> 00:21:50.930 I'll write the answer in terms of the matrix a. 00:21:50.930 --> 00:21:56.590 And the point will be those two columns describe the plane, 00:21:56.590 --> 00:22:01.331 they describe the column space, and I want to project. 00:22:01.331 --> 00:22:01.830 OK. 00:22:01.830 --> 00:22:06.190 And I'm given a vector b that's probably not in the column 00:22:06.190 --> 00:22:06.770 space. 00:22:06.770 --> 00:22:11.280 Of course, if b is in the column space, my projection is simple, 00:22:11.280 --> 00:22:13.420 it's just b. 00:22:13.420 --> 00:22:19.680 But most likely I have an error e, this b minus P 00:22:19.680 --> 00:22:24.500 part, which is probably not zero. 00:22:24.500 --> 00:22:25.560 OK. 00:22:25.560 --> 00:22:30.330 But the beauty is that I know -- 00:22:30.330 --> 00:22:35.880 from geometry or I could get it from calculus or I could get it 00:22:35.880 --> 00:22:38.724 from linear algebra that that this this vector -- 00:22:42.600 --> 00:22:45.830 this is the part of b that's that's 00:22:45.830 --> 00:22:48.600 perpendicular to the plane. 00:22:48.600 --> 00:22:55.390 That e is perpendicular is perpendicular to the plane. 00:22:58.030 --> 00:23:03.530 If your intuition is saying that that's the crucial fact. 00:23:03.530 --> 00:23:06.900 That's going to give us the answer. 00:23:06.900 --> 00:23:07.540 OK. 00:23:07.540 --> 00:23:10.320 So let me, that's the problem. 00:23:10.320 --> 00:23:14.550 Now for the answer. 00:23:14.550 --> 00:23:18.910 So this is a lecture that's really like moving along. 00:23:18.910 --> 00:23:24.940 Because I'm just plotting that problem up there and asking you 00:23:24.940 --> 00:23:27.130 what combination -- 00:23:27.130 --> 00:23:29.920 now, yeah, so what is it? 00:23:29.920 --> 00:23:31.110 What is this projection P? 00:23:31.110 --> 00:23:42.300 P. This is projection P, is some multiple of these basis 00:23:42.300 --> 00:23:46.740 guys, right, some multiple of the columns. 00:23:46.740 --> 00:23:51.700 But I don't like writing out x one a one plus x two a two, 00:23:51.700 --> 00:23:54.265 I would rather right that as ax. 00:23:56.790 --> 00:24:00.512 Well, actually I should put if I'm really doing everything 00:24:00.512 --> 00:24:02.220 right, I should put a little hat on it -- 00:24:02.220 --> 00:24:05.500 to remember that this x -- 00:24:05.500 --> 00:24:08.190 that those are the numbers and I could have a put 00:24:08.190 --> 00:24:14.470 a hat way back there is right, so this 00:24:14.470 --> 00:24:20.360 is this is the projection, P. P is ax bar. 00:24:20.360 --> 00:24:22.830 And I'm looking for x bar. 00:24:22.830 --> 00:24:25.195 So that's what I want an equation for. 00:24:28.410 --> 00:24:31.630 So now I've got hold of the problem. 00:24:31.630 --> 00:24:36.330 The problem is find the right combination of the columns 00:24:36.330 --> 00:24:44.070 so that the error vector is perpendicular to the plane. 00:24:44.070 --> 00:24:48.840 Now let me turn that into an equation. 00:24:48.840 --> 00:24:53.250 So I'll raise the board and just turn that -- 00:24:53.250 --> 00:24:54.910 what we've just done into an equation. 00:24:58.490 --> 00:25:02.020 So let me I'll write again the main point. 00:25:02.020 --> 00:25:06.600 The projection is ax b- x hat. 00:25:06.600 --> 00:25:08.770 And our problem is find x hat. 00:25:12.670 --> 00:25:23.060 And the key is that b minus ax hat, that's the error. 00:25:23.060 --> 00:25:24.800 This is the e. 00:25:24.800 --> 00:25:30.390 Is perpendicular to the plane. 00:25:35.050 --> 00:25:38.490 That's got to give me well what I looking 00:25:38.490 --> 00:25:40.720 for, I'm looking for two equations now 00:25:40.720 --> 00:25:42.845 because I've got an x one and an x two. 00:25:45.640 --> 00:25:49.910 And I'll get two equations because so this thing e 00:25:49.910 --> 00:25:51.680 is perpendicular to the plane. 00:25:55.470 --> 00:25:56.650 So what does that mean? 00:25:56.650 --> 00:26:00.060 I guess it means it's perpendicular to a one 00:26:00.060 --> 00:26:02.540 and also to a two. 00:26:02.540 --> 00:26:06.320 Right, those are two vectors in the plane and the things 00:26:06.320 --> 00:26:08.240 that are perpendicular to the plane 00:26:08.240 --> 00:26:10.810 are perpendicular to a one and a two. 00:26:10.810 --> 00:26:12.350 Let me just repeat. 00:26:12.350 --> 00:26:15.480 This this guy then is perpendicular to the plane 00:26:15.480 --> 00:26:18.460 so it's perpendicular to that vector and that vector. 00:26:18.460 --> 00:26:22.000 Not -- it's perpendicular to that of course. 00:26:22.000 --> 00:26:25.940 But it's perpendicular to everything I the plane. 00:26:25.940 --> 00:26:31.500 And the plane is really told me by a one and a two. 00:26:31.500 --> 00:26:39.150 So really I have the equations a one transpose b minus ax 00:26:39.150 --> 00:26:41.050 is zero. 00:26:41.050 --> 00:26:48.130 And also a two transpose b minus ax is zero. 00:26:52.321 --> 00:26:53.445 Those are my two equations. 00:26:56.950 --> 00:27:02.410 But I want those in matrix form. 00:27:02.410 --> 00:27:04.430 I want to put those two equations together 00:27:04.430 --> 00:27:08.600 as a matrix equation and it's just comes out right. 00:27:08.600 --> 00:27:12.100 Look at the matrix a transpose. 00:27:12.100 --> 00:27:16.050 Put a one a one transpose is its first row, 00:27:16.050 --> 00:27:25.770 a two transpose is its second row, that multiplies this b-ax, 00:27:25.770 --> 00:27:28.440 and gives me the zero and the zero. 00:27:28.440 --> 00:27:29.410 I'm you see the -- 00:27:39.220 --> 00:27:43.050 this is one way -- to come up with this equation. 00:27:43.050 --> 00:27:44.600 So the equation I'm coming up with 00:27:44.600 --> 00:27:52.550 is a transpose b-ax hat is zero. 00:27:55.250 --> 00:27:57.520 OK. 00:27:57.520 --> 00:27:59.791 That's my equation. 00:27:59.791 --> 00:28:00.290 All right. 00:28:00.290 --> 00:28:03.790 Now I want to stop for a moment before I solve it 00:28:03.790 --> 00:28:06.280 and just think about it. 00:28:06.280 --> 00:28:13.230 First of all do you see that that equation back in the very 00:28:13.230 --> 00:28:17.440 first problem I solved on a line, what was -- 00:28:17.440 --> 00:28:23.610 what was on a line the matrix a only had one column, 00:28:23.610 --> 00:28:27.210 it was just little a. 00:28:27.210 --> 00:28:29.980 So in the first problem I solved, 00:28:29.980 --> 00:28:33.680 projecting on a line, this for capital 00:28:33.680 --> 00:28:35.510 a you just change that to little a 00:28:35.510 --> 00:28:39.160 and you have the same equation that we solved before. 00:28:39.160 --> 00:28:42.670 a transpose e equals zero. 00:28:42.670 --> 00:28:43.300 OK. 00:28:43.300 --> 00:28:48.500 Now a second thing, second comment. 00:28:48.500 --> 00:28:54.220 I would like to since I know about these four subspaces, 00:28:54.220 --> 00:28:57.720 I would like to get them into this picture. 00:29:00.890 --> 00:29:07.270 So let me ask the question, what subspace is this thing in? 00:29:07.270 --> 00:29:11.330 Which of the four subspaces is that error vector e, 00:29:11.330 --> 00:29:13.920 this is this is nothing but e -- 00:29:13.920 --> 00:29:21.204 this is this guy, coming in down perpendicular to the plane. 00:29:21.204 --> 00:29:22.120 What subspace is e in? 00:29:22.120 --> 00:29:22.911 From this equation. 00:29:22.911 --> 00:29:34.470 Well the equation is saying a transpose e is zero. 00:29:34.470 --> 00:29:39.630 So I'm learning here that e is in the null space 00:29:39.630 --> 00:29:40.330 of a transpose. 00:29:44.660 --> 00:29:45.360 Right? 00:29:45.360 --> 00:29:47.000 That's my equation. 00:29:47.000 --> 00:29:51.300 And now I just want to see hey of course that that was right. 00:29:51.300 --> 00:29:57.320 Because things that are in the null space of a transpose, 00:29:57.320 --> 00:30:00.310 what do we know about the null space of a transpose? 00:30:03.170 --> 00:30:06.320 So that last lecture gave us the sort 00:30:06.320 --> 00:30:09.210 of the geometry of these subspaces. 00:30:09.210 --> 00:30:11.200 And the orthogonality of them. 00:30:11.200 --> 00:30:12.950 And do you remember what it was? 00:30:12.950 --> 00:30:16.910 What on the right side of our big figure 00:30:16.910 --> 00:30:20.200 we always have the null space of a transpose 00:30:20.200 --> 00:30:27.270 and the column space of a, and they're orthogonal. 00:30:27.270 --> 00:30:31.750 So e in the null space of a transpose 00:30:31.750 --> 00:30:38.730 is saying e is perpendicular to the column space of a. 00:30:38.730 --> 00:30:39.230 Yes. 00:30:44.850 --> 00:30:49.100 I just feel OK, the damn thing came out right. 00:30:49.100 --> 00:30:58.140 The equation for the equation that I struggled to find for e 00:30:58.140 --> 00:31:03.980 really said what I wanted, that the error 00:31:03.980 --> 00:31:08.290 e is perpendicular to the column space of a, just right. 00:31:08.290 --> 00:31:10.710 And from our four fundamental subspaces 00:31:10.710 --> 00:31:14.780 we knew that that is the same as that. 00:31:14.780 --> 00:31:17.530 To say e is in the null space of a transpose says 00:31:17.530 --> 00:31:19.451 e's perpendicular to the column space. 00:31:19.451 --> 00:31:19.950 OK. 00:31:19.950 --> 00:31:21.900 So we've got this equation. 00:31:21.900 --> 00:31:23.360 Now let's just solve it. 00:31:23.360 --> 00:31:23.860 All right. 00:31:23.860 --> 00:31:28.130 Let me just rewrite it as a transpose 00:31:28.130 --> 00:31:34.100 a x hat equals a transpose b. 00:31:34.100 --> 00:31:37.700 That's our equation. 00:31:37.700 --> 00:31:42.500 That gives us x. 00:31:42.500 --> 00:31:48.140 And -- allow me to keep remembering the one-dimensional 00:31:48.140 --> 00:31:48.970 case. 00:31:48.970 --> 00:31:54.260 The one-dimensional case, this was little a. 00:31:54.260 --> 00:31:56.620 So this was just a number, little a transpose, 00:31:56.620 --> 00:32:02.910 a transpose a was just a vector row times a column, a number. 00:32:02.910 --> 00:32:04.560 And this was a number. 00:32:04.560 --> 00:32:07.480 And x was the ratio of those numbers. 00:32:07.480 --> 00:32:12.350 But now we've got matrices, this one is n by n. 00:32:12.350 --> 00:32:15.211 a transpose a is an n by n matrix. 00:32:15.211 --> 00:32:15.710 OK. 00:32:15.710 --> 00:32:20.260 So can I move to the next board for the solution? 00:32:26.260 --> 00:32:27.470 OK. 00:32:27.470 --> 00:32:30.900 This is the -- the key equation. 00:32:30.900 --> 00:32:35.160 Now I'm ready for the formulas that we have to remember. 00:32:35.160 --> 00:32:38.010 What's x hat? 00:32:38.010 --> 00:32:41.570 What's the projection, what's the projection matrix, 00:32:41.570 --> 00:32:43.620 those are my three questions. 00:32:43.620 --> 00:32:46.050 That we answered in the 1-D case and now 00:32:46.050 --> 00:32:49.720 we're ready for in the n-dimensional case. 00:32:49.720 --> 00:32:50.920 So what is x hat? 00:32:50.920 --> 00:32:55.930 Well, what can I say but a transpose 00:32:55.930 --> 00:33:01.075 a inverse, a transpose b. 00:33:03.750 --> 00:33:07.840 That's the solution to -- to our equation. 00:33:07.840 --> 00:33:08.890 OK. 00:33:08.890 --> 00:33:09.940 What's the projection? 00:33:09.940 --> 00:33:11.430 That's more interesting. 00:33:11.430 --> 00:33:13.500 What's the projection? 00:33:13.500 --> 00:33:18.520 The projection is a x hat. 00:33:18.520 --> 00:33:21.700 That's how x hat got into the picture in the first place. 00:33:21.700 --> 00:33:28.050 x hat was the was the combination of columns 00:33:28.050 --> 00:33:31.440 in the I had to look for those numbers and now I found them. 00:33:31.440 --> 00:33:34.020 Was the combination of the columns of a 00:33:34.020 --> 00:33:35.510 that gave me the projection. 00:33:35.510 --> 00:33:37.060 OK. 00:33:37.060 --> 00:33:40.000 So now I know what this guy is. 00:33:40.000 --> 00:33:42.260 So it's just I multiply by a. 00:33:42.260 --> 00:33:50.110 a a transpose a inverse a transpose b. 00:33:56.010 --> 00:34:00.145 That's looking a little messy but it's not bad. 00:34:03.270 --> 00:34:07.600 That that combination is is our like magic combination. 00:34:07.600 --> 00:34:14.860 This is the thing which is which use which is like what's it 00:34:14.860 --> 00:34:16.504 like, what was it in one dimension? 00:34:19.130 --> 00:34:20.820 What was that we had this we must 00:34:20.820 --> 00:34:25.139 have had this thing way back at the beginning of the lecture. 00:34:25.139 --> 00:34:29.929 What did we -- oh that a was just a column so it was little 00:34:29.929 --> 00:34:36.210 a, little a transpose over a transpose a, right, 00:34:36.210 --> 00:34:46.110 that's what it was in 1-D. You see what's happened in more 00:34:46.110 --> 00:34:47.520 dimensions, I -- 00:34:47.520 --> 00:34:49.810 I'm not allowed to to just divide 00:34:49.810 --> 00:34:52.440 because because I don't have a number, I have to put inverse, 00:34:52.440 --> 00:34:55.320 because I have an n by n matrix. 00:34:55.320 --> 00:34:56.440 But same formula. 00:34:59.120 --> 00:35:03.320 And now tell me what's the projection matrix? 00:35:03.320 --> 00:35:08.955 What matrix is multiplying b to give the projection? 00:35:12.300 --> 00:35:13.660 Right there. 00:35:13.660 --> 00:35:15.010 Because there it -- 00:35:15.010 --> 00:35:17.390 I even already underlined it by accident. 00:35:17.390 --> 00:35:24.100 The projection matrix which I use capital P is this, 00:35:24.100 --> 00:35:28.360 it's it's that thing, shall I write it again, a times 00:35:28.360 --> 00:35:32.770 a transpose a inverse times a transpose. 00:35:41.610 --> 00:35:49.080 Now if you'll bear with me I'll think about what have I done 00:35:49.080 --> 00:35:49.640 here. 00:35:49.640 --> 00:35:53.030 I've got this formula. 00:35:53.030 --> 00:36:00.100 Now the first thing that occurs to me is something bad. 00:36:00.100 --> 00:36:04.360 Look why don't I just you know here's 00:36:04.360 --> 00:36:08.100 a product of two matrices and I want its inverse, 00:36:08.100 --> 00:36:10.460 why don't I just use the formula I 00:36:10.460 --> 00:36:14.030 know for the inverse of a product and say OK, 00:36:14.030 --> 00:36:22.110 that's a inverse times a transpose inverse, what 00:36:22.110 --> 00:36:25.480 will happen if I do that? 00:36:25.480 --> 00:36:29.370 What will happen if I say hey this 00:36:29.370 --> 00:36:36.060 is a inverse times a transpose inverse, then shall I do it? 00:36:36.060 --> 00:36:41.100 It's going to go on videotape if I do it, and I don't -- 00:36:41.100 --> 00:36:43.780 all right, I'll put it there, but just like 00:36:43.780 --> 00:36:48.010 don't take the videotape quite so carefully. 00:36:48.010 --> 00:36:48.770 OK. 00:36:48.770 --> 00:36:52.440 So if I put that thing it -- it would be a a inverse 00:36:52.440 --> 00:36:59.387 a transpose inverse a transpose and what's that? 00:36:59.387 --> 00:37:00.220 That's the identity. 00:37:03.610 --> 00:37:06.660 But what's going on? 00:37:06.660 --> 00:37:14.059 So why -- you see my question is somehow I did something wrong. 00:37:14.059 --> 00:37:15.100 That that wasn't allowed. 00:37:15.100 --> 00:37:19.100 And and and why is that? 00:37:19.100 --> 00:37:23.530 Because a is not a square matrix. 00:37:23.530 --> 00:37:25.060 a is not a square matrix. 00:37:25.060 --> 00:37:28.580 It doesn't have an inverse. 00:37:28.580 --> 00:37:31.850 So I have to leave it that way. 00:37:31.850 --> 00:37:33.310 This is not OK. 00:37:33.310 --> 00:37:36.950 If if a was a square invertible matrix, then then 00:37:36.950 --> 00:37:37.800 I couldn't complain. 00:37:40.980 --> 00:37:44.280 Yeah I think -- let me think about that case. 00:37:44.280 --> 00:37:46.740 But you but my main case, the whole reason 00:37:46.740 --> 00:37:51.240 I'm doing all this, is that a is this matrix that 00:37:51.240 --> 00:37:56.360 has x too many rows, it's just got a couple of columns, 00:37:56.360 --> 00:38:01.280 like a one and a two, but lots of rows. 00:38:01.280 --> 00:38:02.390 Not square. 00:38:02.390 --> 00:38:07.810 And if it's not square, this a transpose a is square but I 00:38:07.810 --> 00:38:10.530 can't pull it apart like this -- 00:38:10.530 --> 00:38:16.650 I'm not allowed to do this pull apart, except if a was square. 00:38:16.650 --> 00:38:19.180 Now if a is square what's what's going on 00:38:19.180 --> 00:38:20.780 if a is a square matrix? 00:38:20.780 --> 00:38:25.070 a nice square inv- invertible matrix. 00:38:25.070 --> 00:38:26.290 Think. 00:38:26.290 --> 00:38:29.790 What's up with that what's with that case. 00:38:29.790 --> 00:38:34.300 So this is that the formula ought to work then too. 00:38:34.300 --> 00:38:38.910 If a is a nice square invertible matrix what's its column space, 00:38:38.910 --> 00:38:44.070 so it's a nice n by n invertible everything great matrix, 00:38:44.070 --> 00:38:50.670 what's its column space, the whole of R^n. 00:38:50.670 --> 00:38:54.190 So what's the projection matrix if I'm projecting 00:38:54.190 --> 00:38:56.900 onto the whole space? 00:38:56.900 --> 00:39:00.510 It's the identity matrix right? 00:39:00.510 --> 00:39:04.800 If I'm projecting b onto the whole space, 00:39:04.800 --> 00:39:08.090 not just onto a plane, but onto all of 3-D, 00:39:08.090 --> 00:39:11.550 then b is already in the column space, 00:39:11.550 --> 00:39:15.930 the projection is the identity, and this is gives me 00:39:15.930 --> 00:39:18.720 the correct formula, P is I. 00:39:18.720 --> 00:39:23.000 But if I'm projecting onto a subspace then 00:39:23.000 --> 00:39:27.310 I can't split those apart and I have to stay with that formula. 00:39:27.310 --> 00:39:30.040 OK. 00:39:30.040 --> 00:39:37.240 And what can I say if -- so I remember this formula for 1-D 00:39:37.240 --> 00:39:40.760 and that's what it looks like in n dimensions. 00:39:40.760 --> 00:39:44.350 And what are the properties that I expected for any projection 00:39:44.350 --> 00:39:44.970 matrix? 00:39:44.970 --> 00:39:47.370 And I still expect for this one? 00:39:47.370 --> 00:39:50.170 That matrix should be symmetric and it is. 00:39:50.170 --> 00:39:55.160 P transpose is P. Because if I transpose this, 00:39:55.160 --> 00:39:59.280 this guy's symmetric, and its inverse is symmetric, 00:39:59.280 --> 00:40:03.340 and if I transpose this one when I transpose 00:40:03.340 --> 00:40:08.050 it will pop up there, become a, that a transpose will pop up 00:40:08.050 --> 00:40:11.440 here, and I'm back to P again. 00:40:11.440 --> 00:40:14.720 And do we dare try the other property 00:40:14.720 --> 00:40:18.182 which is P squared equal P? 00:40:24.790 --> 00:40:25.750 It's got to be right. 00:40:28.670 --> 00:40:33.540 Because we know geometrically that the first projection pops 00:40:33.540 --> 00:40:37.090 us onto the column space and the second one leaves us where we 00:40:37.090 --> 00:40:37.680 are. 00:40:37.680 --> 00:40:43.180 So I expect that if I multiply by let me do it -- 00:40:43.180 --> 00:40:48.030 if I multiply by another P, so there's another a, another 00:40:48.030 --> 00:40:59.040 a transpose a inverse a transpose, can you -- 00:40:59.040 --> 00:41:03.960 eight (a)-s in a row is quite obscene but -- 00:41:03.960 --> 00:41:05.415 do you see that it works? 00:41:08.160 --> 00:41:11.240 So I'm squaring that so what do I do-- how do I 00:41:11.240 --> 00:41:12.610 see that multiplication? 00:41:12.610 --> 00:41:17.480 Well, yeah, I just want to put parentheses in good places, 00:41:17.480 --> 00:41:21.910 so I see what's happening, yeah, here's an a transpose a sitting 00:41:21.910 --> 00:41:25.380 together -- so when that a transpose a multiplies its 00:41:25.380 --> 00:41:29.840 inverse, all that stuff goes, right. 00:41:29.840 --> 00:41:32.300 And leaves just the a transpose at the end, 00:41:32.300 --> 00:41:35.480 which is just what we want. 00:41:35.480 --> 00:41:39.090 So P squared equals P. So sure enough those two 00:41:39.090 --> 00:41:40.200 properties hold. 00:41:40.200 --> 00:41:40.730 OK. 00:41:40.730 --> 00:41:45.510 OK we really have got now all the formulas. 00:41:45.510 --> 00:41:50.370 x hat, the projection P, and the projection matrix capital 00:41:50.370 --> 00:41:58.710 P. And now my job is to use them. 00:41:58.710 --> 00:41:59.210 OK. 00:41:59.210 --> 00:42:05.940 So when would I have a bunch of equations, 00:42:05.940 --> 00:42:12.250 too many equations and yet I want the best answer and the -- 00:42:12.250 --> 00:42:19.210 the most important example, the most common example is if I 00:42:19.210 --> 00:42:25.130 have points so here's the -- here's the application. 00:42:25.130 --> 00:42:25.850 v squared. 00:42:25.850 --> 00:42:27.800 Fitting by a line. 00:42:27.800 --> 00:42:28.300 OK. 00:42:42.170 --> 00:42:46.700 So I'll start this application today and there's more in it 00:42:46.700 --> 00:42:50.240 than I can do in this same lecture. 00:42:50.240 --> 00:42:53.800 So that'll give me a chance to recap the formulas 00:42:53.800 --> 00:42:59.910 and there they are, and recap the ideas. 00:42:59.910 --> 00:43:03.050 So let me start the problem today. 00:43:03.050 --> 00:43:10.500 I'm given a bunch of data points. 00:43:10.500 --> 00:43:14.430 And they lie close to a line but not on a line. 00:43:14.430 --> 00:43:15.660 Let me take that. 00:43:15.660 --> 00:43:19.960 Say a t equal to one, two and three, 00:43:19.960 --> 00:43:25.740 I have one, and two and two again. 00:43:25.740 --> 00:43:31.890 So my data points are this is the like the time direction 00:43:31.890 --> 00:43:37.200 and this is like well let me call that b or y or something. 00:43:37.200 --> 00:43:43.790 I'm given these three points and I want to fit them by a line. 00:43:43.790 --> 00:43:46.130 By the best straight line. 00:43:46.130 --> 00:43:55.700 So the problem is fit the points one, one is the first point -- 00:43:55.700 --> 00:44:02.720 the second point is t equals two, b equal one, 00:44:02.720 --> 00:44:06.735 and the third point is t equal three, b equal to two. 00:44:09.570 --> 00:44:14.920 So those are my three points, t equal sorry,that's two. 00:44:14.920 --> 00:44:17.400 Yeah, OK. 00:44:17.400 --> 00:44:19.380 So this is the point one, one. 00:44:19.380 --> 00:44:22.900 This is the point two, two, and that's the point three, two. 00:44:22.900 --> 00:44:27.430 And of course there isn't a -- a line that goes through them. 00:44:27.430 --> 00:44:29.120 So I'm looking for the best line. 00:44:29.120 --> 00:44:33.670 I'm looking for a line that probably goes somewhere, 00:44:33.670 --> 00:44:38.080 do you think it goes somewhere like that? 00:44:38.080 --> 00:44:41.300 I didn't mean to make it go through that point, it won't. 00:44:41.300 --> 00:44:42.720 It'll kind of -- 00:44:42.720 --> 00:44:47.180 it'll go between so the error there and the error there 00:44:47.180 --> 00:44:54.780 and the error there are as small as I can get them. 00:44:54.780 --> 00:45:00.200 OK, what I'd like to do is find the matrix a. 00:45:00.200 --> 00:45:01.710 Because once I've found the matrix 00:45:01.710 --> 00:45:05.030 a the formulas take over. 00:45:05.030 --> 00:45:11.110 So what I'm looking for this line, b is C+Dt. 00:45:11.110 --> 00:45:16.970 So this is in the homework that I sent out for today. 00:45:16.970 --> 00:45:18.310 Find the best line. 00:45:18.310 --> 00:45:21.170 So I'm looking for these numbers. 00:45:21.170 --> 00:45:23.820 C and D. 00:45:23.820 --> 00:45:27.130 That tell me the line and I want them 00:45:27.130 --> 00:45:30.560 to be as close to going through those three points 00:45:30.560 --> 00:45:32.270 as I can get. 00:45:32.270 --> 00:45:35.320 I can't get exactly so there are three equations 00:45:35.320 --> 00:45:37.870 to go through the three points. 00:45:37.870 --> 00:45:41.110 It would it will go exactly through that point 00:45:41.110 --> 00:45:45.050 if let's see that first point has t equal to one, 00:45:45.050 --> 00:45:48.710 so that would say C+D equaled 1. 00:45:48.710 --> 00:45:51.890 This is the one, one. 00:45:51.890 --> 00:45:55.250 The second point t is two. 00:45:55.250 --> 00:46:00.760 So C+2D should come out to equal 2. 00:46:00.760 --> 00:46:05.190 But I also want to get the third equation in and at that third 00:46:05.190 --> 00:46:11.400 equation t is three so C+3D equals only 2. 00:46:17.400 --> 00:46:19.160 That's the key. 00:46:19.160 --> 00:46:23.450 Is to write down what equations we would like to solve 00:46:23.450 --> 00:46:25.010 but can't. 00:46:25.010 --> 00:46:28.080 Reason we if we could solve them that would mean that we could 00:46:28.080 --> 00:46:33.470 put a line through all three points and of course 00:46:33.470 --> 00:46:39.100 if these numbers one, two, two were different, we could do it. 00:46:39.100 --> 00:46:42.130 But with those numbers, one, two, two, we can't. 00:46:42.130 --> 00:46:49.300 So what is our equation Ax equal Ax equal b that we can't solve? 00:46:49.300 --> 00:46:52.840 I just want to say what's the matrix here, 00:46:52.840 --> 00:46:56.090 what's the unknown x, and what's the right-hand side. 00:46:56.090 --> 00:47:00.975 So this is the matrix is one, one, one, one, two, three. 00:47:03.980 --> 00:47:07.650 The unknown is C and D. 00:47:07.650 --> 00:47:10.350 And the right-hand side if one, two, two. 00:47:10.350 --> 00:47:18.080 Right, I've just taken my equations 00:47:18.080 --> 00:47:28.810 and I've said what is Ax and what is b. 00:47:28.810 --> 00:47:33.640 Then there's no solution, this is the typical case where I 00:47:33.640 --> 00:47:36.860 have three equations -- two unknowns, no solution, 00:47:36.860 --> 00:47:40.640 but I'm still looking for the best solution. 00:47:40.640 --> 00:47:45.780 And the best solution is taken is 00:47:45.780 --> 00:47:49.390 is to solve not this equation Ax equal 00:47:49.390 --> 00:47:55.960 b which has which has no solution but the equation that 00:47:55.960 --> 00:48:01.350 does have a solution, which was this one. 00:48:01.350 --> 00:48:03.720 So that's the equation to solve. 00:48:03.720 --> 00:48:06.110 That's the central equation of the subject. 00:48:06.110 --> 00:48:12.540 I can't solve Ax=b but magically when I multiply both sides 00:48:12.540 --> 00:48:22.280 by a transpose I get an equation that I can solve and its 00:48:22.280 --> 00:48:29.770 solution gives me x, the best x, the best projection, 00:48:29.770 --> 00:48:36.510 and I discover what's the matrix that's behind it. 00:48:36.510 --> 00:48:37.260 OK. 00:48:37.260 --> 00:48:44.010 So next time I'll complete an example, numerical example. 00:48:44.010 --> 00:48:49.250 today was all letters, numbers next time. 00:48:49.250 --> 00:48:50.800 Thanks.