WEBVTT 00:00:01.550 --> 00:00:03.920 The following content is provided under a Creative 00:00:03.920 --> 00:00:05.310 Commons license. 00:00:05.310 --> 00:00:07.520 Your support will help MIT OpenCourseWare 00:00:07.520 --> 00:00:11.610 continue to offer high-quality educational resources for free. 00:00:11.610 --> 00:00:14.180 To make a donation or to view additional materials 00:00:14.180 --> 00:00:18.140 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:18.140 --> 00:00:19.026 at ocw.mit.edu. 00:00:22.720 --> 00:00:27.910 GILBERT STRANG: So just to orient where we are, 00:00:27.910 --> 00:00:34.840 today starts a new chapter about low-rank matrices. 00:00:34.840 --> 00:00:38.830 So that's an important bunch of matrices. 00:00:38.830 --> 00:00:44.620 They can be truly low-rank, like uv transpose. 00:00:44.620 --> 00:00:46.720 That's a rank 1. 00:00:46.720 --> 00:00:51.560 And we have some questions about those. 00:00:51.560 --> 00:00:53.230 And then later in the chapter, we'll 00:00:53.230 --> 00:00:59.096 meet matrices that are approximately low-rank, where 00:00:59.096 --> 00:01:02.850 the singular values drop off like crazy. 00:01:02.850 --> 00:01:08.340 And those are quite remarkable matrices. 00:01:08.340 --> 00:01:11.910 So this is my topic at the beginning-- 00:01:11.910 --> 00:01:16.970 and if it doesn't take the whole hour, 00:01:16.970 --> 00:01:24.210 I want to go back to a topic in chapter 2-- 00:01:24.210 --> 00:01:25.200 that should be 2.4-- 00:01:27.750 --> 00:01:30.520 where we were last time. 00:01:30.520 --> 00:01:31.020 Good. 00:01:35.340 --> 00:01:38.640 So that's for later in the hour. 00:01:38.640 --> 00:01:42.080 This is for now. 00:01:42.080 --> 00:01:44.580 Let's focus on this. 00:01:44.580 --> 00:01:46.080 So what's the question there? 00:01:46.080 --> 00:01:48.870 I start with the identity matrix. 00:01:48.870 --> 00:01:54.750 I perturb it by a matrix of rank 1. 00:01:54.750 --> 00:01:57.720 And I ask what the inverse is. 00:01:57.720 --> 00:02:01.830 So I'm making a small change in the matrix. 00:02:01.830 --> 00:02:04.080 When I say small change, I don't mean 00:02:04.080 --> 00:02:06.760 that the numbers are small. 00:02:06.760 --> 00:02:13.150 In fact, uv transpose could be the all 1's matrix, 00:02:13.150 --> 00:02:15.940 or the all millions matrix, even. 00:02:15.940 --> 00:02:19.330 But its rank is small. 00:02:19.330 --> 00:02:23.110 That's the idea of small that's important here. 00:02:23.110 --> 00:02:26.100 And I would like to know what the inverse is. 00:02:28.810 --> 00:02:32.410 So that's the question here in 4.1. 00:02:32.410 --> 00:02:36.220 And there's a famous formula that has at least three names, 00:02:36.220 --> 00:02:41.140 and probably more, and it's also called the matrix inversion 00:02:41.140 --> 00:02:48.350 formula in signal processing. 00:02:54.250 --> 00:02:56.630 And let me write down the example. 00:02:56.630 --> 00:03:00.200 So I start with the identity matrix. 00:03:00.200 --> 00:03:04.760 I do a rank 1 perturbation. 00:03:04.760 --> 00:03:07.610 And I want to know, what's the inverse? 00:03:07.610 --> 00:03:12.800 And I'll write down the answer and check that it's correct. 00:03:12.800 --> 00:03:15.680 So I'm perturbing the identity matrix 00:03:15.680 --> 00:03:20.480 in this case, whose inverse is also the identity matrix. 00:03:20.480 --> 00:03:26.980 So I'm going to write the answer as a perturbation of I. 00:03:26.980 --> 00:03:30.220 And the question is, what is that perturbation? 00:03:30.220 --> 00:03:33.040 And it's a famous formula. 00:03:33.040 --> 00:03:36.870 There is a uv transpose that comes in, 00:03:36.870 --> 00:03:43.870 copied from there, divided by 1 minus v transpose u. 00:03:46.390 --> 00:03:49.270 Now, let's first of all just see that this 00:03:49.270 --> 00:03:52.930 is a reasonable formula. 00:03:52.930 --> 00:03:58.020 So u a column-- u and v are column vectors, as always. 00:03:58.020 --> 00:04:02.910 So uv transpose is a matrix, column times row. 00:04:02.910 --> 00:04:07.900 And it's being subtracted from the identity. 00:04:07.900 --> 00:04:12.450 And over here on this side, I have uv transpose-- again, 00:04:12.450 --> 00:04:14.740 a rank 1 matrix-- 00:04:14.740 --> 00:04:16.220 divided by a number. 00:04:16.220 --> 00:04:17.109 So that's the point. 00:04:17.109 --> 00:04:17.859 This is a number-- 00:04:23.040 --> 00:04:25.650 a 1 by 1 matrix, you could say. 00:04:28.470 --> 00:04:33.690 So the point of this formula is to find the inverse of an n 00:04:33.690 --> 00:04:37.320 by n matrix in terms of the inverse of a 1 00:04:37.320 --> 00:04:42.780 by 1 matrix, which is a lot simpler and easier to do. 00:04:42.780 --> 00:04:46.650 I mean, that right-hand side is clearly easy. 00:04:46.650 --> 00:04:51.160 And let's see what other things it tells us. 00:04:51.160 --> 00:04:56.380 So this was a rank 1 perturbation in the identity. 00:04:56.380 --> 00:04:59.200 Then when I invert, I get-- 00:04:59.200 --> 00:04:59.860 look at this. 00:04:59.860 --> 00:05:01.600 This is also a rank 1. 00:05:01.600 --> 00:05:04.150 That's a number. 00:05:04.150 --> 00:05:06.160 In fact, it's the very same rank 1 00:05:06.160 --> 00:05:10.990 that we had there in this nice case. 00:05:10.990 --> 00:05:20.320 So conclusion-- if I change a matrix by rank 1, 00:05:20.320 --> 00:05:23.290 I change its inverse by rank 1. 00:05:23.290 --> 00:05:26.860 That doesn't seem completely obvious 00:05:26.860 --> 00:05:31.210 until you go to figure it out. 00:05:31.210 --> 00:05:34.240 When you figure it out, you get a formula that tells you. 00:05:34.240 --> 00:05:38.830 Well, so far what I'm perturbing is the identity matrix. 00:05:38.830 --> 00:05:41.985 When I get here, I'm going to perturb any matrix. 00:05:41.985 --> 00:05:44.620 And I'm going to reach the same conclusion. 00:05:44.620 --> 00:05:52.150 If this is rank 1, then the change in the inverse 00:05:52.150 --> 00:05:53.800 is also rank 1. 00:05:53.800 --> 00:05:57.380 But let's see it first for a equal to the identity. 00:05:57.380 --> 00:06:01.480 So I guess I have two or three questions. 00:06:01.480 --> 00:06:04.405 One would be, how do you check that this is correct? 00:06:08.120 --> 00:06:12.520 Again, remember what it's doing. 00:06:12.520 --> 00:06:14.775 It's pretty neat. 00:06:14.775 --> 00:06:21.630 It's computing an n by n inverse in terms of a 1 by 1 inverse. 00:06:21.630 --> 00:06:22.350 It's a number. 00:06:24.970 --> 00:06:27.490 And that's certainly a favorable exchange. 00:06:27.490 --> 00:06:31.060 So it's a formula that you want and a formula 00:06:31.060 --> 00:06:31.900 that's quite useful. 00:06:35.410 --> 00:06:38.350 Let me just check the formula before I 00:06:38.350 --> 00:06:40.930 talk about how it's useful. 00:06:40.930 --> 00:06:42.230 So how shall I check it? 00:06:42.230 --> 00:06:45.190 I guess, if this is claimed to be the inverse, 00:06:45.190 --> 00:06:47.170 then let's just che-- 00:06:47.170 --> 00:06:49.660 so this will be the check. 00:06:49.660 --> 00:07:00.930 I'll multiply by that, I plus uv transpose over 1 00:07:00.930 --> 00:07:05.400 minus v transpose u, the claimed inverse. 00:07:05.400 --> 00:07:09.610 And what am I hoping to get from this multiplication? 00:07:09.610 --> 00:07:10.110 AUDIENCE: I. 00:07:10.110 --> 00:07:14.360 GILBERT STRANG: I. I'm hoping that, that result is 00:07:14.360 --> 00:07:15.990 the identity matrix. 00:07:15.990 --> 00:07:18.950 So I'm just going to do it out, and we'll 00:07:18.950 --> 00:07:21.020 see the identity appear. 00:07:21.020 --> 00:07:24.910 So that times I-- so let me just write that part first, 00:07:24.910 --> 00:07:28.700 I minus uv transpose. 00:07:28.700 --> 00:07:32.030 And now I minus uv transpose times this-- 00:07:32.030 --> 00:07:38.720 so that's I minus uv transpose times 00:07:38.720 --> 00:07:46.040 uv transpose over 1 minus v transpose u. 00:07:46.040 --> 00:07:51.440 It's just multiplying it out and seeing 00:07:51.440 --> 00:07:54.530 that we get the identity. 00:07:54.530 --> 00:07:56.630 I see the identity there. 00:07:56.630 --> 00:08:03.230 So I guess, I hope that all this part reduces to-- 00:08:03.230 --> 00:08:04.490 what does it reduce to? 00:08:07.610 --> 00:08:09.980 Let me do that numerator there. 00:08:09.980 --> 00:08:17.490 So that's uv transpose minus u-- 00:08:17.490 --> 00:08:22.740 ooh-- minus u times v transpose u-- 00:08:22.740 --> 00:08:25.410 do you see what's happening here-- 00:08:25.410 --> 00:08:26.310 v transpose. 00:08:30.390 --> 00:08:33.600 The key is that when I multiplied that times 00:08:33.600 --> 00:08:37.900 that, I've got four things in a row. 00:08:37.900 --> 00:08:40.570 And I do the middle pair first. 00:08:40.570 --> 00:08:43.270 Because that's a number, v transpose u. 00:08:43.270 --> 00:08:49.300 So I had uv transpose from that and uv transpose times 00:08:49.300 --> 00:08:50.590 that number minus-- 00:08:50.590 --> 00:08:53.170 do you see-- what have I got here? 00:08:56.540 --> 00:08:57.170 I've got-- 00:08:57.170 --> 00:08:58.245 AUDIENCE: [INAUDIBLE] 00:08:58.245 --> 00:08:59.120 GILBERT STRANG: Yeah. 00:08:59.120 --> 00:09:01.230 I've got uv transpose. 00:09:01.230 --> 00:09:04.830 Now I factor out a 1 minus v transpose u, 00:09:04.830 --> 00:09:08.390 which is just what I want. 00:09:08.390 --> 00:09:11.540 I factor the 1 minus v transpose u out of here. 00:09:11.540 --> 00:09:12.930 It's there. 00:09:12.930 --> 00:09:14.280 Cancel those. 00:09:14.280 --> 00:09:18.330 And then I'm left with a uv transpose, which cancels 00:09:18.330 --> 00:09:22.200 that and leaves the identity. 00:09:22.200 --> 00:09:24.900 It's kind of magic. 00:09:24.900 --> 00:09:30.130 Of course, somebody had to figure out that formula 00:09:30.130 --> 00:09:33.160 in the first place. 00:09:33.160 --> 00:09:43.510 I could do the next one, if you like, since I'm on a roll. 00:09:43.510 --> 00:09:47.800 Suppose I take this second one, now what's the difference in-- 00:09:47.800 --> 00:09:52.120 it's uv transpose, but those are larger letters, the u 00:09:52.120 --> 00:09:55.980 and the v. So what am I meaning by those? 00:09:55.980 --> 00:09:57.650 Those are matrices. 00:09:57.650 --> 00:10:00.200 This is a bigger rank. 00:10:00.200 --> 00:10:03.230 u has k columns. 00:10:03.230 --> 00:10:06.380 v transpose has k rows. 00:10:06.380 --> 00:10:11.960 That product uv transpose is k by k. 00:10:11.960 --> 00:10:15.830 So let me see what I think the formula would be. 00:10:15.830 --> 00:10:21.260 So now, I minus uv transpose inverse-- 00:10:21.260 --> 00:10:28.810 so this is n by k times k by n. 00:10:28.810 --> 00:10:31.000 So it's n by n matrix. 00:10:31.000 --> 00:10:32.320 And this is the ide-- 00:10:32.320 --> 00:10:33.730 identity of this. 00:10:33.730 --> 00:10:36.670 But its rank is-- 00:10:36.670 --> 00:10:38.763 what is the rank of that now? 00:10:38.763 --> 00:10:39.430 AUDIENCE: k. 00:10:39.430 --> 00:10:40.180 GILBERT STRANG: k. 00:10:40.180 --> 00:10:41.170 Right. 00:10:41.170 --> 00:10:47.200 So I have here, the whole thing is the inverse 00:10:47.200 --> 00:10:49.520 of an n by n matrix. 00:10:49.520 --> 00:10:51.130 So I have an n by n matrix. 00:10:51.130 --> 00:10:56.290 Let me put that up here, n by n matrix to invert. 00:11:02.300 --> 00:11:03.890 There it is. 00:11:03.890 --> 00:11:05.660 I'm going to write down the formula. 00:11:05.660 --> 00:11:07.535 And you're going to be able to write it down, 00:11:07.535 --> 00:11:10.880 because it's just copied from that one. 00:11:10.880 --> 00:11:16.470 And you'll see that it involves a k by k inverse. 00:11:16.470 --> 00:11:18.910 So I have an n by n matrix to invert, 00:11:18.910 --> 00:11:21.000 but I don't have to do that. 00:11:21.000 --> 00:11:23.870 I can switch it to a k by k matrix. 00:11:23.870 --> 00:11:27.300 That's pretty nice. 00:11:27.300 --> 00:11:29.370 So let's do it. 00:11:29.370 --> 00:11:37.090 So I'm basically gonna copy that I plus u-- 00:11:37.090 --> 00:11:40.070 now, I've matrices. 00:11:40.070 --> 00:11:42.890 So that's an inverse, but I can't leave it 00:11:42.890 --> 00:11:43.980 as a denominator. 00:11:43.980 --> 00:11:46.835 Because through here, we're talking about a k by k matrix. 00:11:51.130 --> 00:12:00.070 So I have to put it like any matrix inverse, Ik minus v 00:12:00.070 --> 00:12:06.040 transpose u inverse. 00:12:06.040 --> 00:12:08.170 So I'm just copying this formula, 00:12:08.170 --> 00:12:12.220 wisely taking the u first, then this part, 00:12:12.220 --> 00:12:13.800 then finally, the v transpose. 00:12:18.080 --> 00:12:20.230 I put this one up. 00:12:20.230 --> 00:12:23.620 This one now has made that one obsolete. 00:12:23.620 --> 00:12:25.900 This was the case k equal to 1 over here. 00:12:28.430 --> 00:12:30.990 But I think it helps to see it. 00:12:30.990 --> 00:12:33.640 It certainly helps me to see the formula first 00:12:33.640 --> 00:12:41.860 for a rank 1 perturbation, which is a very likely possibility, 00:12:41.860 --> 00:12:45.610 and then see that we have a totally analogous formula. 00:12:45.610 --> 00:12:48.940 It's sort of fun to check it. 00:12:48.940 --> 00:12:51.710 So I plan to do the same check. 00:12:51.710 --> 00:12:55.690 But I'll just notice here that I have a k by k inverse. 00:13:02.490 --> 00:13:06.390 So I'm exchanging something that's a perturbation of n 00:13:06.390 --> 00:13:11.880 by n identity, an n by n matrix, to have 00:13:11.880 --> 00:13:16.260 to invert something that's a k by k matrix, 00:13:16.260 --> 00:13:21.840 perturbing the k by k identity, much smaller. 00:13:21.840 --> 00:13:25.200 In that case, k was 1. 00:13:25.200 --> 00:13:28.620 So are you good for checking it now? 00:13:28.620 --> 00:13:32.010 So I want to multiply that by this 00:13:32.010 --> 00:13:35.830 and hopefully get In, the identity. 00:13:35.830 --> 00:13:38.155 This is the n by n identity. 00:13:43.390 --> 00:13:47.480 I haven't given credit to the two or three or four, 00:13:47.480 --> 00:13:50.560 or possibly 11 people, who found this formula. 00:13:53.300 --> 00:13:57.200 And I'm not see-- oh, yeah, here are their names. 00:13:57.200 --> 00:14:01.720 Is it OK to make them famous by putting their names up here? 00:14:01.720 --> 00:14:03.430 Yes. 00:14:03.430 --> 00:14:10.440 Sherman-- and I couldn't tell you who did what. 00:14:12.960 --> 00:14:22.840 Morrison, Woodbury, and then no doubt others-- 00:14:22.840 --> 00:14:26.470 maybe one of them did this rank 1 case, 00:14:26.470 --> 00:14:28.450 and then another generalized it. 00:14:28.450 --> 00:14:34.690 And then you could see from the outline there that eventually 00:14:34.690 --> 00:14:37.460 we'll-- we can perturb a matrix A. Of course, 00:14:37.460 --> 00:14:41.770 that's not going to be much harder than the identity. 00:14:41.770 --> 00:14:44.560 Let's do just one more check, and then 00:14:44.560 --> 00:14:45.865 show how it could be used. 00:14:49.630 --> 00:14:51.260 I multiply that by that. 00:14:51.260 --> 00:14:54.080 Can we do that-- 00:14:54.080 --> 00:14:57.680 this thing times this, and hope to get the identity? 00:14:57.680 --> 00:15:01.130 So first, I'll write down this thing. 00:15:01.130 --> 00:15:07.630 So I won't put equal, because I'm multiplying that by that. 00:15:07.630 --> 00:15:11.680 So I get In minus uv transpose. 00:15:14.620 --> 00:15:18.840 On the left side, of course, I'm getting the identity, 00:15:18.840 --> 00:15:21.860 and hoping I'm getting the identity on the right. 00:15:21.860 --> 00:15:23.780 So I'm multiplying by this. 00:15:23.780 --> 00:15:25.010 I did it there. 00:15:25.010 --> 00:15:30.440 Now I have to put I minus uv transposed. 00:15:30.440 --> 00:15:39.110 It takes faith here. u I minus v transpose u inverse v 00:15:39.110 --> 00:15:39.920 transpose-- 00:15:46.100 --> 00:15:46.600 gulp. 00:15:49.210 --> 00:15:52.380 Shall I just leave it there? 00:15:52.380 --> 00:15:56.860 Or you're-- had lunch, you're strong. 00:15:56.860 --> 00:15:58.930 Let's see what we can do. 00:15:58.930 --> 00:16:01.540 This part-- fine. 00:16:01.540 --> 00:16:05.470 This part-- oh, boy. 00:16:05.470 --> 00:16:08.010 What do I do here? 00:16:08.010 --> 00:16:17.120 That trick is going to be that this dopey-looking thing there 00:16:17.120 --> 00:16:20.460 can be written differently. 00:16:20.460 --> 00:16:22.590 Well, just tell me what it equals. 00:16:22.590 --> 00:16:29.520 It equals u minus uv transpose u. 00:16:29.520 --> 00:16:32.740 Everybody sees that. 00:16:32.740 --> 00:16:36.050 But now what am I going to do? 00:16:36.050 --> 00:16:43.640 I'm going to put the parentheses differently. 00:16:43.640 --> 00:16:46.670 I'm bringing u, the u that was on the right up there-- 00:16:46.670 --> 00:16:49.020 I'm going to put it on the left. 00:16:49.020 --> 00:16:50.610 You see that? 00:16:50.610 --> 00:16:55.330 So this is obviously u minus uv transpose u. 00:16:55.330 --> 00:16:58.410 And now I'm going to write it another time. 00:16:58.410 --> 00:17:03.490 It's u times I minus v transpose u. 00:17:03.490 --> 00:17:06.300 I'm going to factor the u out on the left side 00:17:06.300 --> 00:17:11.780 instead of where it originally came on the right side of this. 00:17:11.780 --> 00:17:14.640 You see the great news there? 00:17:14.640 --> 00:17:19.619 This was actually well-organized by your professor-- 00:17:19.619 --> 00:17:21.540 by accident. 00:17:21.540 --> 00:17:24.390 So what do I do now? 00:17:24.390 --> 00:17:25.920 This thing-- look. 00:17:25.920 --> 00:17:30.900 I've got exactly here what I've got inverted there. 00:17:30.900 --> 00:17:36.600 So altogether, I have I minus uv transpose plus this u, 00:17:36.600 --> 00:17:39.070 this cancelled, times v transpose. 00:17:39.070 --> 00:17:45.650 So I get I. So it was just this little sleight of hand, 00:17:45.650 --> 00:17:49.150 where u came in on the right and came out on the left side. 00:17:52.880 --> 00:17:58.020 And there'd be a similar formula with A in there. 00:17:58.020 --> 00:18:01.170 So now, it's a fair question, how 00:18:01.170 --> 00:18:03.770 did anybody come up with these formulas? 00:18:03.770 --> 00:18:08.190 We're proving them correct just by checking that they 00:18:08.190 --> 00:18:11.820 give the identity matrix. 00:18:11.820 --> 00:18:15.780 We should really think how to find 00:18:15.780 --> 00:18:18.750 the formula in the first place. 00:18:18.750 --> 00:18:22.070 I could go back to this one. 00:18:22.070 --> 00:18:24.768 How could you find that formula in the first place? 00:18:24.768 --> 00:18:26.060 Well, here is one way to do it. 00:18:31.890 --> 00:18:36.780 This is then about "to discover the formula." 00:18:43.530 --> 00:18:56.130 And then, over here, let me put over here "to use the formula," 00:18:56.130 --> 00:18:57.860 while I think of it. 00:18:57.860 --> 00:19:02.020 And actually, I have two uses in mind. 00:19:02.020 --> 00:19:04.995 Use one would be to solve-- 00:19:12.110 --> 00:19:15.410 I'm just doing I. Suppose I want to-- 00:19:15.410 --> 00:19:20.750 so this is my new matrix uv transpose x 00:19:20.750 --> 00:19:25.120 equals some right-hand side b. 00:19:25.120 --> 00:19:31.150 Solve a linear system that has that coefficient matrix. 00:19:31.150 --> 00:19:44.790 The use two would be, in least squares, a new measurement 00:19:44.790 --> 00:19:48.760 or observation or data point in these squares. 00:19:56.460 --> 00:19:59.250 So what do I mean by this? 00:19:59.250 --> 00:20:05.250 The old problem was Ax equal b. 00:20:09.690 --> 00:20:13.150 And I'm thinking, when I'm talking about least squares 00:20:13.150 --> 00:20:17.830 here, I'm imagining that A is rectangular. 00:20:17.830 --> 00:20:19.620 Too many equations-- 00:20:19.620 --> 00:20:22.330 A is a tall, thin matrix, just the kind 00:20:22.330 --> 00:20:23.890 we've been talking about. 00:20:23.890 --> 00:20:27.190 So that equation becomes-- 00:20:27.190 --> 00:20:32.720 of course, we know we go to the normal equations-- 00:20:32.720 --> 00:20:39.200 A transpose Ax, the good x, is A transpose b. 00:20:42.840 --> 00:20:44.928 Now we get a new measurement. 00:20:48.280 --> 00:20:50.710 So a new measurement, how does a new measurement 00:20:50.710 --> 00:20:52.150 change the problem? 00:20:52.150 --> 00:20:55.210 So this is the old problem before the measurement 00:20:55.210 --> 00:20:56.470 comes in. 00:20:56.470 --> 00:20:58.060 Now a new measurement arrives. 00:20:58.060 --> 00:21:02.610 So that's another b, a b M plus 1. 00:21:02.610 --> 00:21:05.800 And we get another equation. 00:21:05.800 --> 00:21:07.360 We get another equation. 00:21:07.360 --> 00:21:10.150 That's the new measurement, new point 00:21:10.150 --> 00:21:14.360 to get this straight line it's trying to stay close to. 00:21:14.360 --> 00:21:17.500 So we'll call this equation-- 00:21:17.500 --> 00:21:19.000 I don't know what we should call it. 00:21:19.000 --> 00:21:23.440 Maybe it'll be one more row. 00:21:23.440 --> 00:21:29.510 So now, old now is becoming new. 00:21:29.510 --> 00:21:33.440 I'm sort of more excited about this than proving the formula. 00:21:33.440 --> 00:21:35.840 So I'll just keep going. 00:21:35.840 --> 00:21:42.470 So there is our M measurements, M being large, 00:21:42.470 --> 00:21:44.927 A being a tall, thin matrix. 00:21:44.927 --> 00:21:46.760 And now we're going to give it one more row. 00:21:46.760 --> 00:21:48.560 We're going to make it even taller, 00:21:48.560 --> 00:21:57.320 say maybe, v transpose times this same x is some b 00:21:57.320 --> 00:22:00.980 new, or something like that. 00:22:00.980 --> 00:22:04.970 So there's one more row, one more measurement. 00:22:04.970 --> 00:22:09.192 So what happens to the normal equation? 00:22:09.192 --> 00:22:11.640 This makes it even more likely. 00:22:11.640 --> 00:22:14.630 There's another point, hoping that a straight line will 00:22:14.630 --> 00:22:15.230 go through. 00:22:15.230 --> 00:22:16.960 But if we give it one more point, 00:22:16.960 --> 00:22:21.500 there is even less chance of a straight line going exactly 00:22:21.500 --> 00:22:22.310 through. 00:22:22.310 --> 00:22:24.680 But still, so what do I-- 00:22:24.680 --> 00:22:26.420 this is my new matrix. 00:22:26.420 --> 00:22:29.080 So what's my new normal equation? 00:22:29.080 --> 00:22:41.110 The new normal equation, the A, or transpose-- 00:22:41.110 --> 00:22:44.420 the A has got a new row. 00:22:44.420 --> 00:22:46.280 A has got a new row. 00:22:46.280 --> 00:22:48.590 So a transpose will have a new column. 00:22:48.590 --> 00:22:51.020 I'm just copying the normal equation, 00:22:51.020 --> 00:22:54.030 but I'm giving it its new thing. 00:22:54.030 --> 00:22:56.000 It's got a new column. 00:22:56.000 --> 00:23:01.520 That is my A with a new row. 00:23:01.520 --> 00:23:09.320 This is my x hat, my least squares answer, new. 00:23:09.320 --> 00:23:13.400 And A transpose is this. 00:23:13.400 --> 00:23:22.280 A transpose with a new, now, column times b, so b and b new. 00:23:26.660 --> 00:23:28.930 Pretty OK with this. 00:23:28.930 --> 00:23:31.580 Do you see that this is the new normal equation? 00:23:31.580 --> 00:23:35.210 I'm using the new matrix and the new right-hand side. 00:23:38.030 --> 00:23:42.146 And just one more data point has come 00:23:42.146 --> 00:23:47.650 into the system from the sensor. 00:23:47.650 --> 00:23:49.640 And what's the key point here? 00:23:49.640 --> 00:23:54.880 The key point is I don't want to recompute. 00:23:54.880 --> 00:23:59.170 I don't want to multiply that matrix again. 00:23:59.170 --> 00:24:02.080 I don't want to start over. 00:24:02.080 --> 00:24:08.650 I don't want to compute this A transpose times that A. I want 00:24:08.650 --> 00:24:09.910 to use what I've already done. 00:24:13.330 --> 00:24:15.400 If I multiply those two together, 00:24:15.400 --> 00:24:16.510 what do I actually get? 00:24:19.280 --> 00:24:26.900 So this is, I'm adding a new column here and a new row here. 00:24:26.900 --> 00:24:28.850 Tell me what you think the answer is, and then 00:24:28.850 --> 00:24:30.290 let's just see why. 00:24:30.290 --> 00:24:32.650 I'm asking you what that matrix is. 00:24:35.947 --> 00:24:39.250 What do you think? 00:24:39.250 --> 00:24:42.940 Start me out, anyway. 00:24:42.940 --> 00:24:46.450 I'm just asking for ordinary matrix multiplication. 00:24:46.450 --> 00:24:49.880 Well, I guess I'm asking you to do it 00:24:49.880 --> 00:24:57.220 columns times rows since that's what 18 and 6.5 specializes in. 00:24:57.220 --> 00:25:04.120 So I think I have that times that, A transpose A, 00:25:04.120 --> 00:25:07.540 plus one new column times one new row. 00:25:07.540 --> 00:25:15.520 vv transpose is multiplying this x hat new. 00:25:15.520 --> 00:25:18.190 And over on the right side, I get whatever I had, 00:25:18.190 --> 00:25:25.280 the A transpose b old and the v b new. 00:25:25.280 --> 00:25:29.320 But let me come back to that, because that really shows you 00:25:29.320 --> 00:25:33.250 why you must understand matrix multiplication, 00:25:33.250 --> 00:25:36.930 both the usual row times column-- 00:25:36.930 --> 00:25:41.200 but that would not be so attractive here-- 00:25:41.200 --> 00:25:47.080 and also the new way as columns times rows. 00:25:47.080 --> 00:25:50.200 Because when I see it as columns times rows, 00:25:50.200 --> 00:25:54.160 I see that I have the same columns and same rows there. 00:25:54.160 --> 00:25:56.740 So that's just what I already knew. 00:25:56.740 --> 00:25:59.470 And then I have one new column times one new row. 00:25:59.470 --> 00:26:03.850 Of course, that's column n plus 1, and that's a row n plus 1. 00:26:03.850 --> 00:26:06.580 And they give that rank 1. 00:26:06.580 --> 00:26:10.890 It's a rank 1 change in A transpose A. It's a rank 1 00:26:10.890 --> 00:26:13.280 change in A transpose A. 00:26:13.280 --> 00:26:18.140 So this is like part of least squares. 00:26:18.140 --> 00:26:24.050 I mean, you can see the relevance in a real problem. 00:26:24.050 --> 00:26:27.940 You maybe have a missile flying along. 00:26:27.940 --> 00:26:31.210 You've sent up a satellite, GPS satellite. 00:26:31.210 --> 00:26:32.690 You're tracking it. 00:26:32.690 --> 00:26:34.790 More data comes in. 00:26:34.790 --> 00:26:39.170 The data is just one more position. 00:26:39.170 --> 00:26:41.490 The tracker isn't perfect. 00:26:41.490 --> 00:26:42.590 So we're going to fit-- 00:26:42.590 --> 00:26:47.770 well, here I'm fitting a straight line, maybe. 00:26:47.770 --> 00:26:51.410 But we're fitting to the data. 00:26:51.410 --> 00:26:57.160 And the only change in the left-hand, the big problem, 00:26:57.160 --> 00:27:01.390 the big part of the computation is the A transpose A part. 00:27:01.390 --> 00:27:06.740 And it's just changed by rank 1, or by rank k, 00:27:06.740 --> 00:27:10.190 if we had k new data points. 00:27:10.190 --> 00:27:13.010 If we had k new data points, then this 00:27:13.010 --> 00:27:16.040 would be a rank k matrix. 00:27:16.040 --> 00:27:19.840 So you see, then I go back here-- 00:27:19.840 --> 00:27:20.990 well, OK. 00:27:20.990 --> 00:27:23.150 Now I'm perturbing A transpose A. 00:27:23.150 --> 00:27:28.370 And I haven't given a formula for that one yet. 00:27:28.370 --> 00:27:30.950 I've only perturbed up to, now, the identity. 00:27:30.950 --> 00:27:34.370 But you can believe, since all these formulas are working, 00:27:34.370 --> 00:27:37.520 that Sherman or Morrison or Woodbury 00:27:37.520 --> 00:27:44.130 came up with the correct perturbation for A transpose A. 00:27:44.130 --> 00:27:51.560 So I'm sort of happy that, that application, so natural, 00:27:51.560 --> 00:27:55.565 came out so simply. 00:27:58.440 --> 00:28:02.090 While I'm on that sort of subject, 00:28:02.090 --> 00:28:05.700 have you ever heard of the Kalman filter? 00:28:05.700 --> 00:28:11.010 So that Kalman filter, what's that about? 00:28:11.010 --> 00:28:13.430 It's about exactly this. 00:28:13.430 --> 00:28:15.590 It's about dynamic least squares. 00:28:15.590 --> 00:28:17.270 It's about least squares problems 00:28:17.270 --> 00:28:20.130 in which new data is coming in. 00:28:20.130 --> 00:28:22.900 That's what the Kalman filter-- or in other words, 00:28:22.900 --> 00:28:26.170 let me just write a couple of words up here. 00:28:26.170 --> 00:28:31.210 This is really recursive least squares. 00:28:31.210 --> 00:28:36.700 Recursive least squares-- what I mean by recursive 00:28:36.700 --> 00:28:39.600 is new data comes in. 00:28:39.600 --> 00:28:45.650 It changes the answer, but it doesn't change our method. 00:28:45.650 --> 00:28:56.120 And then the Kalman filter is a very, very big deal 00:28:56.120 --> 00:28:57.585 in guidance. 00:28:57.585 --> 00:29:05.230 If you're sending up a missile, a satellite, you track it. 00:29:05.230 --> 00:29:09.880 You do just what I've been discussing here. 00:29:09.880 --> 00:29:12.860 But the Kalman filter is-- 00:29:12.860 --> 00:29:16.630 it's got more possibilities built in than this one. 00:29:16.630 --> 00:29:20.980 This is the simplest update possible. 00:29:20.980 --> 00:29:25.800 And it would go in this category. 00:29:25.800 --> 00:29:31.810 Kalman went beyond a standard update. 00:29:31.810 --> 00:29:33.620 How did he go beyond? 00:29:33.620 --> 00:29:36.030 Let's see. 00:29:36.030 --> 00:29:38.500 If I've used the words Kalman filter, 00:29:38.500 --> 00:29:42.730 I should tell you what it is that Kalman does 00:29:42.730 --> 00:29:45.460 and what is that least squares problem. 00:29:45.460 --> 00:29:49.150 It's just part of general knowledge, it seems to me. 00:29:52.340 --> 00:29:54.560 So what is it, more gen-- 00:29:54.560 --> 00:29:56.610 what are the additional pieces? 00:29:56.610 --> 00:30:03.010 You've seen the main idea here of getting a simple recursive 00:30:03.010 --> 00:30:06.130 step that doesn't require recomputing all 00:30:06.130 --> 00:30:08.110 that you did before. 00:30:08.110 --> 00:30:13.780 And of course, to this inverse, I'm 00:30:13.780 --> 00:30:16.190 going to apply the Sherman-Morrison-Woodbury 00:30:16.190 --> 00:30:17.140 formula. 00:30:17.140 --> 00:30:20.730 So I'll use the inverse that I had before. 00:30:20.730 --> 00:30:23.340 And this will be a rank 1-- 00:30:23.340 --> 00:30:27.270 this is a rank 1 perturbation of that. 00:30:27.270 --> 00:30:30.270 I'm looking here at A transpose A. Over there, 00:30:30.270 --> 00:30:33.330 I was looking at, I just called it A. 00:30:33.330 --> 00:30:35.850 Or I even called it the identity matrix. 00:30:35.850 --> 00:30:38.490 But it's whatever matrix you have here 00:30:38.490 --> 00:30:40.620 with a rank 1 perturbation. 00:30:40.620 --> 00:30:44.220 And the whole thing has to be inverted. 00:30:44.220 --> 00:30:48.480 And so I was going to say what is additional, 00:30:48.480 --> 00:30:51.970 just so you know about Kalman filters. 00:30:51.970 --> 00:30:53.880 So two things are additional. 00:30:53.880 --> 00:30:59.320 The point is Kalman filters are for dynamic least squares. 00:30:59.320 --> 00:31:02.080 I would say dynamic squares. 00:31:02.080 --> 00:31:07.430 And so there are two ingredients that you haven't seen here. 00:31:07.430 --> 00:31:13.770 One ingredient is the idea of using 00:31:13.770 --> 00:31:16.980 the covariance matrix, which tells you 00:31:16.980 --> 00:31:20.010 how errors are correlated. 00:31:20.010 --> 00:31:23.010 So that would be weighted least squares, 00:31:23.010 --> 00:31:25.485 or correlated least squares. 00:31:29.040 --> 00:31:36.210 So these squares-- let me just remind you, 00:31:36.210 --> 00:31:37.930 if I can write it here. 00:31:37.930 --> 00:31:42.325 So least squares-- standard. 00:31:48.120 --> 00:31:53.970 Standard meaning that data is not correlated. 00:31:53.970 --> 00:31:57.900 It all has the same variance. 00:31:57.900 --> 00:32:00.750 These are the statistics words that I'm 00:32:00.750 --> 00:32:08.770 using last time and this, but that I'll talk about properly. 00:32:08.770 --> 00:32:12.370 So the standard one is the covari-- 00:32:12.370 --> 00:32:20.530 the standard means covariance equal the identity matrix. 00:32:23.890 --> 00:32:28.090 You're doing Gaussian normal probability 00:32:28.090 --> 00:32:35.300 but with just standard Gaussians, standard Gaussians. 00:32:35.300 --> 00:32:42.370 So that's one aspect, which in the work of the Draper lab, 00:32:42.370 --> 00:32:47.320 let's say, they have to think, OK, 00:32:47.320 --> 00:32:50.500 they're getting sensors, different kinds of sensors, 00:32:50.500 --> 00:32:53.680 with different accuracies, different reliability. 00:32:53.680 --> 00:32:57.980 So they have to take account of the covariance matrix. 00:32:57.980 --> 00:33:00.920 Then the other point is-- 00:33:00.920 --> 00:33:04.130 and also-- so that was point one. 00:33:04.130 --> 00:33:07.055 Point two is the dynamic part. 00:33:11.050 --> 00:33:12.530 There is a state equation. 00:33:15.920 --> 00:33:19.290 So I'm really into the edge of control theory. 00:33:19.290 --> 00:33:22.970 So let me just use some words here that you've seen, 00:33:22.970 --> 00:33:23.960 if you've-- 00:33:23.960 --> 00:33:28.550 so control theory has state equation. 00:33:28.550 --> 00:33:29.960 What's a state equation? 00:33:29.960 --> 00:33:31.610 What is the state? 00:33:31.610 --> 00:33:35.900 This is the position, in my example, 00:33:35.900 --> 00:33:38.526 is the position of the satellite. 00:33:43.880 --> 00:33:50.580 So are we looking for a fixed satellite? 00:33:50.580 --> 00:33:51.450 Certainly not. 00:33:51.450 --> 00:33:53.340 The satellite is moving. 00:33:53.340 --> 00:33:59.700 So this state equation tells me how much the satellite-- 00:33:59.700 --> 00:34:04.080 it's Newton's law-- tells me where the satellite should be. 00:34:04.080 --> 00:34:06.090 Where am I looking? 00:34:06.090 --> 00:34:10.469 And then the least squares tells me, it says, 00:34:10.469 --> 00:34:15.929 look around that sort of median position 00:34:15.929 --> 00:34:21.239 for the actual position that the data is giving. 00:34:21.239 --> 00:34:24.750 So just to say-- 00:34:24.750 --> 00:34:28.230 let me-- let me summarize this. 00:34:28.230 --> 00:34:34.040 The Kalman filter is a significantly improved version 00:34:34.040 --> 00:34:35.929 of recursive least squares. 00:34:35.929 --> 00:34:38.270 That's recursive least squares. 00:34:38.270 --> 00:34:43.679 New measurement comes in, changes things 00:34:43.679 --> 00:34:48.270 but leaves a big part unchanged. 00:34:48.270 --> 00:34:54.510 And you find that new x hat. 00:34:54.510 --> 00:35:02.520 With a Kalman filter, there's an covariance matrix in the middle 00:35:02.520 --> 00:35:03.360 here. 00:35:03.360 --> 00:35:05.980 That's where covariance matrices go. 00:35:05.980 --> 00:35:11.520 That's matrix covariance, or inverse covariance, times that. 00:35:11.520 --> 00:35:19.060 And oh, why don't we see the covariances at the right time? 00:35:19.060 --> 00:35:22.650 So maybe those minutes that I've occupied 00:35:22.650 --> 00:35:32.500 were just really to get you to hear that name for the simplest 00:35:32.500 --> 00:35:33.580 update. 00:35:33.580 --> 00:35:38.510 And Kalman's name for a more general update. 00:35:38.510 --> 00:35:40.680 Done. 00:35:40.680 --> 00:35:43.530 Oh, this is also to be done. 00:35:43.530 --> 00:35:45.810 Where is-- yeah. 00:35:45.810 --> 00:35:46.930 No-- up at the top. 00:35:49.750 --> 00:35:53.800 I seem to be into the applications here. 00:35:53.800 --> 00:35:58.270 I promised how to discover the formula. 00:35:58.270 --> 00:35:59.320 Maybe I'll never know. 00:36:03.070 --> 00:36:05.530 Because I'm really more interested in, 00:36:05.530 --> 00:36:06.650 what's it good for? 00:36:11.200 --> 00:36:16.400 Use number one, now, I'm backing up to the easy, easy question. 00:36:16.400 --> 00:36:23.645 Suppose I had-- and let me even change the matrix to A here. 00:36:23.645 --> 00:36:27.680 It makes it more realistic. 00:36:27.680 --> 00:36:30.350 I'm going to copy that here, fully 00:36:30.350 --> 00:36:34.040 on discovering the formula. 00:36:34.040 --> 00:36:37.880 For the moment, let's just use it. 00:36:37.880 --> 00:36:56.360 So I suppose that Au is b is solved for u. 00:36:56.360 --> 00:37:04.075 Now, now solve A plus-- 00:37:06.710 --> 00:37:14.470 or minus a rank-- let me do the rank 1, A minus uv transpose b. 00:37:18.230 --> 00:37:20.990 What's-- x. 00:37:20.990 --> 00:37:24.080 This is the problem that I really would like to solve. 00:37:26.590 --> 00:37:28.360 Let me just be sure I'm doing this right. 00:37:46.900 --> 00:37:51.190 It's similar to what we had in the Kalman situation. 00:37:51.190 --> 00:37:54.490 Suppose I've solved one problem. 00:37:54.490 --> 00:37:59.500 But now I perturb the matrix by rank 1. 00:37:59.500 --> 00:38:02.760 So I have a new problem with a new answer. 00:38:02.760 --> 00:38:04.550 And I want to get that answer quickly. 00:38:04.550 --> 00:38:05.050 Yeah? 00:38:05.050 --> 00:38:07.300 AUDIENCE: Are the u's related in those two lines? 00:38:07.300 --> 00:38:08.650 GILBERT STRANG: Oh, no. 00:38:08.650 --> 00:38:10.000 Thank you. 00:38:10.000 --> 00:38:11.560 Thank you very much. 00:38:11.560 --> 00:38:18.175 Let's call this guy z, or w, maybe w. 00:38:22.040 --> 00:38:23.180 So thank you. 00:38:23.180 --> 00:38:24.620 That's great. 00:38:24.620 --> 00:38:27.500 So that's what I've solved for w. 00:38:30.280 --> 00:38:33.520 In other words, I have found A inverse b. 00:38:33.520 --> 00:38:38.380 And I want to find the answer to that new question. 00:38:38.380 --> 00:38:41.310 So I've perturbed the matrix. 00:38:41.310 --> 00:38:44.290 It's the coefficient matrix in a linear system. 00:38:44.290 --> 00:38:48.030 And I just want to solve that linear system. 00:38:48.030 --> 00:38:52.080 Now, so if I didn't know anything about the formulas, 00:38:52.080 --> 00:38:55.540 I would have a new matrix here. 00:38:55.540 --> 00:39:05.190 It would take n cube steps to do elimination and get 00:39:05.190 --> 00:39:07.920 the new answer. 00:39:07.920 --> 00:39:12.820 But the point is to use the old answer. 00:39:12.820 --> 00:39:15.090 The point is to use the old answer. 00:39:15.090 --> 00:39:17.680 And now let me just say what this is. 00:39:17.680 --> 00:39:22.030 And it's problem three in this section. 00:39:30.040 --> 00:39:40.730 So in other words, we know about A. We've solved that one. 00:39:40.730 --> 00:39:44.607 So what I'm going to do, instead of solving a whole new problem, 00:39:44.607 --> 00:39:45.190 I'm going to-- 00:39:47.780 --> 00:39:49.510 so quickly is the idea. 00:39:52.950 --> 00:39:55.720 And here's the idea. 00:39:55.720 --> 00:39:56.650 I've solved that one. 00:39:56.650 --> 00:39:59.560 And I'm going to solve a second problem, 00:39:59.560 --> 00:40:06.190 also solve, A with the same matrix A-- 00:40:06.190 --> 00:40:10.210 oops-- A times what shall I call the unknown? 00:40:10.210 --> 00:40:12.940 There's z equal u. 00:40:17.060 --> 00:40:18.880 So this is my problem. 00:40:18.880 --> 00:40:21.190 I know the u and v transpose. 00:40:21.190 --> 00:40:25.360 But I don't really want to find the inverse of this matrix 00:40:25.360 --> 00:40:27.130 from scratch. 00:40:27.130 --> 00:40:29.920 So the idea is that if I suitably 00:40:29.920 --> 00:40:33.790 combine the solutions to the original problem, 00:40:33.790 --> 00:40:37.120 the original solution w, and the solution 00:40:37.120 --> 00:40:43.240 to this problem with the new guy u there, that somehow, 00:40:43.240 --> 00:40:49.200 by combining the w and the z, I'm going to get this answer x. 00:40:49.200 --> 00:40:51.130 That's where I'm headed. 00:40:51.130 --> 00:40:56.410 That's where I'm headed, that by figuring out w and z-- so 00:40:56.410 --> 00:40:59.030 do you see what I've done? 00:40:59.030 --> 00:41:01.355 With the matrix A, I've solved two problems. 00:41:04.820 --> 00:41:07.210 Does that take twice as long? 00:41:07.210 --> 00:41:13.695 If I have the same matrix A but different right-hand sides, b 00:41:13.695 --> 00:41:18.170 and u, if I factor A into Lu, all the hard work 00:41:18.170 --> 00:41:21.570 is done there on the left side. 00:41:21.570 --> 00:41:24.200 All the work is done in finding Lu. 00:41:24.200 --> 00:41:29.440 And then I just back substitute to find the second solution. 00:41:29.440 --> 00:41:33.315 This is so fundamental that I'm emphasizing it, 00:41:33.315 --> 00:41:36.800 that if you have multiple right-hand sides, 00:41:36.800 --> 00:41:40.580 you don't every time go back and work on the left side. 00:41:40.580 --> 00:41:43.670 The left side with the same matrix A, 00:41:43.670 --> 00:41:45.560 just would do the same stuff. 00:41:45.560 --> 00:41:49.070 The same rows and pivots and all that stuff 00:41:49.070 --> 00:41:53.620 will happen because it's the same A. You just attach-- 00:41:53.620 --> 00:41:57.270 you really just stick a second column-- 00:41:57.270 --> 00:42:03.980 really, the fast way to do it is A wz. 00:42:03.980 --> 00:42:10.280 The two solutions come from the b and the u. 00:42:10.280 --> 00:42:12.920 I've just put the two problems together 00:42:12.920 --> 00:42:18.290 to emphasize that the matrix A is the same. 00:42:18.290 --> 00:42:20.450 This is where most of the work goes. 00:42:20.450 --> 00:42:22.630 But it only goes there once. 00:42:22.630 --> 00:42:24.260 That's the point. 00:42:24.260 --> 00:42:28.060 Then finally, I'm supposed to-- 00:42:28.060 --> 00:42:30.160 the notes are supposed to tell me 00:42:30.160 --> 00:42:37.040 how to combine the two answers w and z to get the answer x. 00:42:37.040 --> 00:42:39.160 So let me write that down. 00:42:39.160 --> 00:42:47.930 And that will be use number one of 00:42:47.930 --> 00:42:51.140 the Sherman-Morrison-Woodbury formula. 00:42:51.140 --> 00:42:53.570 So I'm planning to just write it down. 00:42:53.570 --> 00:43:00.620 According to this, step two is the answer we want, 00:43:00.620 --> 00:43:02.120 here written x-- 00:43:02.120 --> 00:43:05.600 so I haven't used the same letters as in the notes-- 00:43:05.600 --> 00:43:08.660 is the original w-- 00:43:08.660 --> 00:43:12.590 good-- and then a change. 00:43:12.590 --> 00:43:16.490 Because I've changed the problem. 00:43:16.490 --> 00:43:18.770 So I'm going to change the answer. 00:43:18.770 --> 00:43:23.240 But the thing is, to get this answer, 00:43:23.240 --> 00:43:29.060 I have to divide by that determinant wherever it was, 00:43:29.060 --> 00:43:30.050 the-- 00:43:30.050 --> 00:43:34.826 oh, yeah, the-- 00:43:34.826 --> 00:43:35.840 I'm sorry. 00:43:35.840 --> 00:43:40.040 I'm using an A here, and I haven't given you the formula 00:43:40.040 --> 00:43:46.360 for A. So that'll have to wait. 00:43:49.250 --> 00:43:52.240 So I'll put determinant here. 00:43:52.240 --> 00:43:56.150 And Sherman, Morrison, Woodbury figured that out. 00:43:56.150 --> 00:43:58.150 And then-- oh, I'm sorry. 00:44:03.380 --> 00:44:05.210 We know what that is. 00:44:05.210 --> 00:44:10.360 We want 1 minus v transpose z. 00:44:10.360 --> 00:44:15.130 So it's the v from here, 1 minus v transpose. 00:44:15.130 --> 00:44:17.290 And it's the z there. 00:44:17.290 --> 00:44:18.610 Oh, yeah. 00:44:18.610 --> 00:44:20.740 This is good. 00:44:20.740 --> 00:44:25.270 So I'm going to get a formula that changes the solution 00:44:25.270 --> 00:44:30.880 w by a term that we recognize is coming from Sherman, Morrison, 00:44:30.880 --> 00:44:31.510 Woodbury. 00:44:31.510 --> 00:44:36.480 And that term in the numerator is a multiple of zx. 00:44:44.090 --> 00:44:47.790 So I'm using w-- 00:44:47.790 --> 00:44:50.746 yes-- w times-- 00:44:54.950 --> 00:45:01.830 sorry-- times x, which was-- 00:45:01.830 --> 00:45:02.820 I'm sorry. 00:45:02.820 --> 00:45:05.870 w is xv. 00:45:05.870 --> 00:45:08.620 So the v here, I'm just using-- 00:45:08.620 --> 00:45:15.930 the v is the same v. So v transpose z, 00:45:15.930 --> 00:45:18.030 I think that's got it. 00:45:18.030 --> 00:45:19.390 I think that's got it. 00:45:19.390 --> 00:45:23.990 So again, the point was to-- 00:45:23.990 --> 00:45:27.380 x doesn't appear here, because that's what I'm after-- 00:45:27.380 --> 00:45:31.820 is to use the w and the z, the w and the z. 00:45:31.820 --> 00:45:36.740 And I have to use, of course, the v. And I have to use the u. 00:45:36.740 --> 00:45:40.020 And that got used here. 00:45:40.020 --> 00:45:43.070 So everything that had to be used got used. 00:45:43.070 --> 00:45:44.840 And that was the answer. 00:45:47.420 --> 00:45:51.860 So that's the basic use, is, if I perturb 00:45:51.860 --> 00:45:57.500 the problem, the left-hand side, what's 00:45:57.500 --> 00:45:59.750 the change in the solution? 00:45:59.750 --> 00:46:03.290 Over here with least squares, it was the same. 00:46:03.290 --> 00:46:05.460 I perturbed the left-hand side. 00:46:05.460 --> 00:46:07.070 But because it was least squares, 00:46:07.070 --> 00:46:10.200 there was an A transpose A to deal with. 00:46:10.200 --> 00:46:13.130 So this was one level more difficult, 00:46:13.130 --> 00:46:15.500 because it involved A transpose A's. 00:46:15.500 --> 00:46:21.220 And here, it was just straightforward A. So, yes? 00:46:21.220 --> 00:46:21.720 Thanks. 00:46:21.720 --> 00:46:23.387 AUDIENCE: Would it be also a good idea-- 00:46:23.387 --> 00:46:29.040 so you have A minus uv transpose x plus b-- to write A minus Az 00:46:29.040 --> 00:46:30.992 b transpose x equals b? 00:46:30.992 --> 00:46:32.470 I'm backtracking on the left. 00:46:32.470 --> 00:46:34.250 GILBERT STRANG: I could probably do this other ways. 00:46:34.250 --> 00:46:34.650 Yeah. 00:46:34.650 --> 00:46:35.080 Yeah. 00:46:35.080 --> 00:46:35.580 Yeah. 00:46:35.580 --> 00:46:40.090 I hope you follow up on that and write it down. 00:46:42.610 --> 00:46:48.640 So maybe what I've done is what my goal was for today, first, 00:46:48.640 --> 00:46:53.790 to show you the formula for the inverse; second, 00:46:53.790 --> 00:46:58.470 to recognize that it has interesting importance 00:46:58.470 --> 00:47:01.260 that a rank k change in the matrix 00:47:01.260 --> 00:47:05.910 brings a rank k change in the inverse, and that we can-- 00:47:05.910 --> 00:47:09.210 the formula says to invert an n by n matrix, 00:47:09.210 --> 00:47:12.480 you can switch an inverted k by k matrix. 00:47:12.480 --> 00:47:15.610 And then I spoke about a couple of the applications. 00:47:15.610 --> 00:47:17.220 So the only thing I have not done 00:47:17.220 --> 00:47:21.150 is to give you the full Sherman-Morrison-Woodbury 00:47:21.150 --> 00:47:24.900 formula, the one with an A, the one that's up on the right. 00:47:24.900 --> 00:47:27.600 Can I do that finally? 00:47:27.600 --> 00:47:30.210 You can probably guess what it's going to be. 00:47:32.850 --> 00:47:37.710 Where is-- there's got to be one more blackboard. 00:47:37.710 --> 00:47:43.500 So here Sherman-Morrison-Woodbury. 00:47:43.500 --> 00:47:45.180 And what's the complete thing? 00:47:45.180 --> 00:47:49.050 It's A minus uv transpose inverse. 00:47:49.050 --> 00:47:52.390 So this is n by n. 00:47:52.390 --> 00:47:55.210 And it's an A now instead of an identity matrix. 00:47:55.210 --> 00:47:57.350 That's the difference. 00:47:57.350 --> 00:48:01.270 And this is n by k, k by n. 00:48:01.270 --> 00:48:04.150 So it's rank k perturbation. 00:48:04.150 --> 00:48:10.880 And start me out on the formula. 00:48:10.880 --> 00:48:13.260 What's the first thing I have to write? 00:48:13.260 --> 00:48:14.520 AUDIENCE: A inverse. 00:48:14.520 --> 00:48:16.450 GILBERT STRANG: A inverse. 00:48:16.450 --> 00:48:18.680 So I start with A inverse. 00:48:18.680 --> 00:48:20.770 And then I subtract something. 00:48:20.770 --> 00:48:26.380 And it's going to be a copy of this, except-- 00:48:26.380 --> 00:48:29.860 well, maybe it's just a perfect copy. 00:48:29.860 --> 00:48:33.280 Maybe I just need the u. 00:48:33.280 --> 00:48:35.170 I'm going from that left board. 00:48:35.170 --> 00:48:39.230 Now, Ik, what do I put it in there? 00:48:42.040 --> 00:48:44.980 I am going to have to look. 00:48:44.980 --> 00:48:51.300 So we're writing down now the formula, the full scale-- 00:48:51.300 --> 00:48:52.090 OK. 00:48:52.090 --> 00:48:52.840 Oh, all right. 00:48:58.300 --> 00:48:59.840 Here it is. 00:48:59.840 --> 00:49:03.800 There is an A inverse u. 00:49:03.800 --> 00:49:06.800 Now comes that whole thing inverted 00:49:06.800 --> 00:49:12.770 that you'll expect, I minus v transpose A inverse u, 00:49:12.770 --> 00:49:14.780 all inverse. 00:49:14.780 --> 00:49:19.560 And then there's another two pieces, v transpose A inverse. 00:49:22.140 --> 00:49:24.600 So I didn't look ahead to get it at all on one line. 00:49:24.600 --> 00:49:26.310 But do you see what-- 00:49:26.310 --> 00:49:30.020 this is the-- oh, that's probably-- is that-- 00:49:30.020 --> 00:49:30.520 yeah. 00:49:30.520 --> 00:49:32.460 That's the identity. 00:49:32.460 --> 00:49:35.220 Because we've got enough A inverses. 00:49:35.220 --> 00:49:39.900 I believe that's the final, ultimate formula of life 00:49:39.900 --> 00:49:45.850 here, that we've changed it by rank k. 00:49:45.850 --> 00:49:47.910 Here's the original inverse. 00:49:47.910 --> 00:49:51.325 And presumably, this is a rank k change. 00:49:54.140 --> 00:49:57.250 That will be a rank k change. 00:49:57.250 --> 00:50:03.100 And it only requires us to compute that inverse, where 00:50:03.100 --> 00:50:06.280 that's a k by k matrix. 00:50:06.280 --> 00:50:13.990 Thank you for allowing this, our 50 minutes of formulas. 00:50:13.990 --> 00:50:17.650 So that's really what that comes to. 00:50:17.650 --> 00:50:23.710 So that's section 4.1 of the notes with some applications. 00:50:23.710 --> 00:50:29.440 And we will move on to other circumstances of low rank. 00:50:29.440 --> 00:50:29.950 Good. 00:50:29.950 --> 00:50:31.820 Thank you.