WEBVTT 00:00:01.550 --> 00:00:03.920 The following content is provided under a Creative 00:00:03.920 --> 00:00:05.310 Commons license. 00:00:05.310 --> 00:00:07.520 Your support will help MIT OpenCourseWare 00:00:07.520 --> 00:00:11.610 continue to offer high quality educational resources for free. 00:00:11.610 --> 00:00:14.180 To make a donation, or to view additional materials 00:00:14.180 --> 00:00:18.140 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:18.140 --> 00:00:19.026 at ocw.mit.edu. 00:00:22.780 --> 00:00:24.680 PROFESSOR: So, are we ready to go? 00:00:24.680 --> 00:00:28.810 Any questions on 18065? 00:00:28.810 --> 00:00:34.540 So it will be, as I said before, a mixture of linear algebra 00:00:34.540 --> 00:00:42.430 and math questions, along with online using the material. 00:00:42.430 --> 00:00:43.240 OK. 00:00:43.240 --> 00:00:47.890 So I'm, in this first week or two, 00:00:47.890 --> 00:00:51.880 reviewing the highlights of linear algebra. 00:00:51.880 --> 00:00:59.240 And I've reached this point, to remember-- 00:00:59.240 --> 00:01:00.560 well, so we-- 00:01:00.560 --> 00:01:07.430 I just said two words about multiplying matrices 00:01:07.430 --> 00:01:12.770 by using column times row as a way to do it. 00:01:12.770 --> 00:01:16.040 And now, I want to illustrate that 00:01:16.040 --> 00:01:23.480 by the five key factorizations of matrices. 00:01:23.480 --> 00:01:24.320 OK. 00:01:24.320 --> 00:01:25.620 So what are they? 00:01:25.620 --> 00:01:28.070 And do you recognize them? 00:01:28.070 --> 00:01:29.780 Everybody uses those letters. 00:01:29.780 --> 00:01:33.710 In fact, some of those letters, like LU or QR, 00:01:33.710 --> 00:01:38.450 would be the most used MATLAB commands in linear algebra. 00:01:38.450 --> 00:01:42.020 So a A equal LU of, maybe-- 00:01:42.020 --> 00:01:44.360 say something I'll develop today-- 00:01:44.360 --> 00:01:46.310 but it's about elimination. 00:01:54.590 --> 00:01:57.300 Solving linear systems. 00:01:57.300 --> 00:02:00.080 So that's always the start of a linear algebra course. 00:02:00.080 --> 00:02:03.260 But it will go fast here. 00:02:03.260 --> 00:02:05.510 I just want to show you a different way 00:02:05.510 --> 00:02:09.289 they get to L times U-- lower triangular 00:02:09.289 --> 00:02:11.450 times upper triangular. 00:02:11.450 --> 00:02:13.880 Probably you've seen that-- 00:02:13.880 --> 00:02:16.610 those triangular matrices. 00:02:16.610 --> 00:02:18.330 So do you know what QR is? 00:02:18.330 --> 00:02:21.482 What's QR about? 00:02:21.482 --> 00:02:23.300 AUDIENCE: Least squares? 00:02:23.300 --> 00:02:26.462 PROFESSOR: Least squares is the big application, 00:02:26.462 --> 00:02:28.110 the factorization. 00:02:28.110 --> 00:02:31.335 So what kind of a matrix gets that letter Q? 00:02:31.335 --> 00:02:32.210 AUDIENCE: Orthogonal? 00:02:32.210 --> 00:02:33.127 PROFESSOR: Orthogonal. 00:02:33.127 --> 00:02:35.190 The columns are orthogonal. 00:02:35.190 --> 00:02:37.220 Often orthonormal. 00:02:37.220 --> 00:02:40.320 So orthogonal means they're perpendicular to each other. 00:02:40.320 --> 00:02:44.090 And orthonormal means they're unit vectors. 00:02:44.090 --> 00:02:52.710 So that is-- so Q often represents a matrix 00:02:52.710 --> 00:02:55.830 with orthonormal columns. 00:02:55.830 --> 00:02:58.950 So you-- we could say Gram-Schmidt, 00:02:58.950 --> 00:03:06.300 if you want to remember a couple of old timers 00:03:06.300 --> 00:03:13.440 whose algorithm produces Q and R. 00:03:13.440 --> 00:03:14.640 How about this one? 00:03:14.640 --> 00:03:18.360 This is really a central one in math-- 00:03:18.360 --> 00:03:21.750 pure math, applied math, everywhere-- applications. 00:03:21.750 --> 00:03:24.030 So S stands for symmetric. 00:03:24.030 --> 00:03:26.970 So this is a special factorization 00:03:26.970 --> 00:03:29.340 for symmetric matrices. 00:03:29.340 --> 00:03:31.020 And you can see that it's symmetric. 00:03:31.020 --> 00:03:35.460 This lambda is the diagonal eigenvalue matrix-- 00:03:35.460 --> 00:03:38.850 always lambda for eigenvalues. 00:03:38.850 --> 00:03:41.010 Q is like that Q-- 00:03:41.010 --> 00:03:43.220 different Q, of course. 00:03:43.220 --> 00:03:46.920 That Q you can find just straight forward 00:03:46.920 --> 00:03:48.690 from Gram-Schmidt. 00:03:48.690 --> 00:03:52.110 This Q has the eigenvectors. 00:03:52.110 --> 00:03:55.630 So you don't find eigenvectors without some extra work. 00:03:55.630 --> 00:03:56.130 OK. 00:03:56.130 --> 00:03:58.270 So that's eigenvectors. 00:03:58.270 --> 00:03:58.770 Yeah. 00:03:58.770 --> 00:04:04.380 So that would be worth expanding. 00:04:04.380 --> 00:04:08.670 So here are the eigenvectors of the matrix S-- 00:04:08.670 --> 00:04:11.340 n of them, normalized. 00:04:11.340 --> 00:04:15.540 Here are the eigenvalues lambda 1 to lambda n. 00:04:15.540 --> 00:04:19.170 And here are the eigenvectors now transposed. 00:04:25.600 --> 00:04:29.170 So remind me of the great fact about-- 00:04:29.170 --> 00:04:32.560 two facts, I guess-- one fact about the eigenvalues 00:04:32.560 --> 00:04:34.570 and one fact about eigenvectors. 00:04:34.570 --> 00:04:41.650 This is an important fact statement in linear algebra. 00:04:41.650 --> 00:04:44.120 What do we know about the eigenvectors? 00:04:44.120 --> 00:04:46.780 Oh well, I guess I've given it away. 00:04:46.780 --> 00:04:50.310 The eigenvectors are orthogonal. 00:04:50.310 --> 00:04:52.230 That's very important. 00:04:52.230 --> 00:04:55.310 Makes some matrices-- well, they're beautiful matrices. 00:04:55.310 --> 00:04:59.110 They're the kings of linear algebra. 00:04:59.110 --> 00:05:01.320 Qs are the queens, in my opinion. 00:05:01.320 --> 00:05:03.210 Orthogonal matrices are the queens, 00:05:03.210 --> 00:05:06.270 and symmetric matrices are the kings. 00:05:06.270 --> 00:05:12.630 So these are orthonormal eigenvectors. 00:05:12.630 --> 00:05:14.400 And the key point-- 00:05:14.400 --> 00:05:16.860 an important point that's implicit 00:05:16.860 --> 00:05:19.140 here-- is, there are n of them. 00:05:19.140 --> 00:05:20.430 There is a complete set. 00:05:20.430 --> 00:05:23.280 The matrix can be diagonalized. 00:05:23.280 --> 00:05:28.050 And those-- well, what's special about the eigenvalues? 00:05:28.050 --> 00:05:32.250 Other matrices could be Q lambda Q transpose. 00:05:32.250 --> 00:05:35.730 But symmetric matrices are something additional 00:05:35.730 --> 00:05:36.375 about lambda. 00:05:36.375 --> 00:05:37.500 AUDIENCE: They're all real? 00:05:37.500 --> 00:05:39.330 PROFESSOR: They're all real. 00:05:39.330 --> 00:05:41.530 So eigenvalues are real. 00:05:41.530 --> 00:05:45.330 And eigenvectors are orthonormal-- 00:05:45.330 --> 00:05:48.150 can be chosen orthonormal-- can be chosen, 00:05:48.150 --> 00:05:49.380 I guess I have to say. 00:05:49.380 --> 00:05:50.040 OK. 00:05:50.040 --> 00:05:50.610 Good. 00:05:50.610 --> 00:05:51.110 Good. 00:05:51.110 --> 00:05:51.630 Good. 00:05:51.630 --> 00:05:54.960 Oh, now maybe I'll use that as an example 00:05:54.960 --> 00:05:57.340 of matrix multiplication. 00:05:57.340 --> 00:05:59.160 So let me just do that here. 00:05:59.160 --> 00:06:03.350 Simple matrix multiplication, but it makes the point. 00:06:03.350 --> 00:06:08.100 So Q lambda Q transpose. 00:06:10.870 --> 00:06:11.950 OK. 00:06:11.950 --> 00:06:15.730 Well, what was my point about matrix multiplication? 00:06:15.730 --> 00:06:20.140 Let me-- it really involved two matrices. 00:06:20.140 --> 00:06:21.800 Here, I unfortunately have three. 00:06:21.800 --> 00:06:25.240 So I'm going to have to squeeze lambda in with one of the Qs, 00:06:25.240 --> 00:06:28.840 to see it nicely as two matrices. 00:06:28.840 --> 00:06:30.560 Shall I just do that? 00:06:30.560 --> 00:06:32.050 Yeah. 00:06:32.050 --> 00:06:33.880 Now I've made it two matrices. 00:06:33.880 --> 00:06:35.350 That was easy. 00:06:35.350 --> 00:06:36.160 OK. 00:06:36.160 --> 00:06:38.590 Now what's the rule? 00:06:38.590 --> 00:06:43.000 In the first notes, this was A and this was B. 00:06:43.000 --> 00:06:45.100 And when you multiply two matrices, 00:06:45.100 --> 00:06:54.430 the rule is, this is columns of Q lambda times 00:06:54.430 --> 00:06:59.610 rows of Q transpose. 00:06:59.610 --> 00:07:02.060 I'm multiplying columns by rows. 00:07:02.060 --> 00:07:07.330 And so it's a column vector times a row vector, 00:07:07.330 --> 00:07:09.820 and that gives us a matrix. 00:07:09.820 --> 00:07:13.450 So each-- and it's a special matrix. 00:07:13.450 --> 00:07:14.750 So this is our column. 00:07:14.750 --> 00:07:16.330 This is a row. 00:07:16.330 --> 00:07:22.510 And when I multiply n by 1 times 1 by n, I get an n by n matrix. 00:07:22.510 --> 00:07:24.340 And it's pretty special. 00:07:24.340 --> 00:07:27.250 And what is the special fact about I'm 00:07:27.250 --> 00:07:29.530 sort of recalling from last time. 00:07:29.530 --> 00:07:32.890 What's special about a column times a row? 00:07:32.890 --> 00:07:34.630 It's rank is special. 00:07:34.630 --> 00:07:36.440 It's rank is 1. 00:07:36.440 --> 00:07:42.910 It's column space-- well, the only column around is this one. 00:07:42.910 --> 00:07:46.630 So all columns are multiples of this guy. 00:07:46.630 --> 00:07:50.410 All rows are multiples of this guy, 00:07:50.410 --> 00:07:52.450 as we could see from an example. 00:07:52.450 --> 00:07:54.160 Shall I just do an example? 00:07:54.160 --> 00:08:00.490 1, 2 times 3, 4, to take a random example. 00:08:00.490 --> 00:08:05.020 So that would give us 3, 4, 6, 8. 00:08:05.020 --> 00:08:10.630 And sure enough, the columns are multiples of 1, 2. 00:08:10.630 --> 00:08:13.270 The rows are multiples of 3, 4. 00:08:13.270 --> 00:08:16.610 And the rank is 1. 00:08:16.610 --> 00:08:17.110 OK. 00:08:17.110 --> 00:08:19.360 So those are the building blocks. 00:08:19.360 --> 00:08:21.850 Now I want to build something. 00:08:21.850 --> 00:08:22.720 So here we go. 00:08:25.280 --> 00:08:27.310 So this is a sum of rank 1. 00:08:27.310 --> 00:08:33.100 Sum of rank 1. 00:08:33.100 --> 00:08:36.970 Sum of column times row. 00:08:36.970 --> 00:08:40.210 So I take column 1 times row 1. 00:08:40.210 --> 00:08:42.460 That's my first thing in the sum. 00:08:42.460 --> 00:08:43.840 So column 1 of that. 00:08:43.840 --> 00:08:46.000 So you see I had to sneak the lambda 00:08:46.000 --> 00:08:50.660 to have just two factors. 00:08:50.660 --> 00:08:52.840 So what's column 1 of Q lambda? 00:08:52.840 --> 00:08:54.830 That's a good question. 00:08:54.830 --> 00:08:59.080 So column 1 of Q is Q1-- 00:08:59.080 --> 00:09:01.120 the first eigenvector. 00:09:01.120 --> 00:09:05.400 But now, I am multiplying by this diagonal matrix. 00:09:05.400 --> 00:09:09.360 Do you see in your mind what's the first column of Q lambda? 00:09:12.370 --> 00:09:15.290 Just think about that a second. 00:09:15.290 --> 00:09:23.200 So here's-- we can steal any little corner for a matrix. 00:09:23.200 --> 00:09:27.910 So here's q1, and the rest of the columns. 00:09:27.910 --> 00:09:31.210 And then here is lambda 1. 00:09:31.210 --> 00:09:32.440 And that's Q lambda. 00:09:32.440 --> 00:09:34.600 So I'm putting those together. 00:09:34.600 --> 00:09:38.480 And I'm asking, what's the first column of the answer? 00:09:38.480 --> 00:09:41.510 Can you see how that works? 00:09:41.510 --> 00:09:42.880 AUDIENCE: q1 lambda 1? 00:09:42.880 --> 00:09:43.983 PROFESSOR: It's... sorry?. 00:09:43.983 --> 00:09:44.900 AUDIENCE: q1 lambda 1? 00:09:44.900 --> 00:09:45.940 PROFESSOR: q1 lambda 1. 00:09:45.940 --> 00:09:46.900 Exactly. 00:09:46.900 --> 00:09:49.660 That lambda 1 will multiply q1. 00:09:49.660 --> 00:09:52.250 These other lambdas multiplied later columns. 00:09:52.250 --> 00:09:54.980 So the first column is lambda 1 times q1. 00:09:54.980 --> 00:09:55.480 Right. 00:09:55.480 --> 00:10:00.880 So it's the first guy, lambda 1, q1. 00:10:00.880 --> 00:10:06.820 And then the first row of this will be q1 transpose. 00:10:06.820 --> 00:10:11.770 That's the first guy in our sum of n things. 00:10:11.770 --> 00:10:14.710 And let me put the next one and the last one. 00:10:14.710 --> 00:10:20.200 Lambda 2, q2, q2 transpose. 00:10:20.200 --> 00:10:25.960 And lambda n, qn, qn transpose. 00:10:29.820 --> 00:10:33.320 That's really a nice right way to write 00:10:33.320 --> 00:10:39.300 to breakup the product, q lambda q transpose. 00:10:39.300 --> 00:10:41.650 This is called the spectral theorem. 00:10:41.650 --> 00:10:47.700 So that's the symmetric-- that's S. That's S there, is that. 00:10:47.700 --> 00:10:53.370 So that's-- we've broken up S into rank 1 pieces. 00:10:53.370 --> 00:10:57.750 That's like a constant theme. 00:10:57.750 --> 00:11:01.290 And these rank 1 pieces are quite special because they're 00:11:01.290 --> 00:11:02.700 symmetric. 00:11:02.700 --> 00:11:05.790 Q1q1 transpose will be symmetric. 00:11:05.790 --> 00:11:09.780 And oh, so can we-- 00:11:09.780 --> 00:11:14.280 so let's just-- I follow the rule for multiplying matrices. 00:11:14.280 --> 00:11:19.350 But maybe I could just check that it's the right thing-- 00:11:19.350 --> 00:11:22.350 that it came out right. 00:11:22.350 --> 00:11:23.940 So what do I mean by checking? 00:11:23.940 --> 00:11:28.350 I guess I'll just check about S times q1. 00:11:28.350 --> 00:11:33.810 So look at S-- 00:11:33.810 --> 00:11:38.760 this thing-- times the first eigenvector, and what do I get? 00:11:38.760 --> 00:11:39.990 OK. 00:11:39.990 --> 00:11:43.170 So you'll like this. 00:11:43.170 --> 00:11:47.400 I've split up S into a sum of rank 1 pieces. 00:11:47.400 --> 00:11:48.870 And that splitting is-- 00:11:48.870 --> 00:11:52.620 you see it all over. 00:11:52.620 --> 00:11:56.370 It's really showing you what the pieces of the symmetric matrix 00:11:56.370 --> 00:11:57.150 are. 00:11:57.150 --> 00:11:59.010 And now I'm just going to check that that's 00:11:59.010 --> 00:12:04.530 a correct formula for S, so I'll multiply it by q1. 00:12:04.530 --> 00:12:06.780 And I'm hoping to get the right thing. 00:12:06.780 --> 00:12:08.790 And what do I actually get? 00:12:08.790 --> 00:12:14.160 If I multiply this whole business times q1, 00:12:14.160 --> 00:12:20.310 I get lambda 1, q1, q1 transpose-- 00:12:20.310 --> 00:12:23.400 that's the first guy times my q1-- 00:12:23.400 --> 00:12:26.980 plus-- right? 00:12:26.980 --> 00:12:31.260 I'm multiplying S by q1, and this first term gave me that. 00:12:31.260 --> 00:12:33.010 And what does the next term give me? 00:12:33.010 --> 00:12:37.010 Put me out of my misery here. 00:12:37.010 --> 00:12:40.310 I'm looking for this thing to simplify like mad. 00:12:40.310 --> 00:12:40.830 OK. 00:12:40.830 --> 00:12:42.150 So what's the second term? 00:12:42.150 --> 00:12:47.833 When I multiply this guy by q1, what do I get? 00:12:47.833 --> 00:12:48.760 AUDIENCE: Zero. 00:12:48.760 --> 00:12:49.470 PROFESSOR: Zero. 00:12:49.470 --> 00:12:50.580 That's right. 00:12:50.580 --> 00:12:52.000 That's what we want. 00:12:52.000 --> 00:12:56.640 And when I multiply the last guy by q1, 00:12:56.640 --> 00:13:01.090 I get zero, because the qs are orthogonal. 00:13:01.090 --> 00:13:03.060 So this is all I get. 00:13:03.060 --> 00:13:05.790 And then-- so, I don't need this plus anymore. 00:13:05.790 --> 00:13:07.020 That's it. 00:13:07.020 --> 00:13:10.290 And then what can I do to improve 00:13:10.290 --> 00:13:18.060 that little somewhat repetitive formula for the answer? 00:13:18.060 --> 00:13:19.920 What do I want to do finally? 00:13:19.920 --> 00:13:23.430 I want to remember that the qs are normalized. 00:13:23.430 --> 00:13:25.090 They're unit vectors. 00:13:25.090 --> 00:13:26.652 So what does that tell me here? 00:13:26.652 --> 00:13:27.610 AUDIENCE: Q1 transpose. 00:13:27.610 --> 00:13:30.180 PROFESSOR: Q1 transpose times q1. 00:13:30.180 --> 00:13:34.170 This is just 1. 00:13:34.170 --> 00:13:38.210 It's-- that's what normalized means-- 00:13:38.210 --> 00:13:41.630 that the length squared is the length-- 00:13:41.630 --> 00:13:44.880 the length of the vector squared. 00:13:44.880 --> 00:13:45.900 And it's 1. 00:13:45.900 --> 00:13:48.750 So I can cancel that term. 00:13:48.750 --> 00:13:50.370 And I'm getting the right answer. 00:13:50.370 --> 00:13:53.850 That's all-- that's all this was about. 00:13:53.850 --> 00:13:59.600 I was just checking and wanted to see how it would fall out. 00:13:59.600 --> 00:14:05.100 And it falls right out that this formula 00:14:05.100 --> 00:14:08.250 is the correct matrix S, because it's 00:14:08.250 --> 00:14:11.210 got the right eigenvectors, qs. 00:14:11.210 --> 00:14:13.460 And it's got the right eigenvalues lambdas. 00:14:13.460 --> 00:14:18.390 So it's gotta be the right matrix S. Is that OK? 00:14:18.390 --> 00:14:24.830 That's like a first example to see how this-- 00:14:28.490 --> 00:14:31.680 splitting into rank 1s-- 00:14:31.680 --> 00:14:35.400 gives you back what you expect easily enough. 00:14:35.400 --> 00:14:38.200 It gives you the information you expect. 00:14:38.200 --> 00:14:38.700 OK. 00:14:38.700 --> 00:14:43.940 So that's the symmetric eigenvalue picture 00:14:43.940 --> 00:14:45.470 for symmetric matrices. 00:14:45.470 --> 00:14:47.360 And we'll see it again. 00:14:47.360 --> 00:14:54.410 It's-- well, all five of these are big, are important. 00:14:54.410 --> 00:14:56.660 I don't know if you know this one, 00:14:56.660 --> 00:15:02.090 but it's going to be a foundational factorization 00:15:02.090 --> 00:15:05.220 for this course and for all of data science. 00:15:05.220 --> 00:15:07.130 Do you know its name? 00:15:07.130 --> 00:15:09.920 So, what does it mean, first of all? 00:15:09.920 --> 00:15:12.950 Just a comment on this, and then we'll 00:15:12.950 --> 00:15:15.450 save it for a couple of weeks. 00:15:15.450 --> 00:15:19.130 So this view is actually an orthogonal matrix. 00:15:19.130 --> 00:15:23.870 And so is V. So it has two orthogonal matrices. 00:15:23.870 --> 00:15:27.230 So that's why people call them U and V 00:15:27.230 --> 00:15:33.470 rather than Q1 and Q2, which was too much to get subscript. 00:15:33.470 --> 00:15:37.210 So orthogonal times diagonal times orthogonal. 00:15:37.210 --> 00:15:45.880 And we'd say, orthogonal, diagonal, orthogonal. 00:15:45.880 --> 00:15:51.860 And 18.06 would, as of now, reach this topic 00:15:51.860 --> 00:15:54.710 because it's jumped up in importance. 00:15:54.710 --> 00:15:55.640 And it's called? 00:15:55.640 --> 00:15:57.265 AUDIENCE: Singular value decomposition. 00:15:57.265 --> 00:16:00.210 PROFESSOR: Singular value decomposition. 00:16:00.210 --> 00:16:04.910 Well, those are long words, so everybody calls it the SVD-- 00:16:04.910 --> 00:16:07.470 the singular value decomposition. 00:16:07.470 --> 00:16:15.210 The point is it works for every matrix-- 00:16:15.210 --> 00:16:16.650 rectangular matrices. 00:16:16.650 --> 00:16:18.690 There's no issue of, does it have 00:16:18.690 --> 00:16:21.630 enough eigenvectors or not? 00:16:21.630 --> 00:16:23.730 That's an issue here. 00:16:23.730 --> 00:16:25.320 Well, it's an issue here. 00:16:25.320 --> 00:16:29.910 Not every matrix has got enough eigenvectors to make that work. 00:16:29.910 --> 00:16:35.820 Every matrix that one works because instead 00:16:35.820 --> 00:16:39.780 of one set of vectors it's got two 00:16:39.780 --> 00:16:45.330 matrices-- two different sets of singular vectors. 00:16:45.330 --> 00:16:47.470 Oh, we'll see that. 00:16:47.470 --> 00:16:50.900 That's important. 00:16:50.900 --> 00:16:51.420 OK. 00:16:51.420 --> 00:16:55.490 So that's really a quick overview 00:16:55.490 --> 00:16:59.210 of fundamental factorizations. 00:16:59.210 --> 00:17:03.970 And I'd like to say just another word about elimination, 00:17:03.970 --> 00:17:09.540 A equal LU, and then we'll leave it alone. 00:17:09.540 --> 00:17:11.609 So elimination. 00:17:11.609 --> 00:17:12.109 Yeah. 00:17:12.109 --> 00:17:20.079 Do you remember that first beginning of linear algebra, 00:17:20.079 --> 00:17:22.290 when you're solving ax equal b? 00:17:22.290 --> 00:17:24.460 You do these row operations. 00:17:24.460 --> 00:17:28.990 Can I just-- what I want to say is, all those row operations 00:17:28.990 --> 00:17:35.330 that you do are perfectly expressed by L times U. 00:17:35.330 --> 00:17:40.000 And so that's a key point in 18.06, 00:17:40.000 --> 00:17:42.670 but I have a different way to look at it. 00:17:42.670 --> 00:17:44.170 So that's what I wanted to show you. 00:17:44.170 --> 00:17:45.050 I have a-- 00:17:45.050 --> 00:17:51.430 I want to show you a sum of rank 1's, or row times column. 00:17:51.430 --> 00:17:53.020 It fits in today. 00:17:53.020 --> 00:18:02.230 So I'd just like to see, why does a matrix invertible-- 00:18:02.230 --> 00:18:05.780 this is a square matrix, now invertible. 00:18:05.780 --> 00:18:11.260 And it factors-- if all goes well with elimination 00:18:11.260 --> 00:18:13.270 and the pivots are non-zero-- 00:18:13.270 --> 00:18:17.140 it factors into lower triangular times upper triangular. 00:18:17.140 --> 00:18:20.260 So that's a key step that is-- 00:18:20.260 --> 00:18:23.500 that MATLAB would do with a lu of A-- 00:18:23.500 --> 00:18:24.970 would produce those two factors. 00:18:28.480 --> 00:18:32.890 Now I want to do them in a column times row 00:18:32.890 --> 00:18:39.760 way, which I just realized late, was a neat way to do it. 00:18:39.760 --> 00:18:47.850 So can I take a matrix and do elimination? 00:18:47.850 --> 00:18:51.690 How big a matrix shall I take? 00:18:51.690 --> 00:18:55.140 2 by 2 [INAUDIBLE]. 00:18:55.140 --> 00:18:59.250 3 by 3 Somebody's not convinced totally by 2 by 2 00:18:59.250 --> 00:19:01.860 Let me do it 2 by 2 and then if you want-- if you really 00:19:01.860 --> 00:19:05.140 want a 3 by 3 do it. 00:19:05.140 --> 00:19:05.640 OK. 00:19:05.640 --> 00:19:11.930 Here's 2 by 2 2, 4, 3, 7. 00:19:11.930 --> 00:19:13.760 How's that? 00:19:13.760 --> 00:19:14.260 OK. 00:19:17.110 --> 00:19:18.650 So what is-- yeah. 00:19:18.650 --> 00:19:23.900 So let's remember what elimination does. 00:19:23.900 --> 00:19:31.430 It subtracts a multiple of that row from that one, 00:19:31.430 --> 00:19:34.420 to get to 2, 3. 00:19:34.420 --> 00:19:35.780 And the multiple is 2. 00:19:35.780 --> 00:19:37.910 So it knocks out the 4. 00:19:37.910 --> 00:19:39.140 Two 3's are 6. 00:19:39.140 --> 00:19:40.820 So it leaves a 1. 00:19:40.820 --> 00:19:42.590 And we're-- oh, yeah. 00:19:42.590 --> 00:19:44.930 Thanks for allowing me to do 2 by 2. 00:19:44.930 --> 00:19:46.520 I've already done it now. 00:19:46.520 --> 00:19:50.720 I've reached U. So here's A. And here's U-- 00:19:50.720 --> 00:19:54.290 the upper triangular guy with the pivots on the diagonal. 00:19:56.810 --> 00:20:01.920 And then the question is, express 00:20:01.920 --> 00:20:05.910 that step in matrix language. 00:20:05.910 --> 00:20:10.440 And the right answer is l times U. So the right answer 00:20:10.440 --> 00:20:14.970 is that this A is-- 00:20:14.970 --> 00:20:18.660 so I'll just erase that letter U-- 00:20:18.660 --> 00:20:23.430 is l times U. So what is l? 00:20:23.430 --> 00:20:26.130 L is the lower triangular guy. 00:20:26.130 --> 00:20:31.470 And it has there the number that you used here. 00:20:31.470 --> 00:20:32.490 And what was that? 00:20:32.490 --> 00:20:35.790 I subtracted 2 of that row from this. 00:20:35.790 --> 00:20:37.450 So I want a 2 there. 00:20:44.790 --> 00:20:46.170 So that would be-- 00:20:46.170 --> 00:20:48.090 I would call that a multiplier. 00:20:48.090 --> 00:20:50.850 I multiplied row 1 by 2. 00:20:50.850 --> 00:20:53.280 Subtracted to get that 0. 00:20:53.280 --> 00:20:57.410 And for a 2 by 2 example, I was finished. 00:20:57.410 --> 00:20:57.910 OK. 00:20:57.910 --> 00:21:00.810 Now I want to see this-- 00:21:00.810 --> 00:21:02.770 so there is l times U. It happened. 00:21:02.770 --> 00:21:04.770 Right. 00:21:04.770 --> 00:21:10.470 I would like to see how l times U comes out of this row-- 00:21:10.470 --> 00:21:12.590 column times row. 00:21:12.590 --> 00:21:14.370 So let me start-- 00:21:14.370 --> 00:21:15.980 let me think again. 00:21:20.140 --> 00:21:23.490 So really the point of elimination-- 00:21:23.490 --> 00:21:26.220 why did we do this in the first place? 00:21:26.220 --> 00:21:30.313 Because here we had two coupled equations. 00:21:30.313 --> 00:21:31.480 There were coupled together. 00:21:31.480 --> 00:21:33.660 We couldn't solve them instantly. 00:21:33.660 --> 00:21:38.250 That step of elimination reduced me to-- 00:21:38.250 --> 00:21:39.840 down here in this corner-- 00:21:39.840 --> 00:21:44.820 one equation, that I've eliminated the first unknown x 00:21:44.820 --> 00:21:47.440 from the second equation. 00:21:47.440 --> 00:21:52.140 So the second equation is 0x plus 1y equal right hand side. 00:21:52.140 --> 00:21:54.800 And I solve it immediately. 00:21:54.800 --> 00:21:55.800 OK. 00:21:55.800 --> 00:22:01.680 So how did I get to that 1 by 1 problem, 00:22:01.680 --> 00:22:03.480 with these guys removed? 00:22:03.480 --> 00:22:06.822 Well-- yeah. 00:22:06.822 --> 00:22:09.130 I'll just write-- can I write here the-- 00:22:09.130 --> 00:22:13.500 my parallel way to think of it? 00:22:13.500 --> 00:22:15.960 2 by 2 is pretty small, I admit. 00:22:15.960 --> 00:22:16.620 OK. 00:22:16.620 --> 00:22:19.260 So I start with 2, 3, 4, 7. 00:22:22.090 --> 00:22:24.790 I want to split it into-- 00:22:24.790 --> 00:22:30.770 I want to get the first row and column in one piece. 00:22:30.770 --> 00:22:32.780 Something goes there. 00:22:32.780 --> 00:22:38.610 And the other piece is something there. 00:22:38.610 --> 00:22:40.350 OK. 00:22:40.350 --> 00:22:42.750 That's what elimination has done. 00:22:42.750 --> 00:22:45.210 It's taken the original matrix. 00:22:45.210 --> 00:22:47.440 It's split-- these are both rank 1. 00:22:54.080 --> 00:22:56.030 So let's just-- first of all, you 00:22:56.030 --> 00:23:01.280 could tell me what goes in that blank space in the first rank 1 00:23:01.280 --> 00:23:01.790 matrix. 00:23:01.790 --> 00:23:05.120 So what-- can I say this in words? 00:23:05.120 --> 00:23:11.630 The first stage of elimination pulls off from A-- 00:23:11.630 --> 00:23:14.090 so A is some big matrix. 00:23:14.090 --> 00:23:19.070 It pulls off from A. It takes account of the first column 00:23:19.070 --> 00:23:20.540 and row. 00:23:20.540 --> 00:23:25.910 So it writes A as-- 00:23:25.910 --> 00:23:33.560 here we go-- as a first column, say column 00:23:33.560 --> 00:23:40.100 1, row 1, plus the easy part. 00:23:40.100 --> 00:23:45.740 The easy part will be a matrix with all zeros there-- 00:23:45.740 --> 00:23:47.090 all zeros there. 00:23:47.090 --> 00:23:49.100 And here I have a 2. 00:23:49.100 --> 00:23:51.740 Can I call it a 2? 00:23:51.740 --> 00:23:56.690 This is my way now to think about what 00:23:56.690 --> 00:23:58.670 elimination is really doing. 00:23:58.670 --> 00:24:01.400 It's starting with an n by n matrix. 00:24:01.400 --> 00:24:04.580 It's pulling off a rank 1 matrix, 00:24:04.580 --> 00:24:09.140 which gets that column and that row correct. 00:24:09.140 --> 00:24:11.660 And it gets whatever it has in here. 00:24:11.660 --> 00:24:16.400 And then the rest of what's in there is A2. 00:24:16.400 --> 00:24:19.520 Do you see that we've done that here? 00:24:19.520 --> 00:24:23.270 The first step got the first row and column correct. 00:24:23.270 --> 00:24:25.400 And if it's rank 1, what number goes there? 00:24:25.400 --> 00:24:26.140 AUDIENCE: 6. 00:24:26.140 --> 00:24:27.928 PROFESSOR: 6. 00:24:27.928 --> 00:24:29.269 6 for there. 00:24:29.269 --> 00:24:31.340 And then this is the rest. 00:24:31.340 --> 00:24:35.900 This is what we have, one size smaller to work on. 00:24:35.900 --> 00:24:38.380 And it looks like it was 7. 00:24:38.380 --> 00:24:39.600 6 has been used. 00:24:39.600 --> 00:24:40.340 So it's a 1. 00:24:43.520 --> 00:24:52.130 That's really-- I want to think of this rank 1 matrix 00:24:52.130 --> 00:24:58.610 as the first column of l times the first row of u. 00:24:58.610 --> 00:25:01.670 And then this guy is the second column 00:25:01.670 --> 00:25:05.110 of l times the second row of u. 00:25:11.850 --> 00:25:13.270 OK. 00:25:13.270 --> 00:25:17.820 I haven't presented as proof in a class before. 00:25:20.380 --> 00:25:23.560 And for 2 by 2, it's looking like overkill to me. 00:25:23.560 --> 00:25:24.610 I mean, why? 00:25:24.610 --> 00:25:26.710 You don't have to do all that deep thinking 00:25:26.710 --> 00:25:29.540 to get the pieces. 00:25:29.540 --> 00:25:35.230 But my idea is that it gives the breakdown. 00:25:35.230 --> 00:25:39.370 And this, of course, is, by our column times 00:25:39.370 --> 00:25:43.140 row rule, that's LU. 00:25:43.140 --> 00:25:48.510 So we're starting with A, and we're breaking it up into LU, 00:25:48.510 --> 00:25:49.890 where lu-- 00:25:49.890 --> 00:25:52.140 the first piece of lu-- 00:25:52.140 --> 00:25:55.920 is the first column times row. 00:25:55.920 --> 00:26:01.710 And then the next pieces are the rest of the matrix. 00:26:01.710 --> 00:26:06.240 And those get broken down-- the next stage of elimination would 00:26:06.240 --> 00:26:07.020 break-- 00:26:07.020 --> 00:26:10.650 if I had a 3 by 3, this stage peeled off 00:26:10.650 --> 00:26:12.540 the first column and row-- 00:26:12.540 --> 00:26:15.790 then the next stage would peel off the second-- 00:26:15.790 --> 00:26:17.970 the new second column and row. 00:26:17.970 --> 00:26:21.370 And the third stage would have the third column and row-- 00:26:21.370 --> 00:26:25.500 just the last pivot does this make any sense to you? 00:26:25.500 --> 00:26:30.000 You could email me and say it's not that great. 00:26:30.000 --> 00:26:32.420 But I think it's-- 00:26:32.420 --> 00:26:39.410 to see that the final result of elimination is l times u is-- 00:26:39.410 --> 00:26:45.720 there's a little magic in seeing what you're doing. 00:26:45.720 --> 00:26:48.380 And I think this is a way to see what you're doing-- 00:26:48.380 --> 00:26:52.970 that you're peeling off a first part 00:26:52.970 --> 00:26:55.790 to leave a second part like that. 00:26:55.790 --> 00:26:58.250 Then the second part, you would peel off 00:26:58.250 --> 00:27:01.970 the second column times the second row, 00:27:01.970 --> 00:27:05.030 maybe divide by the pivot to make it correct. 00:27:05.030 --> 00:27:09.020 And that would put something in the rest of the box. 00:27:09.020 --> 00:27:14.640 And then A3 would be the rest of that box. 00:27:14.640 --> 00:27:15.420 OK. 00:27:15.420 --> 00:27:17.430 I'm stopping here. 00:27:17.430 --> 00:27:23.040 I'm glad you let me do 2 by 2, since I see that 3 by 3 00:27:23.040 --> 00:27:24.840 would have ruined the day. 00:27:24.840 --> 00:27:25.370 Yeah. 00:27:25.370 --> 00:27:26.245 OK. 00:27:26.245 --> 00:27:30.450 A question, or let me pause for a minute. 00:27:30.450 --> 00:27:35.770 So I've talked about these factorizations. 00:27:35.770 --> 00:27:38.200 This one we won't see again. 00:27:38.200 --> 00:27:41.440 This one we will see, big time. 00:27:41.440 --> 00:27:43.510 And this one we will. 00:27:43.510 --> 00:27:44.770 And this one we will. 00:27:44.770 --> 00:27:46.270 Yeah. 00:27:46.270 --> 00:27:49.510 2, 3 and 5 are the ones that we're really 00:27:49.510 --> 00:27:51.820 going to see a lot of. 00:27:51.820 --> 00:27:54.250 Questions or thoughts or-- 00:27:54.250 --> 00:27:55.450 OK. 00:27:55.450 --> 00:28:00.760 I guess I want to tell you now to complete today's-- 00:28:00.760 --> 00:28:03.100 moving forward in this subject-- 00:28:03.100 --> 00:28:06.070 the fundamental theorem of linear algebra-- 00:28:06.070 --> 00:28:08.420 the fundamental theorem of linear algebra. 00:28:08.420 --> 00:28:08.920 OK. 00:28:08.920 --> 00:28:10.890 Ready for that? 00:28:10.890 --> 00:28:13.590 Or you may have seen it already, because it's 00:28:13.590 --> 00:28:17.970 like the highlight of this subject-- 00:28:17.970 --> 00:28:21.510 of the basic ideas in this subject. 00:28:21.510 --> 00:28:22.010 Right. 00:28:22.010 --> 00:28:23.300 And then maybe I can-- 00:28:23.300 --> 00:28:25.010 after I tell you that theorem-- 00:28:28.600 --> 00:28:32.560 people around the world send me homework problems to do. 00:28:32.560 --> 00:28:35.680 Now, you would think any sensible professor would never 00:28:35.680 --> 00:28:37.000 do those problems. 00:28:37.000 --> 00:28:39.190 He would say, it's your problem. 00:28:39.190 --> 00:28:43.990 But I get carried away and I solve them sometimes. 00:28:43.990 --> 00:28:47.110 So one came from India last week, 00:28:47.110 --> 00:28:50.290 and it involved the fundamental theorem of linear algebra. 00:28:50.290 --> 00:28:53.970 Whoever teaching it there really was on the ball. 00:28:53.970 --> 00:28:56.560 And well, I'll tell you that problem 00:28:56.560 --> 00:29:00.160 after the fundamental theorem. 00:29:00.160 --> 00:29:00.760 OK. 00:29:00.760 --> 00:29:01.600 Fundamental theorem. 00:29:06.190 --> 00:29:09.410 It's about four sub spaces. 00:29:09.410 --> 00:29:13.720 So I invented the name four fundamental sub spaces. 00:29:13.720 --> 00:29:16.390 So can I list the four sub places? 00:29:16.390 --> 00:29:20.830 Fundamental sub spaces. 00:29:20.830 --> 00:29:23.980 Well, we know one of them already. 00:29:23.980 --> 00:29:29.670 The column space-- so, for a matrix. 00:29:29.670 --> 00:29:32.060 We are given a matrix A-- 00:29:32.060 --> 00:29:35.090 that's m by n of rank r. 00:29:37.660 --> 00:29:39.600 That's our normal starting point. 00:29:43.490 --> 00:29:45.220 So what are those four sub spaces, 00:29:45.220 --> 00:29:46.540 and how are they related? 00:29:46.540 --> 00:29:47.860 And what's their dimension? 00:29:47.860 --> 00:29:51.830 And what-- those are key facts. 00:29:51.830 --> 00:29:52.330 OK. 00:29:52.330 --> 00:29:54.010 We already know the column space-- 00:29:59.960 --> 00:30:02.780 column space of a matrix. 00:30:02.780 --> 00:30:07.760 And actually, we already know the row space of a matrix. 00:30:07.760 --> 00:30:09.770 And we have the notation for that, 00:30:09.770 --> 00:30:13.430 column space of A transpose. 00:30:13.430 --> 00:30:16.010 And what is the dimension-- 00:30:16.010 --> 00:30:18.410 so that was the key point in the first lecture. 00:30:18.410 --> 00:30:20.300 Anybody who missed the first lecture, 00:30:20.300 --> 00:30:28.880 should go back to the notes of 1.1 00:30:28.880 --> 00:30:36.950 for the thinking that goes into the dimension equals what? 00:30:36.950 --> 00:30:39.580 Which of those three numbers do I want to-- 00:30:39.580 --> 00:30:42.530 is the dimension of the column space? 00:30:42.530 --> 00:30:46.990 R. And what can I say about r, right away, 00:30:46.990 --> 00:30:47.950 compared to m and n? 00:30:47.950 --> 00:30:49.733 AUDIENCE: [INAUDIBLE]. 00:30:49.733 --> 00:30:50.400 PROFESSOR: Yeah. 00:30:50.400 --> 00:30:52.320 Less or equal. 00:30:52.320 --> 00:30:57.120 r couldn't-- I couldn't have more independent columns than I 00:30:57.120 --> 00:30:58.760 have columns. 00:30:58.760 --> 00:31:01.260 So I've got n columns. 00:31:01.260 --> 00:31:04.890 So r of them are independent. 00:31:04.890 --> 00:31:09.120 So r is somewhere, less or equal-- hopefully equal to n. 00:31:09.120 --> 00:31:11.440 What about the dimension of the row space? 00:31:11.440 --> 00:31:15.116 How many independent rows has the matrix got? 00:31:15.116 --> 00:31:15.616 AUDIENCE: R. 00:31:15.616 --> 00:31:16.710 PROFESSOR: R. Thank you. 00:31:16.710 --> 00:31:23.910 That's the great fact with a new proof last time 00:31:23.910 --> 00:31:26.910 in section 1.1-- 00:31:26.910 --> 00:31:30.670 that those have the same dimension, same dimension. 00:31:30.670 --> 00:31:33.700 Which is-- you think, oh, OK. 00:31:33.700 --> 00:31:35.680 You look at a simple example. 00:31:35.680 --> 00:31:36.760 It's true. 00:31:36.760 --> 00:31:42.760 But if you're given a matrix that's 50 by 100, 00:31:42.760 --> 00:31:46.580 really the fact that those 100 columns 00:31:46.580 --> 00:31:50.450 have the same number of independent ones as those 50 00:31:50.450 --> 00:31:51.110 row-- 00:31:51.110 --> 00:31:53.840 that's like great. 00:31:53.840 --> 00:31:54.470 OK. 00:31:54.470 --> 00:31:59.700 Now the other spaces are the null space 00:31:59.700 --> 00:32:03.820 of the matrix, N of A. 00:32:03.820 --> 00:32:07.250 And just to make everything naturally symmetric, 00:32:07.250 --> 00:32:11.650 the null space of A transpose. 00:32:14.170 --> 00:32:16.600 Those are the last two. 00:32:16.600 --> 00:32:21.020 Those are the four fundamental sub spaces, which you've seen. 00:32:21.020 --> 00:32:25.150 And they're even on the cover of the linear algebra textbook. 00:32:25.150 --> 00:32:26.680 OK. 00:32:26.680 --> 00:32:28.822 So what's the null space? 00:32:28.822 --> 00:32:30.640 AUDIENCE: It's the set of [INAUDIBLE].. 00:32:30.640 --> 00:32:32.920 PROFESSOR: It's the set of solutions to Ax 00:32:32.920 --> 00:32:34.510 equals 0, right. 00:32:34.510 --> 00:32:48.130 Null space is all solutions to Ax equal 0. 00:32:48.130 --> 00:32:50.050 So the null space has vector-- 00:32:50.050 --> 00:32:51.720 these vectors in it-- the x's. 00:32:54.270 --> 00:32:57.410 The null space isn't taken from the matrix. 00:32:57.410 --> 00:33:01.100 The row space and the column space-- those numbers 00:33:01.100 --> 00:33:03.170 are sitting in the matrix. 00:33:03.170 --> 00:33:07.100 The null space, and the null space of A transpose, 00:33:07.100 --> 00:33:09.930 are solutions to-- 00:33:09.930 --> 00:33:14.670 the word null is reflecting the fact that that's a 0. 00:33:14.670 --> 00:33:16.710 And that's what makes it a space. 00:33:16.710 --> 00:33:18.210 Now can you just-- 00:33:18.210 --> 00:33:20.890 let me just ask you to think again. 00:33:20.890 --> 00:33:23.580 What's implied when I say-- 00:33:23.580 --> 00:33:25.480 when I use the word space-- 00:33:25.480 --> 00:33:27.645 a space of vectors? 00:33:27.645 --> 00:33:29.020 AUDIENCE: Closed under addition-- 00:33:29.020 --> 00:33:30.040 PROFESSOR: I can add-- 00:33:30.040 --> 00:33:31.020 yeah. 00:33:31.020 --> 00:33:37.930 So I can do the most important operations of linear algebra 00:33:37.930 --> 00:33:39.460 in that space. 00:33:39.460 --> 00:33:41.120 I can add two vectors. 00:33:41.120 --> 00:33:42.970 Here, let me just add them. 00:33:42.970 --> 00:33:45.760 So here I'll have a vector x, and let 00:33:45.760 --> 00:33:49.960 me say another one, a vector y. 00:33:49.960 --> 00:33:53.140 Then, I do addition. 00:33:53.140 --> 00:33:54.970 I follow the rules. 00:33:54.970 --> 00:34:00.830 I see that this can be written as Ax plus y is 0 plus 0. 00:34:00.830 --> 00:34:03.340 So what have I learned? 00:34:03.340 --> 00:34:07.090 I've learned that if x is in the null space, 00:34:07.090 --> 00:34:10.750 and y is in the null space, then x plus y. 00:34:10.750 --> 00:34:13.659 So the null space is, as you said, closed, 00:34:13.659 --> 00:34:16.449 meaning I don't go outside it. 00:34:16.449 --> 00:34:19.929 If x is in it, and y is in it, then the sum is in it. 00:34:19.929 --> 00:34:27.429 And similarly, from Ax equals 0, I get to A times cx equals 0. 00:34:27.429 --> 00:34:31.420 Just multiply by c-- by a number c. 00:34:31.420 --> 00:34:33.460 So those two facts-- 00:34:33.460 --> 00:34:37.210 that means I can do linear algebra. 00:34:37.210 --> 00:34:39.760 I can multiply by numbers. 00:34:39.760 --> 00:34:40.960 And I can add. 00:34:40.960 --> 00:34:44.199 In other words, I can take linear combinations. 00:34:44.199 --> 00:34:46.480 That's what you do with vectors. 00:34:46.480 --> 00:34:48.550 And the point is, if I do it-- 00:34:48.550 --> 00:34:52.719 if I take combinations of two null space guys, 00:34:52.719 --> 00:34:55.310 I'm still in the null space. 00:34:55.310 --> 00:34:56.300 OK. 00:34:56.300 --> 00:35:01.350 So that's the point of the null space. 00:35:01.350 --> 00:35:06.800 And-- well now-- so now part of the fundamental theorem 00:35:06.800 --> 00:35:11.630 is to figure out how many independent vectors are 00:35:11.630 --> 00:35:12.680 in the null space. 00:35:12.680 --> 00:35:16.490 How many solutions-- independent solutions-- 00:35:16.490 --> 00:35:19.260 does that system of equations have? 00:35:19.260 --> 00:35:21.380 So that would be the dimension. 00:35:21.380 --> 00:35:25.070 And I have to ask you what it is. 00:35:25.070 --> 00:35:27.080 Let me draw a picture, while you're 00:35:27.080 --> 00:35:29.180 thinking about those spaces. 00:35:32.190 --> 00:35:35.710 It's fantastic to have these beautiful clean boards. 00:35:35.710 --> 00:35:36.210 OK. 00:35:36.210 --> 00:35:39.870 So here's my picture of the row space. 00:35:43.630 --> 00:35:48.680 Row-- that's the column space of A transpose. 00:35:48.680 --> 00:35:58.265 And here's my picture of the null space, N of A. 00:35:58.265 --> 00:36:01.085 And that's the solutions to Ax equals 0. 00:36:03.680 --> 00:36:06.650 And why have I put these two together, 00:36:06.650 --> 00:36:08.910 and these two together? 00:36:08.910 --> 00:36:17.120 So-- and the other pair will be the column space, C of A. 00:36:17.120 --> 00:36:26.172 and the null space of A transpose. 00:36:26.172 --> 00:36:27.380 So there are the four spaces. 00:36:29.930 --> 00:36:33.380 Their relationship is the fundamental theorem 00:36:33.380 --> 00:36:35.630 of linear algebra. 00:36:35.630 --> 00:36:40.800 So first of all, what-- 00:36:40.800 --> 00:36:42.285 so I have an m by n matrix. 00:36:45.650 --> 00:36:48.360 So that tells me that my row-- 00:36:48.360 --> 00:36:51.550 a typical row has n components, right? 00:36:51.550 --> 00:36:53.250 I look at an m by n matrix. 00:36:53.250 --> 00:36:55.805 Let's do a 2 by 3 matrix. 00:37:00.060 --> 00:37:04.890 So if I look at the row space, this is m. 00:37:04.890 --> 00:37:06.380 And this is n. 00:37:06.380 --> 00:37:08.270 So I see three-- 00:37:08.270 --> 00:37:12.280 the rows have length three. 00:37:12.280 --> 00:37:14.950 And of course, they multiply the x's, 00:37:14.950 --> 00:37:19.030 which also have the length three, x1, x2, x3. 00:37:19.030 --> 00:37:21.220 That's why these are together, because they're 00:37:21.220 --> 00:37:23.650 both in n dimensional space. 00:37:29.150 --> 00:37:31.250 Then why are these together? 00:37:31.250 --> 00:37:36.540 Because the columns are in two dimensional space, 00:37:36.540 --> 00:37:38.050 for this example. 00:37:38.050 --> 00:37:41.350 And the null space of A transpose 00:37:41.350 --> 00:37:49.390 would be just two components, like y1 and y2, to give 0s. 00:37:49.390 --> 00:37:51.580 So do you see that this is R-- 00:37:51.580 --> 00:37:54.730 these guys are in Rn? 00:37:54.730 --> 00:37:56.980 So that's the first-- 00:37:56.980 --> 00:37:58.930 get things straight. 00:37:58.930 --> 00:38:02.770 Two spaces in Rn, two spaces in Rm. 00:38:08.040 --> 00:38:12.570 Now, what am I going to ask about these spaces? 00:38:12.570 --> 00:38:14.430 I guess I've already started asking 00:38:14.430 --> 00:38:16.610 and didn't wait for an answer. 00:38:16.610 --> 00:38:18.270 Their dimension. 00:38:18.270 --> 00:38:24.690 So this has dimension R. And what's the dimension-- 00:38:24.690 --> 00:38:31.450 how many-- this is really such a key fact. 00:38:31.450 --> 00:38:37.880 If I have m equations, Ax equals 0. 00:38:37.880 --> 00:38:41.630 And if R of those equations are independent, 00:38:41.630 --> 00:38:43.730 how many solutions? 00:38:43.730 --> 00:38:45.500 So the dimension of this space is 00:38:45.500 --> 00:38:48.950 going to tell me how many solutions 00:38:48.950 --> 00:38:53.360 to Ax in two m equations, but really 00:38:53.360 --> 00:39:02.610 only are genuine independent equations in this system Ax 00:39:02.610 --> 00:39:03.870 equals 0. 00:39:03.870 --> 00:39:04.840 How many? 00:39:04.840 --> 00:39:06.650 So can I ask the question again? 00:39:06.650 --> 00:39:12.570 And I want the answer in terms of m, and n, and R. So I have-- 00:39:12.570 --> 00:39:15.610 I really have R equations. 00:39:15.610 --> 00:39:17.680 If I look at Ax equals 0, it looks 00:39:17.680 --> 00:39:19.660 like m separate equations. 00:39:19.660 --> 00:39:23.020 But m minus R-- 00:39:23.020 --> 00:39:27.210 of those-- are just copies or combinations of others. 00:39:27.210 --> 00:39:30.180 So there are independent equations. 00:39:30.180 --> 00:39:32.648 So what's-- how many have I got? 00:39:32.648 --> 00:39:35.483 AUDIENCE: [INAUDIBLE]. 00:39:35.483 --> 00:39:37.900 PROFESSOR: And And that's what I'm going to write in here. 00:39:40.870 --> 00:39:41.740 So-- yeah. 00:39:41.740 --> 00:39:45.970 So x has n components. 00:39:45.970 --> 00:39:51.490 And there are real active equations 00:39:51.490 --> 00:39:53.680 that they have to satisfy. 00:39:53.680 --> 00:39:58.282 And that leaves n minus R. That's the key point. 00:39:58.282 --> 00:40:04.790 That's the key point that there are n components of x-- 00:40:04.790 --> 00:40:07.610 n unknowns-- n unknowns. 00:40:07.610 --> 00:40:11.020 And there are R constraints-- 00:40:11.020 --> 00:40:13.170 independent constraints. 00:40:13.170 --> 00:40:16.270 So those n get-- 00:40:16.270 --> 00:40:19.960 if I want to satisfy those constraints that knocks out 00:40:19.960 --> 00:40:23.050 dimension R, and leaves n minus R. So that's 00:40:23.050 --> 00:40:23.920 the dimension here. 00:40:26.860 --> 00:40:31.960 And the beauty of the count is that those two numbers add up 00:40:31.960 --> 00:40:33.070 to n. 00:40:33.070 --> 00:40:35.110 Everybody's accounted for. 00:40:35.110 --> 00:40:39.590 Every vector has a piece in the row space 00:40:39.590 --> 00:40:41.450 and a piece and the null space. 00:40:41.450 --> 00:40:47.680 And that's-- those two pieces give you back the vector. 00:40:47.680 --> 00:40:48.650 Do you see that? 00:40:48.650 --> 00:40:51.300 That's just nice, that the numbers come out right. 00:40:51.300 --> 00:40:54.560 And of course, they come out right here, too. 00:40:54.560 --> 00:40:57.200 You could say, just transpose the matrix 00:40:57.200 --> 00:40:59.660 and write it-- write the same thing again. 00:40:59.660 --> 00:41:01.580 What's the dimension of the column space? 00:41:05.537 --> 00:41:06.130 Equals? 00:41:06.130 --> 00:41:08.370 The other column space in the matrix has dimension-- 00:41:08.370 --> 00:41:09.360 AUDIENCE: R. 00:41:09.360 --> 00:41:11.190 PROFESSOR: R. Right. 00:41:11.190 --> 00:41:15.870 And the row-- this guy is left out of some linear algebra 00:41:15.870 --> 00:41:18.660 books, as if it doesn't belong. 00:41:18.660 --> 00:41:23.610 But isn't it clear that without it, everything 00:41:23.610 --> 00:41:28.050 is only three quarters done? 00:41:28.050 --> 00:41:29.820 We have to have this guy. 00:41:29.820 --> 00:41:32.526 And its dimension is 00:41:32.526 --> 00:41:34.280 AUDIENCE: M minus R. 00:41:34.280 --> 00:41:37.580 PROFESSOR: M minus R. Yeah. 00:41:37.580 --> 00:41:41.780 That count is just, for A transpose, what 00:41:41.780 --> 00:41:44.390 this count was for A. Yeah. 00:41:44.390 --> 00:41:50.120 So we've got those dimensions, R and n minus R, R and m 00:41:50.120 --> 00:41:51.830 minus R. Yeah. 00:41:54.580 --> 00:41:57.780 You'll have known this, but we need 00:41:57.780 --> 00:42:05.070 to see it once again in 2018, before we start using it. 00:42:05.070 --> 00:42:06.980 Now is that the fundamental theorem? 00:42:06.980 --> 00:42:10.040 Is that all to it there is? 00:42:10.040 --> 00:42:12.330 No. 00:42:12.330 --> 00:42:14.910 There is another piece to the fundamental theorem, 00:42:14.910 --> 00:42:19.170 which is, sort of you could say, the geometry. 00:42:19.170 --> 00:42:20.890 Here I have a sub space. 00:42:20.890 --> 00:42:26.630 Here I have a subspace of this big n dimensional space. 00:42:26.630 --> 00:42:31.010 So I visualize those sub spaces as some kind of a plane-- 00:42:31.010 --> 00:42:36.840 an R dimensional plane- and an n minus R dimensional plane. 00:42:36.840 --> 00:42:41.970 And I want to see how are those two planes connected. 00:42:41.970 --> 00:42:45.050 How are those two planes connected? 00:42:45.050 --> 00:42:49.240 And let me get a piece of the-- 00:42:49.240 --> 00:42:52.460 blank piece of the board to remember the final step. 00:42:52.460 --> 00:42:53.350 Right. 00:42:53.350 --> 00:43:02.860 So we've got dimensions r and n minus r, and then over here r, 00:43:02.860 --> 00:43:05.470 and m minus r. 00:43:05.470 --> 00:43:05.970 OK. 00:43:05.970 --> 00:43:09.310 And this is for the rows. 00:43:09.310 --> 00:43:12.590 This is for the null space. 00:43:12.590 --> 00:43:15.400 So this has the rows in it. 00:43:15.400 --> 00:43:16.540 This has the null-- 00:43:16.540 --> 00:43:19.210 the solutions to Ax equals 0. 00:43:19.210 --> 00:43:23.850 What is the beautiful geometry-- 00:43:26.470 --> 00:43:30.005 how do you visualize those two spaces? 00:43:30.005 --> 00:43:30.880 How do you visualize? 00:43:30.880 --> 00:43:32.445 Let me take an example. 00:43:32.445 --> 00:43:39.250 Let A be 1, 2, 4, 2, 4, 8. 00:43:39.250 --> 00:43:40.380 Sorry about that. 00:43:40.380 --> 00:43:41.980 That's kind of a-- 00:43:41.980 --> 00:43:43.765 you see it hoked up example. 00:43:46.330 --> 00:43:49.480 So this is 2 by 3. 00:43:49.480 --> 00:43:52.250 So there n is 3. 00:43:52.250 --> 00:43:54.440 What's in the null space of this matrix? 00:43:58.770 --> 00:44:04.660 Can you see a vector that solves Ax equals 0? 00:44:04.660 --> 00:44:06.780 And in fact, how many will there be? 00:44:06.780 --> 00:44:09.150 What's-- yeah, what's r for this matrix? 00:44:09.150 --> 00:44:11.910 Just tell me all the good stuff. 00:44:11.910 --> 00:44:20.110 For that example, m is 2, n is 3, and r is-- 00:44:20.110 --> 00:44:21.010 AUDIENCE: 1. 00:44:21.010 --> 00:44:22.260 PROFESSOR: 1. 00:44:22.260 --> 00:44:24.780 Everybody sees 1 for the rank? 00:44:24.780 --> 00:44:28.060 The rows are dependent. 00:44:28.060 --> 00:44:30.300 There's only one independent row. 00:44:30.300 --> 00:44:31.980 The columns are dependent. 00:44:31.980 --> 00:44:35.910 There's only-- every column is a multiple of 1, 2. 00:44:35.910 --> 00:44:38.120 It's a rank 1 matrix. 00:44:38.120 --> 00:44:38.700 OK. 00:44:38.700 --> 00:44:45.640 What about its-- so its row space has dimension-- 00:44:45.640 --> 00:44:46.140 AUDIENCE: 1. 00:44:46.140 --> 00:44:47.250 PROFESSOR: 1. 00:44:47.250 --> 00:44:51.760 And it's null space has dimension-- 00:44:51.760 --> 00:44:53.030 AUDIENCE: 2. 00:44:53.030 --> 00:44:57.390 PROFESSOR: 2 So, cause n minus r will be 2. 00:44:57.390 --> 00:45:01.950 So I'm looking for a couple of vectors that both give 0. 00:45:01.950 --> 00:45:02.910 I believe there-- 00:45:02.910 --> 00:45:07.670 I think I've only got one independent row there. 00:45:07.670 --> 00:45:10.910 So I should be able to find two different vectors 00:45:10.910 --> 00:45:17.180 that solve Ax equals 0. 00:45:17.180 --> 00:45:20.630 So what what's the solution to Ax equals 0? 00:45:20.630 --> 00:45:22.010 AUDIENCE: 0 minus 2, 1. 00:45:22.010 --> 00:45:27.020 PROFESSOR: 0 minus 2, 1. 00:45:27.020 --> 00:45:29.070 Yeah, that works. 00:45:29.070 --> 00:45:31.280 And what's an independent solution? 00:45:31.280 --> 00:45:32.700 AUDIENCE: 4, 0, negative 1? 00:45:32.700 --> 00:45:36.180 PROFESSOR: 4, 0-- don't throw me off-- 00:45:36.180 --> 00:45:38.290 4, 0 and-- 00:45:38.290 --> 00:45:39.040 AUDIENCE: Minus 1. 00:45:39.040 --> 00:45:39.480 PROFESSOR: Minus 1. 00:45:39.480 --> 00:45:40.140 Yeah. 00:45:40.140 --> 00:45:42.080 That looks good. 00:45:42.080 --> 00:45:44.220 That looks good. 00:45:44.220 --> 00:45:46.980 And then the claim is that every solution would 00:45:46.980 --> 00:45:49.390 be a combination of those two. 00:45:49.390 --> 00:45:51.930 And this is how many there are. 00:45:51.930 --> 00:45:56.240 And now, it's the geometry I'm completing. 00:45:56.240 --> 00:45:58.700 So we have two minutes left in this lecture. 00:45:58.700 --> 00:46:01.380 You just have to tell me how-- 00:46:01.380 --> 00:46:06.060 what's the relation between these guys in the row space, 00:46:06.060 --> 00:46:08.780 and that guy in the null space? 00:46:08.780 --> 00:46:12.350 What's the relation between the rows of A, 00:46:12.350 --> 00:46:15.200 the solutions to Ax equals 0? 00:46:15.200 --> 00:46:17.960 Between-- if you see it-- if you saw that vector and that 00:46:17.960 --> 00:46:18.710 vector-- 00:46:18.710 --> 00:46:23.070 well, A times x is 0. 00:46:23.070 --> 00:46:24.470 So what does that tell us? 00:46:24.470 --> 00:46:29.570 What do we see for the relation between 1, 2, 4, and 0 00:46:29.570 --> 00:46:30.627 minus 2, 1. 00:46:30.627 --> 00:46:31.460 AUDIENCE: Orthogonal 00:46:31.460 --> 00:46:32.752 PROFESSOR: They are orthogonal. 00:46:32.752 --> 00:46:33.560 Terrific. 00:46:33.560 --> 00:46:34.550 Yes. 00:46:34.550 --> 00:46:37.985 Orthogonal I test by the dot product, 0 minus 4, 00:46:37.985 --> 00:46:39.840 4, add to 0. 00:46:39.840 --> 00:46:40.340 Yes. 00:46:40.340 --> 00:46:46.280 So the-- and that's a completely general fact. 00:46:46.280 --> 00:46:50.420 When I look at Ax equals 0, it's telling me 00:46:50.420 --> 00:46:53.250 that x is orthogonal to the rows. 00:46:53.250 --> 00:46:54.450 Do you see that? 00:46:54.450 --> 00:46:57.380 Just to put it in again here. 00:46:57.380 --> 00:47:03.080 If I look at Ax, A has a bunch of rows. 00:47:03.080 --> 00:47:05.480 X has one column. 00:47:05.480 --> 00:47:06.590 And I get 0. 00:47:06.590 --> 00:47:08.135 That's the point of the null space. 00:47:11.360 --> 00:47:17.600 And that equation is just saying that row 1 is orthogonal, 00:47:17.600 --> 00:47:20.750 because that's the dot product of row 1 with x. 00:47:20.750 --> 00:47:24.550 So here is row 1, row 2, row 3, and row 4 with x. 00:47:24.550 --> 00:47:26.360 The rows with x-- 00:47:26.360 --> 00:47:27.920 and I get 0s. 00:47:27.920 --> 00:47:36.310 So the point is then, these two spaces are at 90 degree angles. 00:47:36.310 --> 00:47:41.380 That's really a neat picture of the four sub spaces. 00:47:41.380 --> 00:47:43.990 And these two are for the same reason-- 00:47:43.990 --> 00:47:45.550 at 90 degree angles-- 00:47:45.550 --> 00:47:49.900 off in m dimensional space. 00:47:49.900 --> 00:47:54.220 So this is the fundamental theorem of linear algebra-- 00:47:54.220 --> 00:47:57.670 to see that the dimensions come out right, 00:47:57.670 --> 00:48:00.160 and the geometry comes out right. 00:48:00.160 --> 00:48:00.670 Yeah. 00:48:00.670 --> 00:48:02.950 And then, now, next time-- 00:48:02.950 --> 00:48:06.140 following the notes-- and I have a few more copies 00:48:06.140 --> 00:48:11.810 of the one hand out. 00:48:11.810 --> 00:48:18.610 We'll move on quickly next week to eigenvalues and positive 00:48:18.610 --> 00:48:20.980 definite matrices. 00:48:20.980 --> 00:48:25.650 Good this is really linear algebra moving on.