WEBVTT 00:00:01.550 --> 00:00:03.920 The following content is provided under a Creative 00:00:03.920 --> 00:00:05.310 Commons license. 00:00:05.310 --> 00:00:07.520 Your support will help MIT OpenCourseWare 00:00:07.520 --> 00:00:11.610 continue to offer high quality educational resources for free. 00:00:11.610 --> 00:00:14.180 To make a donation or to view additional materials 00:00:14.180 --> 00:00:18.140 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:18.140 --> 00:00:19.026 at ocw.mit.edu. 00:00:23.490 --> 00:00:26.130 GILBERT STRANG: Shall we start? 00:00:26.130 --> 00:00:30.780 Let me just say, this is a great adventure for me 00:00:30.780 --> 00:00:33.900 to be here all on my own, teaching a course that 00:00:33.900 --> 00:00:35.400 involves learning from data. 00:00:35.400 --> 00:00:39.450 So it's an exciting subject, and a lot of linear algebra 00:00:39.450 --> 00:00:40.290 goes into it. 00:00:40.290 --> 00:00:43.350 So it's a second course on linear algebra. 00:00:45.960 --> 00:00:52.900 Can I just-- so there is a Stellar site established, 00:00:52.900 --> 00:00:56.770 and that will be the basic thing that we use. 00:00:56.770 --> 00:00:59.470 This is a public site-- 00:00:59.470 --> 00:01:05.310 math.mit.edu/learningfromdata. 00:01:05.310 --> 00:01:11.630 So a book is coming pretty quickly as we speak, 00:01:11.630 --> 00:01:20.410 or after we speak, and that site has the table of contents 00:01:20.410 --> 00:01:23.140 of the book, which would give you an idea of what 00:01:23.140 --> 00:01:24.580 could be in the course. 00:01:24.580 --> 00:01:28.480 And I printed out a copy for everybody just of that one 00:01:28.480 --> 00:01:28.980 page. 00:01:28.980 --> 00:01:30.340 This is probably the final-- 00:01:30.340 --> 00:01:32.820 first and last handout-- 00:01:32.820 --> 00:01:35.780 maybe-- with a table of contents, 00:01:35.780 --> 00:01:37.930 which you'll see there. 00:01:37.930 --> 00:01:41.050 And also, you'll see there the first two sections 00:01:41.050 --> 00:01:44.820 of the book, which is what I'll talk about today 00:01:44.820 --> 00:01:47.830 and a little bit into Friday. 00:01:47.830 --> 00:01:51.400 So that's linear algebra, of course, because the course 00:01:51.400 --> 00:01:54.820 begins with linear algebra-- 00:01:54.820 --> 00:01:57.640 actually, things that you would know from 18.06, 00:01:57.640 --> 00:02:04.310 but this is a way to say this is really important stuff. 00:02:04.310 --> 00:02:06.820 So that's what I'll do today. 00:02:06.820 --> 00:02:11.410 I'd like to start on the linear algebra today. 00:02:13.960 --> 00:02:18.205 Here's a great fact about the course. 00:02:21.310 --> 00:02:23.830 So we taught it last year, several of us 00:02:23.830 --> 00:02:28.840 together, and we knew there wouldn't be a final exam, 00:02:28.840 --> 00:02:33.580 but we imagined there might be quizzes along the way. 00:02:33.580 --> 00:02:36.610 But then we couldn't think of anything to put on the quizzes. 00:02:36.610 --> 00:02:41.590 So we canceled those. 00:02:41.590 --> 00:02:43.930 But you do we learn a lot, nevertheless. 00:02:43.930 --> 00:02:48.410 And so I guess we base the grades on the homeworks. 00:02:48.410 --> 00:02:53.140 So the homeworks will be partly linear algebra questions 00:02:53.140 --> 00:02:58.990 and partly online, like recognizing handwriting, 00:02:58.990 --> 00:03:03.880 stitching images together, many other things. 00:03:03.880 --> 00:03:07.330 And I'll talk about those as we go. 00:03:07.330 --> 00:03:08.980 Good. 00:03:08.980 --> 00:03:11.710 So that's the general picture. 00:03:11.710 --> 00:03:15.310 And I'll say more about it today. 00:03:15.310 --> 00:03:17.140 And I could answer any question about it. 00:03:17.140 --> 00:03:20.740 So we're getting videotaped. 00:03:20.740 --> 00:03:25.990 So if anybody's bashful, sit in the far back. 00:03:25.990 --> 00:03:29.860 But it'll be fun. 00:03:29.860 --> 00:03:33.430 You may know the videos for 18.06. 00:03:33.430 --> 00:03:37.270 So this is the next step, 18.065. 00:03:37.270 --> 00:03:41.170 It's pretty exciting for me. 00:03:41.170 --> 00:03:46.210 So any question, or shall I do a little math? 00:03:46.210 --> 00:03:46.990 Why not? 00:03:46.990 --> 00:03:53.620 And then I'll say a little more about the course, 00:03:53.620 --> 00:03:57.610 just so you have an idea of what's ahead. 00:03:57.610 --> 00:04:00.910 And it looks like this room is exactly the right size to me, 00:04:00.910 --> 00:04:02.140 so I'm pleased. 00:04:08.500 --> 00:04:10.210 So what's the deal in linear algebra? 00:04:13.330 --> 00:04:15.970 Forgive me if I start with what I 00:04:15.970 --> 00:04:18.730 do the very first day in 18.06, which is 00:04:18.730 --> 00:04:22.770 multiply a matrix by a vector. 00:04:22.770 --> 00:04:25.290 And then I'll graduate to multiplying a matrix 00:04:25.290 --> 00:04:27.410 by a matrix. 00:04:27.410 --> 00:04:29.640 And you will say, I know that stuff. 00:04:29.640 --> 00:04:31.500 But do you know it the right way? 00:04:31.500 --> 00:04:33.660 Do you think of the multiplication the right way? 00:04:33.660 --> 00:04:37.320 So let me tell you what I believe to be the right way. 00:04:37.320 --> 00:04:56.190 So let me take a matrix, say, 2, 3, 5, 1, 1, 7, and 3, 4, 12. 00:04:56.190 --> 00:05:02.470 And I'll always call matrices A. 00:05:02.470 --> 00:05:07.750 So first step is just A times x, A times a vector. 00:05:07.750 --> 00:05:12.640 So I multiply A by, let's say, x1, x2, x3. 00:05:12.640 --> 00:05:17.310 And how do I look at that answer? 00:05:17.310 --> 00:05:22.020 So the choices-- think of the rows of the matrix 00:05:22.020 --> 00:05:24.440 or think of the columns. 00:05:24.440 --> 00:05:27.110 And if you think of the rows, which 00:05:27.110 --> 00:05:32.690 is the standard way to multiply, you would take the dot product. 00:05:32.690 --> 00:05:43.280 So the first way is dot products of row dotted with x. 00:05:46.190 --> 00:05:49.200 2x1 plus x2 plus 3x3. 00:05:49.200 --> 00:05:54.330 It gives you the answer a component at a time. 00:05:54.330 --> 00:05:56.630 That's the low level way. 00:05:56.630 --> 00:06:00.140 The good way to see it is vector-wise. 00:06:00.140 --> 00:06:08.690 See this as x1 multiplies that first column, x2 times 00:06:08.690 --> 00:06:14.360 the second column, 1, 1, 7, and x3 times 00:06:14.360 --> 00:06:17.530 the third column, 3, 4, 12. 00:06:21.994 --> 00:06:24.480 Good. 00:06:24.480 --> 00:06:27.750 So it's a combination of vectors, 00:06:27.750 --> 00:06:30.740 and of course, it produces a vector. 00:06:30.740 --> 00:06:33.800 And here, we have a 3 by 3 matrix on our vectors 00:06:33.800 --> 00:06:35.960 are in R3. 00:06:35.960 --> 00:06:40.100 And most vectors will be in R3 or Rn for this course. 00:06:42.770 --> 00:06:44.400 So that's the right answer. 00:06:44.400 --> 00:06:46.430 And of course, the first component 00:06:46.430 --> 00:06:50.950 is 2x1 and 1x2 and 3x3. 00:06:50.950 --> 00:06:54.510 The same 2, 1, 3-- the same dot product, it comes out right. 00:06:54.510 --> 00:06:57.920 But you see it all at once instead of piecemeal. 00:07:00.720 --> 00:07:01.860 Part of the course-- 00:07:01.860 --> 00:07:04.740 I guess part of what I hope to get across 00:07:04.740 --> 00:07:11.590 is thinking of a matrix as a whole thing, 00:07:11.590 --> 00:07:15.270 not just a bunch of nine or m times n numbers. 00:07:15.270 --> 00:07:19.280 But thinking of it as a thing. 00:07:19.280 --> 00:07:25.020 A matrix multiplies a vector to give another vector. 00:07:25.020 --> 00:07:32.040 So when I say Ax, you immediately think that-- 00:07:32.040 --> 00:07:35.580 you immediately think, OK, Ax has a clear meaning, 00:07:35.580 --> 00:07:38.310 it's a combination of the columns of A. 00:07:38.310 --> 00:07:44.530 So now let me take that next step. 00:07:44.530 --> 00:07:49.540 And the next step is think about all combinations 00:07:49.540 --> 00:07:52.645 of the columns of A. We take the matrix A, 00:07:52.645 --> 00:07:57.670 and we take all x's and we imagine all the outputs. 00:07:57.670 --> 00:08:01.330 And what I want to ask you is, what does that look like? 00:08:01.330 --> 00:08:06.070 If I just take 1 vector x, I get a vector output. 00:08:06.070 --> 00:08:08.410 It takes a vector to a vector. 00:08:08.410 --> 00:08:10.990 But now I take all x's-- 00:08:10.990 --> 00:08:17.060 all of vectors x in 3D and I get all these answers, 00:08:17.060 --> 00:08:18.435 and I think of them all together. 00:08:21.020 --> 00:08:23.390 So I've got a bunch of vectors-- 00:08:23.390 --> 00:08:25.940 infinitely many vectors, actually. 00:08:25.940 --> 00:08:30.290 And the question is, if I plot those infinitely many vectors, 00:08:30.290 --> 00:08:32.179 what do I have? 00:08:32.179 --> 00:08:35.480 And the beauty of linear algebra is that questions like this-- 00:08:35.480 --> 00:08:40.429 you can answer them and you intuitively see it. 00:08:40.429 --> 00:08:45.680 You certainly see it in 3D and you have the right idea 00:08:45.680 --> 00:08:48.620 in 10 dimensions, even. 00:08:48.620 --> 00:08:52.290 Most of us don't see too well in 10d-- 00:08:52.290 --> 00:08:53.090 in R10. 00:08:53.090 --> 00:08:54.740 But here we got three. 00:08:57.250 --> 00:08:58.850 So do you see what I'm saying? 00:08:58.850 --> 00:09:00.950 I'm taking all x's. 00:09:00.950 --> 00:09:09.800 So all Ax gives us a big bunch of vectors. 00:09:09.800 --> 00:09:13.400 And that collection of vectors is 00:09:13.400 --> 00:09:20.460 called the column space of A. It's a space, in other words. 00:09:20.460 --> 00:09:23.412 That's the key word there, the column space of A. 00:09:23.412 --> 00:09:29.900 And I'll just write it as C of A when I need letters for it. 00:09:29.900 --> 00:09:32.540 So I'm going to ask you, what does this column space 00:09:32.540 --> 00:09:34.190 look like? 00:09:34.190 --> 00:09:36.650 And that depends on the matrix. 00:09:36.650 --> 00:09:41.230 Sometimes that column space would be the whole of R3. 00:09:41.230 --> 00:09:47.070 Sometimes it's a smaller set in R3, a space. 00:09:47.070 --> 00:09:48.700 Do you know what it is in this case? 00:09:48.700 --> 00:09:53.110 Do I get all of 3D out of these guys 00:09:53.110 --> 00:09:57.380 by choosing all x1 and x2 and x3? 00:09:57.380 --> 00:10:01.090 It seems like if it was a random matrix, 00:10:01.090 --> 00:10:04.010 the answer would be, surely, yes. 00:10:04.010 --> 00:10:07.720 If a random three by three matrix is-- 00:10:07.720 --> 00:10:10.390 it's column space is going to be all of our three, 00:10:10.390 --> 00:10:13.050 its columns are going to be independent, 00:10:13.050 --> 00:10:14.990 its rows are going to be independent, 00:10:14.990 --> 00:10:17.890 it's going to be invertible, it's going to be great. 00:10:17.890 --> 00:10:19.420 But is this matrix-- 00:10:19.420 --> 00:10:21.930 what's up with that matrix? 00:10:21.930 --> 00:10:24.210 No, it's not. 00:10:24.210 --> 00:10:28.560 So what do I get instead of all of R3? 00:10:28.560 --> 00:10:30.390 I get a plane, yeah. 00:10:30.390 --> 00:10:32.910 If you get that insight. 00:10:32.910 --> 00:10:35.280 So by taking x's-- 00:10:35.280 --> 00:10:38.170 everybody's with me-- all x's here. 00:10:38.170 --> 00:10:42.460 So that means that it fills in whatever. 00:10:42.460 --> 00:10:45.250 And, well, because it's linear algebra, 00:10:45.250 --> 00:10:52.960 it's going to be likely all of R3 or a plane, or even a line. 00:10:52.960 --> 00:10:56.390 Let me just bat over here for a moment. 00:10:56.390 --> 00:11:03.650 Give me a matrix whose column space would only be a line. 00:11:03.650 --> 00:11:04.780 All 1s? 00:11:04.780 --> 00:11:05.300 OK. 00:11:05.300 --> 00:11:09.500 Wow, that's a-- let me liven it up a little. 00:11:09.500 --> 00:11:14.180 3, 3, 3, 8, 8, 8. 00:11:18.350 --> 00:11:21.410 So I think that the combinations of those columns 00:11:21.410 --> 00:11:23.870 are all on the same line. 00:11:23.870 --> 00:11:27.800 That says that the column space is just a line. 00:11:27.800 --> 00:11:30.220 And I would say then that the matrix-- 00:11:30.220 --> 00:11:34.520 so column space of this A is a line. 00:11:34.520 --> 00:11:38.930 Another way I could say this is that the rank of the matrix 00:11:38.930 --> 00:11:41.450 is 1. 00:11:41.450 --> 00:11:45.470 The rank is sort of the dimension of the column space. 00:11:45.470 --> 00:11:47.180 Well, not sort, that's what it is. 00:11:47.180 --> 00:11:49.430 The rank is the dimension of the column-- 00:11:49.430 --> 00:11:51.530 everybody sees that you get a line? 00:11:51.530 --> 00:11:54.410 Because any combination x1 of that plus x2 00:11:54.410 --> 00:12:00.150 of that plus x3 of that is going to go along that line. 00:12:00.150 --> 00:12:03.050 Here's the first column, here's the second column. 00:12:03.050 --> 00:12:04.580 They're all on a line. 00:12:04.580 --> 00:12:07.070 So I'll never get off that line. 00:12:07.070 --> 00:12:10.470 If I allow all x's, I'll get the whole line. 00:12:10.470 --> 00:12:14.070 Now here, you said not a line. 00:12:14.070 --> 00:12:18.410 What was the column space for this guy? 00:12:18.410 --> 00:12:19.730 Plane. 00:12:19.730 --> 00:12:23.130 Now, why isn't it all of R3? 00:12:23.130 --> 00:12:25.860 What do you see that's special there-- 00:12:25.860 --> 00:12:28.050 because it is special-- 00:12:28.050 --> 00:12:34.350 that's making the column space special, a plane instead 00:12:34.350 --> 00:12:35.820 of the whole thing? 00:12:35.820 --> 00:12:38.430 Yeah, what's up with these three columns? 00:12:38.430 --> 00:12:40.820 AUDIENCE: The third column is the sum of it. 00:12:40.820 --> 00:12:46.160 GILBERT STRANG: The third column is the sum of these two. 00:12:46.160 --> 00:12:49.660 So the first column is fine, 2, 3, 5. 00:12:49.660 --> 00:12:52.210 Second column is in a different direction. 00:12:52.210 --> 00:12:55.190 And when I take combinations of the first two columns, 00:12:55.190 --> 00:12:57.200 what will I get? 00:12:57.200 --> 00:12:59.900 Sorry to keep asking you questions, but that's-- 00:13:03.600 --> 00:13:06.960 anyway, that's what I always do. 00:13:06.960 --> 00:13:10.470 So the combinations of those first two columns are--? 00:13:10.470 --> 00:13:11.420 AUDIENCE: A plane. 00:13:11.420 --> 00:13:13.170 GILBERT STRANG: A plane because they're 00:13:13.170 --> 00:13:15.790 in different-- everybody sees that picture. 00:13:15.790 --> 00:13:18.960 We've got column one going that way and column two 00:13:18.960 --> 00:13:19.630 going that way. 00:13:19.630 --> 00:13:22.500 And then if I take any multiple of column one, 00:13:22.500 --> 00:13:25.890 I've got a whole line, any multiple of column two. 00:13:25.890 --> 00:13:27.840 And then when I put the two together, 00:13:27.840 --> 00:13:32.940 it fills in the plane. 00:13:32.940 --> 00:13:33.440 Yeah. 00:13:33.440 --> 00:13:35.910 Your intuition says that's right. 00:13:35.910 --> 00:13:38.640 So this is a matrix of rank. 00:13:38.640 --> 00:13:40.350 What's the rank of this matrix? 00:13:40.350 --> 00:13:41.240 AUDIENCE: Two. 00:13:41.240 --> 00:13:43.030 GILBERT STRANG: Two. 00:13:43.030 --> 00:13:45.550 Because it's got two independent columns, 00:13:45.550 --> 00:13:47.800 but the third column is dependent. 00:13:47.800 --> 00:13:50.510 The third column is a combination of the others. 00:13:50.510 --> 00:13:56.620 So matrices like this are really the building blocks 00:13:56.620 --> 00:13:59.290 of linear algebra, they're the building blocks 00:13:59.290 --> 00:14:00.930 of data science. 00:14:00.930 --> 00:14:03.100 They're rank one matrices. 00:14:03.100 --> 00:14:07.390 And let me show you a special way to write those rank one 00:14:07.390 --> 00:14:10.010 matrices. 00:14:10.010 --> 00:14:15.100 I think of this matrix as the column vector 1, 1, 00:14:15.100 --> 00:14:19.720 1 times the row vector 1, 3, 8. 00:14:19.720 --> 00:14:23.050 So it's a column times a row. 00:14:25.860 --> 00:14:28.640 That's a rank one matrix. 00:14:28.640 --> 00:14:31.850 Do you see-- that's a true multiplication there. 00:14:31.850 --> 00:14:35.960 It looks a little weird, but it's a 3 by 1 matrix times a 1 00:14:35.960 --> 00:14:37.580 by 3 matrix. 00:14:37.580 --> 00:14:39.350 These numbers have to be the same, 00:14:39.350 --> 00:14:42.110 and then the output is 3 by 3. 00:14:42.110 --> 00:14:43.020 And it's that. 00:14:45.720 --> 00:14:48.780 And do you see that it factors? 00:14:48.780 --> 00:14:51.820 So I'm going to move on to that idea. 00:14:51.820 --> 00:15:01.600 The next idea coming up will be that we can see-- 00:15:01.600 --> 00:15:06.570 well, that's coming, that we're going to see matrices 00:15:06.570 --> 00:15:08.950 with two factors. 00:15:08.950 --> 00:15:13.630 Let me move to that, but back for this original matrix. 00:15:17.000 --> 00:15:19.380 So what's with this? 00:15:19.380 --> 00:15:21.860 The column space is a plane. 00:15:26.090 --> 00:15:30.110 Think about now the key idea of independent columns. 00:15:30.110 --> 00:15:33.777 How many independent columns have I got here? 00:15:33.777 --> 00:15:34.360 AUDIENCE: Two. 00:15:34.360 --> 00:15:35.740 GILBERT STRANG: Two, correct. 00:15:35.740 --> 00:15:36.970 Two. 00:15:36.970 --> 00:15:41.290 The third column, if I want to especially pick on that one-- 00:15:41.290 --> 00:15:42.850 I'll often go left to right. 00:15:42.850 --> 00:15:46.230 So I'll say the first guy is good, second guy is good, 00:15:46.230 --> 00:15:51.080 the third guy is not independent of the others. 00:15:51.080 --> 00:15:54.290 So I just have two independent columns. 00:15:54.290 --> 00:16:01.080 And those two columns would be a basis for the column space. 00:16:01.080 --> 00:16:04.280 So that's the critical idea of linear algebra. 00:16:04.280 --> 00:16:07.160 That's what you compute when you find a basis, 00:16:07.160 --> 00:16:09.910 and everything in the column space 00:16:09.910 --> 00:16:14.780 is a combination of these, including that is also-- 00:16:14.780 --> 00:16:16.430 that's already a combination of those. 00:16:16.430 --> 00:16:18.920 But everything else in the column space 00:16:18.920 --> 00:16:22.010 is a combination of those two. 00:16:22.010 --> 00:16:23.600 So they're a basis for it. 00:16:26.200 --> 00:16:31.060 So you have the idea of A times x. 00:16:31.060 --> 00:16:34.090 You have the idea of column space of A, 00:16:34.090 --> 00:16:36.470 which allows all x's. 00:16:36.470 --> 00:16:43.990 Then now, we're moving into the idea of independent columns, 00:16:43.990 --> 00:16:48.230 and the number of independent columns is the rank. 00:16:48.230 --> 00:16:51.640 So the rank-- shall I write that somewhere? 00:16:51.640 --> 00:16:52.630 Maybe here. 00:16:52.630 --> 00:16:55.765 The rank is the number of independent columns. 00:17:01.690 --> 00:17:04.880 And right now, what do I mean by independent columns? 00:17:04.880 --> 00:17:10.730 Well, let's just see what that means by using it. 00:17:10.730 --> 00:17:11.720 Are we good? 00:17:11.720 --> 00:17:13.520 I know I'm reviewing here. 00:17:13.520 --> 00:17:19.690 But allow me for this first class and part of next time 00:17:19.690 --> 00:17:20.589 also to review. 00:17:20.589 --> 00:17:21.970 But you'll see something new. 00:17:21.970 --> 00:17:24.220 In fact, why don't we see something new right away. 00:17:28.020 --> 00:17:32.520 Let me follow up on the idea of independence 00:17:32.520 --> 00:17:34.900 in a systematic way. 00:17:34.900 --> 00:17:41.370 So here's my matrix A. Can I write that again? 00:17:41.370 --> 00:17:50.430 2, 3, 5, 1, 1, 7, and this guy was the sum of those. 00:17:50.430 --> 00:17:53.970 So that's my matrix A. 00:17:53.970 --> 00:17:59.520 So let's start from scratch and find a basis for the column 00:17:59.520 --> 00:18:02.670 space in the most natural way. 00:18:02.670 --> 00:18:05.760 So I'm going to take a basis-- 00:18:05.760 --> 00:18:07.500 what's a basis? 00:18:07.500 --> 00:18:13.270 A basis is independent columns. 00:18:13.270 --> 00:18:16.840 So all three together would not be a basis. 00:18:16.840 --> 00:18:20.170 But they have to be not just independent, 00:18:20.170 --> 00:18:23.290 but they have to fill the space-- their combinations have 00:18:23.290 --> 00:18:24.760 to fill the space. 00:18:24.760 --> 00:18:27.580 So 2, 3, 5-- 00:18:27.580 --> 00:18:30.860 let's say I want to create a basis. 00:18:30.860 --> 00:18:35.080 I'll call the matrix C, a basis for the column space. 00:18:35.080 --> 00:18:37.460 So here's a natural way to do it. 00:18:37.460 --> 00:18:38.620 I look at the first column. 00:18:41.170 --> 00:18:42.370 It's not 0s. 00:18:42.370 --> 00:18:45.925 If it was all 0s, I wouldn't want that in a basis. 00:18:48.530 --> 00:18:49.760 But it's not. 00:18:49.760 --> 00:18:51.530 So I put it in. 00:18:51.530 --> 00:18:55.930 So that's the first vector in my basis. 00:18:55.930 --> 00:18:59.130 Then I go on to the second column. 00:18:59.130 --> 00:19:04.780 If that column was 4, 6, 10, what would I do? 00:19:04.780 --> 00:19:07.040 If that second column was 4, 6, 10, 00:19:07.040 --> 00:19:09.084 would I put it in the basis? 00:19:09.084 --> 00:19:11.060 No. 00:19:11.060 --> 00:19:13.100 But 1, 1, 7, is OK, right? 00:19:13.100 --> 00:19:15.390 1, 1, 7 is in a different direction. 00:19:15.390 --> 00:19:18.380 It's not a combination of what we've already got. 00:19:18.380 --> 00:19:20.980 So I say, OK, that adds something new. 00:19:20.980 --> 00:19:23.500 Put it in. 00:19:23.500 --> 00:19:26.590 Then I move on to the third column. 00:19:26.590 --> 00:19:29.730 Do I put that into a basis? 00:19:29.730 --> 00:19:31.110 You know the answer by now. 00:19:31.110 --> 00:19:32.130 No. 00:19:32.130 --> 00:19:37.140 Because I'm looking to see, is it a combination of these guys? 00:19:37.140 --> 00:19:38.640 And it is. 00:19:38.640 --> 00:19:40.650 It's one of that plus one of that. 00:19:40.650 --> 00:19:42.120 So it's not independent. 00:19:42.120 --> 00:19:45.730 So I've finished now. 00:19:45.730 --> 00:19:51.974 I've got a matrix C, which was taken directly from A, 00:19:51.974 --> 00:19:55.100 and I kept only independent columns 00:19:55.100 --> 00:19:58.360 and I worked from left to right. 00:19:58.360 --> 00:20:02.230 And I can see that right away now that the rank is two-- 00:20:02.230 --> 00:20:06.350 the column rank, I should say-- the column rank. 00:20:06.350 --> 00:20:08.490 The number of independent columns is two. 00:20:12.720 --> 00:20:13.220 Good? 00:20:16.060 --> 00:20:19.830 Now comes the key step. 00:20:19.830 --> 00:20:26.790 I'm going to produce a third matrix, R, 00:20:26.790 --> 00:20:32.090 which is going to tell me how to get these columns 00:20:32.090 --> 00:20:35.160 from these columns. 00:20:35.160 --> 00:20:37.290 And its shape is going to be-- 00:20:37.290 --> 00:20:40.140 well, its shape-- I don't have any choice. 00:20:40.140 --> 00:20:45.960 This is three by two, so Rx would be two by something. 00:20:45.960 --> 00:20:48.840 So I'm like so. 00:20:48.840 --> 00:20:51.360 I guess it has to be two by three because I want to come 00:20:51.360 --> 00:20:55.230 out this way, two by three. 00:20:55.230 --> 00:20:57.670 What am I going to do here? 00:20:57.670 --> 00:20:59.310 I'm just going to put it in the numbers 00:20:59.310 --> 00:21:00.555 R that make this correct. 00:21:04.150 --> 00:21:07.600 So this is a first matrix factorization. 00:21:07.600 --> 00:21:10.510 It's not-- well, it is a famous one, actually. 00:21:10.510 --> 00:21:14.930 When we see it, we'll recognize what it is. 00:21:14.930 --> 00:21:17.850 It's famous in teaching linear algebra, 00:21:17.850 --> 00:21:22.210 but now, actually, C times R, columns times rows 00:21:22.210 --> 00:21:28.120 has become very, very important in large scale 00:21:28.120 --> 00:21:31.490 numerical linear algebra. 00:21:31.490 --> 00:21:38.610 So let's figure out what goes into R. 00:21:38.610 --> 00:21:40.560 What am I thinking here? 00:21:40.560 --> 00:21:44.940 I'm putting in R. So every one of these columns 00:21:44.940 --> 00:21:46.350 is a combination of these. 00:21:46.350 --> 00:21:48.300 That's the whole point. 00:21:48.300 --> 00:21:52.080 And I'm just going to put in the numbers that you need. 00:21:52.080 --> 00:21:54.960 So what goes in the first column of R? 00:21:54.960 --> 00:21:57.300 What goes in the first column of R? 00:21:57.300 --> 00:21:59.460 So I want to look, what combination 00:21:59.460 --> 00:22:03.450 of that column and that column gives me this one? 00:22:03.450 --> 00:22:03.950 Yeah? 00:22:03.950 --> 00:22:04.795 AUDIENCE: 1, 0. 00:22:04.795 --> 00:22:05.670 GILBERT STRANG: 1, 0. 00:22:11.210 --> 00:22:16.750 Because you remember how we multiply a matrix by a vector? 00:22:16.750 --> 00:22:18.860 When I multiply that matrix by that vector, 00:22:18.860 --> 00:22:21.410 I take one of this plus zero of that. 00:22:21.410 --> 00:22:25.580 I see it vector-wise, and of course I get it right. 00:22:25.580 --> 00:22:28.460 And what about the second column of R? 00:22:28.460 --> 00:22:30.980 So the second column should be the combination 00:22:30.980 --> 00:22:34.820 that produces the second column of A correctly. 00:22:34.820 --> 00:22:35.720 What will that be? 00:22:35.720 --> 00:22:36.415 AUDIENCE: 0, 1. 00:22:36.415 --> 00:22:37.290 GILBERT STRANG: 0, 1. 00:22:37.290 --> 00:22:39.570 Thanks. 00:22:39.570 --> 00:22:41.736 And finally the third column? 00:22:41.736 --> 00:22:42.650 AUDIENCE: 1, 1. 00:22:42.650 --> 00:22:44.040 GILBERT STRANG: 1, 1. 00:22:44.040 --> 00:22:45.570 Yes, right. 00:22:45.570 --> 00:22:50.690 Because one of this plus one of this produces the third column. 00:22:50.690 --> 00:22:54.000 So all I did was put in the right numbers there, really. 00:23:03.220 --> 00:23:07.070 And this is correct now. 00:23:07.070 --> 00:23:12.380 A is C times R. And so this is-- 00:23:12.380 --> 00:23:15.590 what I've done here is the first two pages 00:23:15.590 --> 00:23:20.395 of section 1.1 in these notes. 00:23:20.395 --> 00:23:25.900 So 1.1. 00:23:25.900 --> 00:23:28.600 And actually, I'll tell you literally 00:23:28.600 --> 00:23:32.980 what happened earlier this year. 00:23:32.980 --> 00:23:34.510 I had finished this. 00:23:34.510 --> 00:23:38.830 I wrote this down with a different example. 00:23:38.830 --> 00:23:44.290 And then I realized something, that sitting here 00:23:44.290 --> 00:23:48.850 in front of me was the first great theorem 00:23:48.850 --> 00:23:55.030 in linear algebra, the fact that the column rank equals the row 00:23:55.030 --> 00:23:57.700 rank. 00:23:57.700 --> 00:24:03.980 The fact that if I have a matrix where 00:24:03.980 --> 00:24:06.750 that column plus that column gives that one. 00:24:06.750 --> 00:24:07.250 Oh. 00:24:10.360 --> 00:24:12.120 So what am I going to say here? 00:24:12.120 --> 00:24:14.920 I'm getting nervous about it. 00:24:14.920 --> 00:24:20.070 I believe that a combination of the rows gives 0. 00:24:20.070 --> 00:24:21.570 Do you believe that? 00:24:21.570 --> 00:24:22.830 You've got to believe it. 00:24:22.830 --> 00:24:24.120 This is linear algebra. 00:24:24.120 --> 00:24:28.410 The matrix is not invertible, it's square. 00:24:28.410 --> 00:24:32.020 But the columns are dependent. 00:24:32.020 --> 00:24:34.800 So the rows have to be dependent. 00:24:34.800 --> 00:24:38.230 And I don't exactly see-- 00:24:38.230 --> 00:24:41.280 there's some combination of that row and that row 00:24:41.280 --> 00:24:43.080 that gives that one. 00:24:43.080 --> 00:24:45.390 And of course, when I looked at the first column, 00:24:45.390 --> 00:24:49.980 I thought, OK, it's going to be too easy. 00:24:49.980 --> 00:24:51.810 One of that and one of that gives that. 00:24:51.810 --> 00:24:55.320 But then my eye went over to the second column, 00:24:55.320 --> 00:24:57.850 and I realized it's not easy at all. 00:24:57.850 --> 00:25:03.810 So you're entitled to pull out your phone and figure this out. 00:25:03.810 --> 00:25:06.894 But there is some damn combination of those-- 00:25:06.894 --> 00:25:08.136 [LAUGHTER] 00:25:08.136 --> 00:25:11.220 --of those two rows that gives the third row. 00:25:11.220 --> 00:25:13.900 Otherwise, the course is over, we stop. 00:25:17.720 --> 00:25:20.580 Well, and maybe we're going to find somehow. 00:25:24.200 --> 00:25:26.050 So this is the theorem. 00:25:26.050 --> 00:25:30.880 So I have to tell you, I was really pleased. 00:25:30.880 --> 00:25:34.450 So the first two pages got two more pages 00:25:34.450 --> 00:25:36.910 to follow up on that idea, that here, we 00:25:36.910 --> 00:25:38.710 were seeing something that-- 00:25:38.710 --> 00:25:45.400 I proved in 18.06, but not in the first lecture, that's 00:25:45.400 --> 00:25:49.330 for sure, and not maybe so clearly. 00:25:49.330 --> 00:25:54.310 But now I can try to prove it. 00:25:54.310 --> 00:25:56.890 There won't be a lot of proofs in this class, 00:25:56.890 --> 00:25:59.650 but this is such an important fact-- 00:25:59.650 --> 00:26:02.590 A equals CR is an important factorization. 00:26:02.590 --> 00:26:06.730 And out of it, we can connect them. 00:26:06.730 --> 00:26:08.650 So what am I saying? 00:26:08.650 --> 00:26:12.445 I am saying that all-- 00:26:16.630 --> 00:26:18.550 so what's the row rank? 00:26:18.550 --> 00:26:20.110 I have to back up here. 00:26:20.110 --> 00:26:22.690 What's the row rank? 00:26:22.690 --> 00:26:24.160 What's the row space? 00:26:24.160 --> 00:26:29.740 So the row rank is going to be the dimension of that space. 00:26:29.740 --> 00:26:33.040 So I look at my matrix A. What's the row space of A? 00:26:36.000 --> 00:26:38.590 I'm going to look at its rows. 00:26:38.590 --> 00:26:42.760 Now, maybe just so we don't get whole new letters 00:26:42.760 --> 00:26:43.750 for the row space-- 00:26:46.960 --> 00:26:52.540 for me, the row space of A of a matrix-- 00:26:52.540 --> 00:26:56.180 so first of all, tell me in words what it is. 00:26:56.180 --> 00:26:59.072 What's the row space of the matrix? 00:26:59.072 --> 00:27:00.280 AUDIENCE: [INAUDIBLE] 00:27:00.280 --> 00:27:04.960 GILBERT STRANG: All combinations of the rows. 00:27:04.960 --> 00:27:09.520 All combinations of the rows, that's the space. 00:27:09.520 --> 00:27:13.960 So I would take all combinations of those rows. 00:27:13.960 --> 00:27:16.930 To get combinations of the rows-- 00:27:16.930 --> 00:27:19.630 well, two ways I can get combinations of the rows. 00:27:19.630 --> 00:27:25.250 And the way I'll do it is, I'll just transpose the matrix. 00:27:25.250 --> 00:27:27.730 So those rows become columns, and then I'm 00:27:27.730 --> 00:27:29.770 back to what I did. 00:27:29.770 --> 00:27:36.850 So the row space of A is the column space of A transpose. 00:27:36.850 --> 00:27:38.920 And this has the advantage that we 00:27:38.920 --> 00:27:41.260 don't introduce a new letter. 00:27:41.260 --> 00:27:45.560 So it'll be the column space of A transpose. 00:27:48.140 --> 00:27:49.780 So we don't introduce a new letter. 00:27:49.780 --> 00:27:52.600 We keep the convention that vectors 00:27:52.600 --> 00:27:57.990 are column vectors, which would be the MATLAB and Julia 00:27:57.990 --> 00:28:03.730 and Python convention. 00:28:03.730 --> 00:28:05.290 So is that OK? 00:28:05.290 --> 00:28:08.020 The row space is the combination of these rows. 00:28:08.020 --> 00:28:10.840 But I'll flip-- I'll make them stand up-- 00:28:10.840 --> 00:28:15.470 2, 1, 3-- to be column vectors. 00:28:15.470 --> 00:28:17.630 So it's a totally different space. 00:28:17.630 --> 00:28:22.520 And actually, I happened to take a three by three example, 00:28:22.520 --> 00:28:27.320 so that the column space is part of R3 00:28:27.320 --> 00:28:29.780 and the row space is also part of R3. 00:28:29.780 --> 00:28:32.270 Because my matrix is three by three. 00:28:32.270 --> 00:28:36.980 A better example-- and the whole point of data-- data 00:28:36.980 --> 00:28:39.770 doesn't come in square matrices. 00:28:39.770 --> 00:28:44.420 Fortunately for us, data very, very often comes in matrices. 00:28:44.420 --> 00:28:48.125 But the two-- the columns might be sample, 00:28:48.125 --> 00:28:51.050 it might be patients, and the rows 00:28:51.050 --> 00:28:54.530 might be diseases or something. 00:28:54.530 --> 00:28:56.540 They're different spaces. 00:28:56.540 --> 00:29:03.820 So matrices are not likely to be square. 00:29:03.820 --> 00:29:07.120 But anyway, we're good here. 00:29:07.120 --> 00:29:09.640 So the row space. 00:29:09.640 --> 00:29:11.310 Now can I come back to the proof? 00:29:11.310 --> 00:29:14.170 Because what I want to say is that the proof 00:29:14.170 --> 00:29:17.800 of this fundamental fact is staring at us, 00:29:17.800 --> 00:29:19.860 but we don't quite see it yet. 00:29:23.150 --> 00:29:25.620 And I want to see it. 00:29:25.620 --> 00:29:37.745 So I claim that these rows are a basis for the row space. 00:29:40.690 --> 00:29:45.450 And we already saw that these columns are 00:29:45.450 --> 00:29:48.690 a basis for the column space. 00:29:53.450 --> 00:29:55.940 And two equals two, right? 00:29:55.940 --> 00:29:59.740 Two vectors here were a basis for the column space. 00:29:59.740 --> 00:30:03.820 Now if I can see why it shows me that these two vectors are 00:30:03.820 --> 00:30:08.110 a basis for the row space, then my example 00:30:08.110 --> 00:30:11.830 is right, that both of these will give two. 00:30:11.830 --> 00:30:14.320 The column rank is two, two columns, 00:30:14.320 --> 00:30:16.210 the row rank is two, two rows. 00:30:21.200 --> 00:30:27.050 I have to explain why I believe that these two rows are 00:30:27.050 --> 00:30:31.040 a basis for the row space. 00:30:31.040 --> 00:30:31.822 Are you with me? 00:30:31.822 --> 00:30:32.530 I have to prove-- 00:30:32.530 --> 00:30:34.210 I have to see why. 00:30:34.210 --> 00:30:39.240 First, so when I say basis, what do I have to check? 00:30:39.240 --> 00:30:45.660 Basis is the critical idea. 00:30:45.660 --> 00:30:49.060 I have to check that they're independent, so I 00:30:49.060 --> 00:30:50.770 haven't got too many vectors-- 00:30:50.770 --> 00:30:53.290 I haven't got any extra vectors in there. 00:30:53.290 --> 00:30:57.760 And I also have to check that their combinations produce--? 00:31:01.080 --> 00:31:01.830 All the rows. 00:31:04.570 --> 00:31:05.670 Should I say that again? 00:31:05.670 --> 00:31:07.350 Because that's what I'm going to check. 00:31:07.350 --> 00:31:09.475 I'm going to check that those guys are independent. 00:31:09.475 --> 00:31:11.458 Well, you can see that they're independent. 00:31:14.330 --> 00:31:19.070 And I'm going to check that their combinations produce 00:31:19.070 --> 00:31:22.370 all three of these rows. 00:31:22.370 --> 00:31:27.200 We didn't create those numbers for this purpose, 00:31:27.200 --> 00:31:31.150 but what I'm saying is they work. 00:31:31.150 --> 00:31:35.860 So I claim that this is a basis, because what combination 00:31:35.860 --> 00:31:40.330 of those two rows would produce this first row? 00:31:40.330 --> 00:31:44.110 Yeah, let me just ask you that. 00:31:44.110 --> 00:31:47.440 What numbers should I multiply these two rows by 00:31:47.440 --> 00:31:49.410 to get the first row of A? 00:31:49.410 --> 00:31:50.410 AUDIENCE: 2, 1. 00:31:50.410 --> 00:31:52.630 GILBERT STRANG: 2 and 1. 00:31:52.630 --> 00:31:56.510 And where do you find 2 and 1? 00:31:56.510 --> 00:32:04.550 It's sitting there in C. Will it work again? 00:32:04.550 --> 00:32:10.720 Does three of this plus one of that give 3, 1, 4? 00:32:10.720 --> 00:32:12.190 Yes. 00:32:12.190 --> 00:32:13.930 So far, so good. 00:32:13.930 --> 00:32:19.660 Does five of this and seven of that-- see, I'm multiplying-- 00:32:19.660 --> 00:32:23.560 I guess I'm doing matrix multiplication a backward way 00:32:23.560 --> 00:32:25.120 or a different way. 00:32:25.120 --> 00:32:31.300 I'm taking combinations of the rows of the second guy. 00:32:31.300 --> 00:32:33.430 The wonderful thing about matrix multiplication 00:32:33.430 --> 00:32:36.220 is you can do it a lot of ways, it comes out 00:32:36.220 --> 00:32:40.590 the same every way, and each way tells you something. 00:32:40.590 --> 00:32:44.950 So five of that row plus seven of that row, sure enough, 00:32:44.950 --> 00:32:45.450 is here. 00:32:48.240 --> 00:32:50.595 Do you see that that is not accident? 00:32:54.040 --> 00:32:57.100 The proof is really to look at this multiplication, 00:32:57.100 --> 00:32:59.680 C times R, two ways. 00:32:59.680 --> 00:33:02.800 Look at it first as combinations of columns of C 00:33:02.800 --> 00:33:04.380 to give the columns. 00:33:04.380 --> 00:33:09.100 Look at it second to get as combinations of the rows of R, 00:33:09.100 --> 00:33:11.750 and that produces the rows. 00:33:11.750 --> 00:33:19.910 So that factorization A equals CR was the key idea. 00:33:25.020 --> 00:33:32.690 And actually, this R that we've come up with has a name. 00:33:32.690 --> 00:33:36.770 Anybody remember enough 18.06? 00:33:36.770 --> 00:33:39.690 Have you all taken 18.06? 00:33:39.690 --> 00:33:40.310 No. 00:33:40.310 --> 00:33:43.660 I see-- how many have? 00:33:43.660 --> 00:33:44.240 Yes. 00:33:44.240 --> 00:33:44.940 OK. 00:33:44.940 --> 00:33:45.440 Good. 00:33:48.100 --> 00:33:52.540 For a while, 18.06 was taught in a very abstract way. 00:33:52.540 --> 00:33:53.980 I said, what's going on? 00:33:53.980 --> 00:33:58.120 But anyway, so if you took it that semester, 00:33:58.120 --> 00:34:01.210 you maybe never heard of the column space. 00:34:01.210 --> 00:34:02.900 I'm not sure. 00:34:02.900 --> 00:34:04.630 Or by a different name. 00:34:04.630 --> 00:34:06.070 It has another name. 00:34:06.070 --> 00:34:07.840 What's its other name? 00:34:07.840 --> 00:34:09.579 The column space of a matrix? 00:34:12.280 --> 00:34:16.010 Range-- I think it's the range. 00:34:16.010 --> 00:34:16.600 Yeah. 00:34:16.600 --> 00:34:21.460 And of course, all this is fundamental in mathematics. 00:34:21.460 --> 00:34:27.909 So of course, everything here has different languages 00:34:27.909 --> 00:34:30.190 and different emphasis. 00:34:30.190 --> 00:34:33.310 But you see what the emphasis is here. 00:34:33.310 --> 00:34:38.080 So you see the proof, that A equals CR just 00:34:38.080 --> 00:34:40.560 reveals everything. 00:34:40.560 --> 00:34:45.199 So it's our first idea of a factorization of a matrix. 00:34:45.199 --> 00:34:51.360 And we've multiplied C times R. I could just say-- 00:34:51.360 --> 00:34:55.980 so really, you've seen now the main point of Section 1.1 00:34:55.980 --> 00:35:00.750 of the notes to come up with that factorization 00:35:00.750 --> 00:35:01.980 and that conclusion. 00:35:05.020 --> 00:35:09.280 And you see why C has the same number of-- 00:35:09.280 --> 00:35:13.000 the number of columns of C equals the number of rows of R, 00:35:13.000 --> 00:35:16.510 and those are the column rank and the row rank. 00:35:16.510 --> 00:35:18.970 Yeah, it's just pretty neat. 00:35:18.970 --> 00:35:24.670 And here was the special case where those the column space is 00:35:24.670 --> 00:35:27.730 all multiples of U-- it's a line through U. 00:35:27.730 --> 00:35:30.490 The row space is all multiples of V-- 00:35:30.490 --> 00:35:34.200 it's a line through V. And that's the basic building 00:35:34.200 --> 00:35:34.700 block. 00:35:37.870 --> 00:35:39.820 Can I just say another little word 00:35:39.820 --> 00:35:43.310 before I push on beyond CR? 00:35:43.310 --> 00:35:45.970 That this has become-- 00:35:45.970 --> 00:35:51.800 if you have a giant matrix, like size 10 to the 5th, 00:35:51.800 --> 00:35:54.630 you can't put that into fast memory. 00:35:54.630 --> 00:35:56.870 It's a mess. 00:35:56.870 --> 00:35:59.360 How do you deal with a matrix of size 10 to the 5th, 00:35:59.360 --> 00:36:02.670 when you cannot deal with all the entries? 00:36:02.670 --> 00:36:06.030 That's just not possible. 00:36:06.030 --> 00:36:08.230 Well, you sample it. 00:36:08.230 --> 00:36:10.480 So later in the course, we'll be doing 00:36:10.480 --> 00:36:12.970 random sampling of a matrix. 00:36:12.970 --> 00:36:16.460 So how could you sample a matrix? 00:36:16.460 --> 00:36:17.900 So you have a matrix. 00:36:17.900 --> 00:36:21.440 Of course you're looking at it, but it's-- 00:36:21.440 --> 00:36:26.920 and you want to get some typical columns. 00:36:26.920 --> 00:36:28.240 Here's the natural idea. 00:36:30.760 --> 00:36:33.640 You just look at A times x. 00:36:33.640 --> 00:36:37.900 Let x be a random vector. 00:36:37.900 --> 00:36:43.810 Rand of-- so it's got m rows and one column. 00:36:43.810 --> 00:36:44.440 It's a vector. 00:36:48.880 --> 00:36:51.410 And what can I say about Ax? 00:36:54.080 --> 00:36:56.623 It's in the-- what space is it in? 00:36:56.623 --> 00:36:57.450 AUDIENCE: Column. 00:36:57.450 --> 00:36:58.200 GILBERT STRANG: Column space. 00:36:58.200 --> 00:36:58.740 Thanks. 00:36:58.740 --> 00:37:02.380 That was the first idea in this lecture. 00:37:02.380 --> 00:37:04.120 Ax is in the column space. 00:37:04.120 --> 00:37:06.930 So if you want a random vector in the column space, 00:37:06.930 --> 00:37:09.210 I wouldn't suggest to just randomly pick 00:37:09.210 --> 00:37:11.170 one of the columns. 00:37:11.170 --> 00:37:14.440 Better to take a mixture of columns 00:37:14.440 --> 00:37:18.520 by taking a random vector x, and looking at Ax. 00:37:18.520 --> 00:37:20.680 And if you wanted 100 random vectors, 00:37:20.680 --> 00:37:25.540 you'd take a 100 random x's, and that would give you 00:37:25.540 --> 00:37:30.460 a pretty good idea, in many cases, of what the column 00:37:30.460 --> 00:37:32.290 space looks like. 00:37:32.290 --> 00:37:34.410 That would be enough to work with often. 00:37:37.390 --> 00:37:40.370 Can I just throw in another question? 00:37:40.370 --> 00:37:44.030 So Ax is in the column space of A. 00:37:44.030 --> 00:37:46.060 Let me just ask you this question. 00:37:46.060 --> 00:37:55.000 Is ABCx-- is that in the column space of A? 00:37:55.000 --> 00:37:59.710 Suppose I have matrices A, B, and C, and a vector x, 00:37:59.710 --> 00:38:01.540 and I take their product. 00:38:01.540 --> 00:38:04.420 Does that give me something in the column space of A? 00:38:04.420 --> 00:38:05.710 AUDIENCE: Yes. 00:38:05.710 --> 00:38:07.870 GILBERT STRANG: Yes, good for you. 00:38:07.870 --> 00:38:09.126 How do you know that? 00:38:09.126 --> 00:38:11.360 AUDIENCE: [INAUDIBLE] 00:38:11.360 --> 00:38:14.800 GILBERT STRANG: Yeah, it's A times something. 00:38:14.800 --> 00:38:16.120 Right. 00:38:16.120 --> 00:38:18.100 Putting parentheses in the right place 00:38:18.100 --> 00:38:20.050 is the key to linear algebra. 00:38:20.050 --> 00:38:23.082 And there it is. 00:38:23.082 --> 00:38:24.790 It's a question that just occurred to me. 00:38:24.790 --> 00:38:29.040 And I thought, well, I wonder if you'd do it. 00:38:29.040 --> 00:38:34.210 So we have still time to multiply matrices. 00:38:34.210 --> 00:38:36.840 Oh, I was going to say about C and R-- 00:38:36.840 --> 00:38:42.660 so these are real columns from A. But R is-- 00:38:42.660 --> 00:38:47.370 the rows are not taken directly from the rows of A. Actually, 00:38:47.370 --> 00:38:49.200 there is a name for this. 00:38:49.200 --> 00:38:53.580 It's called the row reduced echelon form of the matrix, 00:38:53.580 --> 00:38:57.880 and it's a big goal in 18.06. 00:38:57.880 --> 00:39:02.970 It has the identity there, and then the other columns 00:39:02.970 --> 00:39:04.770 tell you the right combinations. 00:39:08.510 --> 00:39:15.500 Another big factorization would be to take columns from A-- 00:39:15.500 --> 00:39:17.680 so this is another-- 00:39:17.680 --> 00:39:19.070 so I'll put maybe or-- 00:39:22.630 --> 00:39:26.810 and we won't be doing this for a month. 00:39:26.810 --> 00:39:31.040 We could start to take columns of A 00:39:31.040 --> 00:39:34.550 and put them into C, if they were independent. 00:39:34.550 --> 00:39:37.910 And suppose I took rows of A. Now I'm 00:39:37.910 --> 00:39:43.850 going to take literally rows of A, and put them into R-- 00:39:43.850 --> 00:39:47.200 well, shall I call it R twiddle or something, 00:39:47.200 --> 00:39:51.260 because it'll be a different R. I'm not 00:39:51.260 --> 00:39:55.850 going to use those rows, but I'm going to take two 00:39:55.850 --> 00:39:59.630 actual rows of A. Then what? 00:39:59.630 --> 00:40:01.260 So that's an important factorization, 00:40:01.260 --> 00:40:02.135 but it's not correct. 00:40:05.230 --> 00:40:09.620 If I took two other rows of R, it wouldn't work. 00:40:09.620 --> 00:40:12.430 So you have to put it in the middle some two 00:40:12.430 --> 00:40:17.410 by two matrix U that makes it correct. 00:40:17.410 --> 00:40:24.310 You'll see in that Section 1.1 that I got excited 00:40:24.310 --> 00:40:28.000 and wrote a page about CUR. 00:40:28.000 --> 00:40:32.160 Yeah, so I'll just mention that. 00:40:32.160 --> 00:40:34.770 So now I'm ready-- oh, I wanted to say something 00:40:34.770 --> 00:40:35.540 about the course. 00:40:38.070 --> 00:40:43.210 I got excited thinking about math, but there is this course. 00:40:43.210 --> 00:40:44.080 So what's up? 00:40:46.930 --> 00:40:50.820 So there'll be linear algebra problems. 00:40:50.820 --> 00:40:55.810 But what makes this course special is the other homeworks, 00:40:55.810 --> 00:40:58.600 which are online. 00:40:58.600 --> 00:41:00.235 And you would use-- 00:41:03.090 --> 00:41:03.590 let's see. 00:41:03.590 --> 00:41:07.910 In principle, you could use any of the languages-- 00:41:07.910 --> 00:41:12.990 MATLAB, Python, which has become the biggest-- 00:41:12.990 --> 00:41:16.740 most used for deep learning, or Julia, 00:41:16.740 --> 00:41:18.705 which is the hot, new language. 00:41:22.910 --> 00:41:28.900 So last time, last year, the problems-- 00:41:28.900 --> 00:41:32.290 oh, so what happened last year? 00:41:32.290 --> 00:41:35.410 Well, everything in this course is 00:41:35.410 --> 00:41:40.090 owed to a professor who visited from the University 00:41:40.090 --> 00:41:41.740 of Michigan, Professor Rao-- 00:41:41.740 --> 00:41:47.490 Raj Rao, who gave most of the lectures a year ago, 00:41:47.490 --> 00:41:50.580 brought these homework problems-- 00:41:50.580 --> 00:41:52.770 online homework problems, so that people 00:41:52.770 --> 00:41:55.950 brought laptops to class and we did things in class. 00:41:58.650 --> 00:42:02.990 So he had and has a very successful course 00:42:02.990 --> 00:42:07.500 at EE in Michigan. 00:42:07.500 --> 00:42:10.950 But he was a PhD from here, and he came back on a sabbatical, 00:42:10.950 --> 00:42:13.980 and he created this-- got us started. 00:42:13.980 --> 00:42:15.650 And we really owe a lot to him. 00:42:19.130 --> 00:42:23.850 Also, Professor Edelman was involved with this course. 00:42:23.850 --> 00:42:26.310 And you maybe know that he created Julia. 00:42:26.310 --> 00:42:29.040 How many know what Julia is? 00:42:29.040 --> 00:42:30.660 Oh, wonderful. 00:42:30.660 --> 00:42:31.920 That will make his day. 00:42:34.480 --> 00:42:37.050 He tells me every time I see him, Julia is good. 00:42:37.050 --> 00:42:38.910 And I tell him, I believe it. 00:42:38.910 --> 00:42:40.680 [LAUGHTER] 00:42:40.680 --> 00:42:44.220 Anyway-- and it's become-- 00:42:44.220 --> 00:42:47.520 Professor Johnson in 18.06 has used Julia. 00:42:47.520 --> 00:42:54.120 And every semester, Steven Johnson gives in the first week 00:42:54.120 --> 00:42:58.440 a tutorial on Julia. 00:42:58.440 --> 00:43:02.370 And so that's arranged, and I promised to tell you 00:43:02.370 --> 00:43:05.910 where and when that is. 00:43:05.910 --> 00:43:08.220 So I think if you don't know anything about Julia, 00:43:08.220 --> 00:43:09.360 try to go. 00:43:09.360 --> 00:43:11.250 It's in Stata. 00:43:11.250 --> 00:43:14.570 It's on Friday at 5:00-- 00:43:14.570 --> 00:43:16.140 5:00 to 7:00. 00:43:16.140 --> 00:43:22.545 So Julia from Professor Johnson. 00:43:25.320 --> 00:43:27.210 So he's done this multiple times. 00:43:27.210 --> 00:43:28.390 He's good at it. 00:43:32.280 --> 00:43:35.130 I don't think we know yet what-- 00:43:35.130 --> 00:43:39.090 I guess I'm hoping that you'll have an option to use 00:43:39.090 --> 00:43:41.170 any of the three languages. 00:43:41.170 --> 00:43:47.880 But the online thing that we give you was created in Julia. 00:43:47.880 --> 00:43:51.290 So professor Rao had to learn Julia last spring, 00:43:51.290 --> 00:43:53.160 and the class did, too. 00:43:53.160 --> 00:43:55.740 And there was a certain amount of bitching about it. 00:43:55.740 --> 00:43:57.990 [LAUGHTER] 00:43:57.990 --> 00:44:01.750 But I think, with maybe one exception, 00:44:01.750 --> 00:44:06.105 who still got an A, everybody was OK 00:44:06.105 --> 00:44:08.330 and was glad to learn Julia. 00:44:08.330 --> 00:44:12.330 And Professor Rao now uses Julia entirely. 00:44:12.330 --> 00:44:16.650 So he's creating a new on ramp with Julia. 00:44:16.650 --> 00:44:19.920 MATLAB, by the way, has just issued an on ramp 00:44:19.920 --> 00:44:22.450 to deep learning that I'll tell you about, 00:44:22.450 --> 00:44:25.380 and probably get a MATLAB-- 00:44:25.380 --> 00:44:30.630 somebody from Math Works to say something about it. 00:44:30.630 --> 00:44:36.960 So that's what's coming, that we don't quite know exactly how 00:44:36.960 --> 00:44:39.150 well organized those homeworks. 00:44:39.150 --> 00:44:43.720 We'll just take the first one and see what happens. 00:44:43.720 --> 00:44:48.990 So I'll certainly say more about homeworks when-- 00:44:48.990 --> 00:44:50.610 maybe even Friday. 00:44:50.610 --> 00:44:54.282 But are there other questions that I should answer? 00:44:54.282 --> 00:44:55.990 Because some people will be thinking, OK, 00:44:55.990 --> 00:45:00.180 am I going to do this or am I going to sit in 6.036, 00:45:00.180 --> 00:45:03.960 or some other course in deep learning? 00:45:03.960 --> 00:45:06.090 Anything on your mind? 00:45:06.090 --> 00:45:08.910 And you can email me. 00:45:08.910 --> 00:45:12.310 So we will have a Stellar site and you'll see all the-- 00:45:12.310 --> 00:45:14.550 the TAs are still to be named. 00:45:14.550 --> 00:45:17.970 But the wonderful thing is that the undergraduates who 00:45:17.970 --> 00:45:22.080 took this course last year are volunteering 00:45:22.080 --> 00:45:25.170 to be graders for you guys. 00:45:25.170 --> 00:45:30.990 So they will know what those online homeworks were about. 00:45:30.990 --> 00:45:37.860 So that's a first word about what's coming 00:45:37.860 --> 00:45:41.670 and about the language. 00:45:41.670 --> 00:45:47.160 I'm going to finish by a very important topic, 00:45:47.160 --> 00:45:51.050 multiplying A times B. Oh, look, a clean board. 00:45:51.050 --> 00:45:57.110 So now I want to multiply a matrix by a vector. 00:45:57.110 --> 00:46:00.680 Everybody knows how to do it. 00:46:00.680 --> 00:46:07.650 You take a row of A-- so you take a row of A 00:46:07.650 --> 00:46:13.330 and you take a column of B, and you take the dot product. 00:46:13.330 --> 00:46:14.720 So you get a dot product. 00:46:17.480 --> 00:46:18.380 Row dot column. 00:46:24.310 --> 00:46:30.180 That's, again, low level, OK for beginners. 00:46:30.180 --> 00:46:36.510 But we want to see that matrix multiplication, AB, in a deeper 00:46:36.510 --> 00:46:38.250 way. 00:46:38.250 --> 00:46:42.840 And the deeper way is columns times row. 00:46:42.840 --> 00:46:48.060 Columns of A, rows of B. Columns times row. 00:46:48.060 --> 00:46:49.950 Oh, we had a column times a row. 00:46:52.690 --> 00:46:54.355 That was this rank one example. 00:46:57.130 --> 00:47:00.040 We had a column times a row, and it produced a matrix. 00:47:00.040 --> 00:47:02.040 And that's what it looked like. 00:47:02.040 --> 00:47:04.600 And its rank was one. 00:47:04.600 --> 00:47:05.890 So those are what-- 00:47:10.260 --> 00:47:11.850 so it's a combination of. 00:47:14.520 --> 00:47:16.740 It's very like Ax. 00:47:16.740 --> 00:47:20.040 I'm really just extending the Ax idea to AB. 00:47:22.550 --> 00:47:25.340 So this is the old way. 00:47:25.340 --> 00:47:26.750 The new way is columns. 00:47:26.750 --> 00:47:32.150 So there's column K. It will multiply. 00:47:32.150 --> 00:47:36.980 Sure enough, it multiplies row K. Everybody sees 00:47:36.980 --> 00:47:39.420 that it will happen that way. 00:47:39.420 --> 00:47:41.210 If you do it the old way, when you 00:47:41.210 --> 00:47:44.540 do a dot product of something here, something here, 00:47:44.540 --> 00:47:46.280 you're doing these multiplications. 00:47:46.280 --> 00:47:50.000 And when you hit column K here, you hit row K there. 00:47:50.000 --> 00:47:51.942 So these are connected. 00:47:56.370 --> 00:48:08.130 So I get things like column K of A times row K of B. 00:48:08.130 --> 00:48:10.010 And I don't know what notation to use, 00:48:10.010 --> 00:48:14.430 so I just wrote the words. 00:48:14.430 --> 00:48:21.000 But now that's one piece of the final answer, AB. 00:48:21.000 --> 00:48:22.650 That's a rank one piece. 00:48:22.650 --> 00:48:24.570 That's a building block. 00:48:24.570 --> 00:48:31.920 So I add from K equals 1 column one times row one. 00:48:31.920 --> 00:48:35.150 Column one of A times row one of B plus-- 00:48:35.150 --> 00:48:38.380 da, da, da, plus column K plus-- 00:48:38.380 --> 00:48:48.500 and of course, I stop at column n of A times row n of B. 00:48:48.500 --> 00:48:51.890 So it's a sum of outer products. 00:48:54.560 --> 00:48:57.650 So everybody sees a sum because I 00:48:57.650 --> 00:49:05.690 have column one times row 1, column K times row K, 00:49:05.690 --> 00:49:10.720 column n times row n, and then I add those pieces. 00:49:10.720 --> 00:49:17.440 It's just the generalization of Ax to a matrix B there. 00:49:17.440 --> 00:49:30.310 So it's a sum of Column K row K of A row K of B. And maybe, 00:49:30.310 --> 00:49:34.780 should we check that that gives us the right answer? 00:49:34.780 --> 00:49:38.350 I won't do that here, but all we're doing 00:49:38.350 --> 00:49:44.210 is the same multiplications in a different order. 00:49:44.210 --> 00:49:46.550 Actually, let's just quit with one minute. 00:49:46.550 --> 00:49:49.430 We can figure out how many multiplications are there. 00:49:49.430 --> 00:49:52.760 How many more applications to do an m by n 00:49:52.760 --> 00:49:58.570 matrix A times an n by p matrix B? 00:49:58.570 --> 00:50:02.610 So that's A times B. How many individual numbers-- 00:50:02.610 --> 00:50:05.180 because this would determine the cost of it. 00:50:05.180 --> 00:50:09.870 How many numbers would we need? 00:50:09.870 --> 00:50:18.240 Well, suppose we do it the old way, by inner product, 00:50:18.240 --> 00:50:19.440 row times column. 00:50:19.440 --> 00:50:23.460 So how many multiplications to do a row times column 00:50:23.460 --> 00:50:26.580 and get one entry in the answer? 00:50:26.580 --> 00:50:29.060 n, right? 00:50:29.060 --> 00:50:31.070 The row has length n, the column has 00:50:31.070 --> 00:50:33.860 length n, n multiplications. 00:50:33.860 --> 00:50:34.800 So that's n. 00:50:34.800 --> 00:50:38.150 And now how many of those do I have to do? 00:50:38.150 --> 00:50:39.950 AUDIENCE: mp. 00:50:39.950 --> 00:50:41.330 GILBERT STRANG: mp. 00:50:41.330 --> 00:50:44.120 Because what's the size of this answer? 00:50:44.120 --> 00:50:46.340 The size of this answer is m by p. 00:50:49.580 --> 00:50:54.225 So if I do it in that old order, like n multiplication 00:50:54.225 --> 00:50:56.310 is to do a dot product. 00:50:56.310 --> 00:50:59.250 And I've got this many dot products in the answer. 00:50:59.250 --> 00:51:02.540 So I've mnp multiplied. 00:51:07.630 --> 00:51:09.120 Now suppose I do it this way. 00:51:12.020 --> 00:51:16.440 How many multiplications to do one of those guys? 00:51:16.440 --> 00:51:18.450 To multiply our column by a row? 00:51:23.716 --> 00:51:29.291 This is an m by 1, and this is a 1 by p. 00:51:29.291 --> 00:51:30.980 One column, one row. 00:51:30.980 --> 00:51:34.390 How many multiplications for that guy? 00:51:34.390 --> 00:51:36.530 mp. 00:51:36.530 --> 00:51:40.718 And how many of those rank ones do I have to do? 00:51:40.718 --> 00:51:43.640 n. 00:51:43.640 --> 00:51:44.940 You got it? 00:51:44.940 --> 00:51:46.740 mp times n. 00:51:46.740 --> 00:51:56.580 Now, the other way was n times mp. 00:51:56.580 --> 00:52:00.330 So it gives the same answer, mnp multiplication. 00:52:00.330 --> 00:52:02.940 In fact, they're exactly the same multiplications, just 00:52:02.940 --> 00:52:04.000 a different order. 00:52:04.000 --> 00:52:04.500 OK. 00:52:04.500 --> 00:52:06.540 We're at 1:55. 00:52:06.540 --> 00:52:08.760 Thanks for coming today. 00:52:08.760 --> 00:52:11.640 I'll talk more about the class and about linear algebra 00:52:11.640 --> 00:52:12.570 on Friday. 00:52:12.570 --> 00:52:13.984 Thank you.