1 00:00:00 --> 00:00:01 2 00:00:01 --> 00:00:02 The following content is provided under a Creative 3 00:00:02 --> 00:00:03 Commons license. 4 00:00:03 --> 00:00:06 Your support will help MIT OpenCourseWare continue to 5 00:00:06 --> 00:00:10 offer high-quality educational resources for free. 6 00:00:10 --> 00:00:13 To make a donation or to view additional materials from 7 00:00:13 --> 00:00:16 hundreds of MIT courses, visit MIT OpenCourseWare 8 00:00:16 --> 00:00:22 at ocw.mit.edu. 9 00:00:22 --> 00:00:28 PROFESSOR STRANG: Just to give an overview in three lines: the 10 00:00:28 --> 00:00:32 text is the book of that name, Computational Science 11 00:00:32 --> 00:00:33 and Engineering. 12 00:00:33 --> 00:00:38 That was completed just last year, so it really ties 13 00:00:38 --> 00:00:41 pretty well with the course. 14 00:00:41 --> 00:00:43 I don't cover everything in the book, by all means. 15 00:00:43 --> 00:00:47 And I don't, certainly, don't stand here and read the book. 16 00:00:47 --> 00:00:50 That would be no good. 17 00:00:50 --> 00:00:55 But you'll be able, if you miss a class -- well, 18 00:00:55 --> 00:00:56 don't miss a class. 19 00:00:56 --> 00:01:01 But if you miss a class, you'll be able, probably, 20 00:01:01 --> 00:01:05 to see roughly what we did. 21 00:01:05 --> 00:01:08 OK, so the first part of the semester is applied 22 00:01:08 --> 00:01:10 linear algebra. 23 00:01:10 --> 00:01:13 And I don't know how many of you have had a linear algebra 24 00:01:13 --> 00:01:16 course, and that's why I thought I would start 25 00:01:16 --> 00:01:19 with a quick review. 26 00:01:19 --> 00:01:23 And you'll catch on. 27 00:01:23 --> 00:01:26 I want matrices to come to life, actually. 28 00:01:26 --> 00:01:31 You know, instead of just being a four by four array of 29 00:01:31 --> 00:01:34 numbers, there are four by four, or n by n or m by n 30 00:01:34 --> 00:01:36 array of special numbers. 31 00:01:36 --> 00:01:38 They have a meaning. 32 00:01:38 --> 00:01:41 When they multiply a vector, they do something. 33 00:01:41 --> 00:01:47 And so it's just part of this first step is just, like, 34 00:01:47 --> 00:01:50 getting to recognize, what's that matrix doing? 35 00:01:50 --> 00:01:52 Where does it come from? 36 00:01:52 --> 00:01:53 What are its properties? 37 00:01:53 --> 00:01:57 So that's a theme at the start. 38 00:01:57 --> 00:02:04 Then differential equations, like Laplace's equation, 39 00:02:04 --> 00:02:06 are beautiful examples. 40 00:02:06 --> 00:02:11 So here we get, especially, to numerical methods; 41 00:02:11 --> 00:02:14 finite differences, finite elements, above all. 42 00:02:14 --> 00:02:17 So I think in this class you'll really see how finite elements 43 00:02:17 --> 00:02:20 work, and other ideas. 44 00:02:20 --> 00:02:21 All sorts of ideas. 45 00:02:21 --> 00:02:25 And then the last part of the course is about Fourier. 46 00:02:25 --> 00:02:29 That's Fourier series, that you may have seen, and 47 00:02:29 --> 00:02:30 Fourier integrals. 48 00:02:30 --> 00:02:34 But also, highly important, Discrete Fourier 49 00:02:34 --> 00:02:36 Transform, DFT. 50 00:02:36 --> 00:02:40 That's a fundamental step for understanding what 51 00:02:40 --> 00:02:42 a signal contains. 52 00:02:42 --> 00:02:46 Yeah, so that's great stuff, Fourier. 53 00:02:46 --> 00:02:52 OK, what else should I say before I start? 54 00:02:52 --> 00:02:56 I said this was my favorite course, and maybe I 55 00:02:56 --> 00:03:01 elaborate a little. 56 00:03:01 --> 00:03:06 Well, I think what I want to say is that I really feel my 57 00:03:06 --> 00:03:12 life is here to teach you and not to grade you. 58 00:03:12 --> 00:03:15 I'm not going to spend this semester worrying about 59 00:03:15 --> 00:03:18 grades, and please don't. 60 00:03:18 --> 00:03:19 They come out fine. 61 00:03:19 --> 00:03:22 We've got lots to learn. 62 00:03:22 --> 00:03:26 And I'll do my very best to explain it clearly. 63 00:03:26 --> 00:03:30 And I know you'll do your best. 64 00:03:30 --> 00:03:31 I know from experience. 65 00:03:31 --> 00:03:36 This class goes for it and does it right. 66 00:03:36 --> 00:03:40 So that's what makes it so good. 67 00:03:40 --> 00:03:41 OK. 68 00:03:41 --> 00:03:46 Homeworks, by the way, well, the first homework will simply 69 00:03:46 --> 00:03:50 be a way to get a grade list, a list of everybody 70 00:03:50 --> 00:03:52 taking the course. 71 00:03:52 --> 00:03:55 They won't be graded in great detail. 72 00:03:55 --> 00:03:59 Too large a class. 73 00:03:59 --> 00:04:03 And you're allowed to talk to each other about homework. 74 00:04:03 --> 00:04:05 So homework is not an exam at all. 75 00:04:05 --> 00:04:09 So let me just leave any discussion of exams and 76 00:04:09 --> 00:04:12 grades for the future. 77 00:04:12 --> 00:04:14 I'll tell you, you'll see how informally the 78 00:04:14 --> 00:04:18 first homework will be. 79 00:04:18 --> 00:04:21 And I hope it'll go up on the website. 80 00:04:21 --> 00:04:23 The first homework will be for Monday. 81 00:04:23 --> 00:04:29 So it's a bit early, but it's pretty open-ended. 82 00:04:29 --> 00:04:33 If you could take three problems from 1.1, the first 83 00:04:33 --> 00:04:38 section of the book, any three, and any three problems from 84 00:04:38 --> 00:04:45 1.2, and print your name on the homework -- because we're going 85 00:04:45 --> 00:04:48 to use that to create the grade list -- I'll 86 00:04:48 --> 00:04:50 be completely happy. 87 00:04:50 --> 00:04:52 Well, especially if you get them right and do them 88 00:04:52 --> 00:04:53 neatly and so on. 89 00:04:53 --> 00:04:59 But actually we won't know. 90 00:04:59 --> 00:05:02 So that's for Monday. 91 00:05:02 --> 00:05:03 OK. 92 00:05:03 --> 00:05:05 And we'll talk more about it. 93 00:05:05 --> 00:05:11 I'll announce the TA on the website and the TA hours, the 94 00:05:11 --> 00:05:12 office hours, and everything. 95 00:05:12 --> 00:05:17 There'll be a Friday afternoon office hour, because homeworks 96 00:05:17 --> 00:05:20 will typically come Monday. 97 00:05:20 --> 00:05:20 OK. 98 00:05:20 --> 00:05:27 Questions about the course before I just start? 99 00:05:27 --> 00:05:30 OK. 100 00:05:30 --> 00:05:31 Another time for questions, too. 101 00:05:31 --> 00:05:41 OK, so can we just start with that matrix? 102 00:05:41 --> 00:05:45 So I said about matrices, I'm interested in their properties. 103 00:05:45 --> 00:05:47 Like, I'm going to ask you about that. 104 00:05:47 --> 00:05:51 And then, I'm interested in their meaning. 105 00:05:51 --> 00:05:53 Where do they come from? 106 00:05:53 --> 00:05:56 You know, why that matrix instead of some other? 107 00:05:56 --> 00:06:01 And then, the numerical part is how do we deal with them? 108 00:06:01 --> 00:06:05 How do we solve a linear system with that coefficient matrix? 109 00:06:05 --> 00:06:07 What can we say about the solution? 110 00:06:07 --> 00:06:09 So the purpose. 111 00:06:09 --> 00:06:10 Right. 112 00:06:10 --> 00:06:15 OK, now help me out. 113 00:06:15 --> 00:06:18 So I guess my plan with the video taping is, whatever 114 00:06:18 --> 00:06:20 you say, I'll repeat. 115 00:06:20 --> 00:06:27 So say it as clearly as possible, and it's fantastic 116 00:06:27 --> 00:06:30 to have discussion, conversation here. 117 00:06:30 --> 00:06:34 So I'll just repeat it so that it safely gets on the tape. 118 00:06:34 --> 00:06:35 So tell me its properties. 119 00:06:35 --> 00:06:41 Tell me the first property that you notice about that matrix. 120 00:06:41 --> 00:06:41 Symmetric. 121 00:06:41 --> 00:06:42 Symmetric. 122 00:06:42 --> 00:06:44 Right. 123 00:06:44 --> 00:06:46 I could have slowed down a little and everybody probably 124 00:06:46 --> 00:06:48 would have said that at once. 125 00:06:48 --> 00:06:51 So that's a symmetric matrix. 126 00:06:51 --> 00:06:54 Now we might as well pick up some matrix notation. 127 00:06:54 --> 00:06:58 How do I express the fact that this a symmetric matrix? 128 00:06:58 --> 00:07:03 In simple matrix notation, I would say that K is 129 00:07:03 --> 00:07:07 the same as K transpose. 130 00:07:07 --> 00:07:11 The transpose, everybody knows, it comes from -- oh, I 131 00:07:11 --> 00:07:14 shouldn't say this -- flipping it across the diagonal. 132 00:07:14 --> 00:07:17 That's not a very "math" thing to do. 133 00:07:17 --> 00:07:21 But that's the way to visualize it. 134 00:07:21 --> 00:07:27 And let me use a capital T for transpose. 135 00:07:27 --> 00:07:30 So it's symmetric. 136 00:07:30 --> 00:07:31 Very important. 137 00:07:31 --> 00:07:32 Very, very important. 138 00:07:32 --> 00:07:34 That's the most important class of matrices, 139 00:07:34 --> 00:07:35 symmetric matrices. 140 00:07:35 --> 00:07:39 We'll see them all the time, because they come from 141 00:07:39 --> 00:07:41 equilibrium problems. 142 00:07:41 --> 00:07:44 They come from all sorts of -- they come everywhere 143 00:07:44 --> 00:07:47 in applications. 144 00:07:47 --> 00:07:49 And we will be doing applications. 145 00:07:49 --> 00:07:55 The first week or week and a half, you'll see pretty much 146 00:07:55 --> 00:08:00 discussion of matrices and the reasons, what their meaning is. 147 00:08:00 --> 00:08:03 And then we'll get to physical applications; 148 00:08:03 --> 00:08:05 mechanics and more. 149 00:08:05 --> 00:08:06 OK. 150 00:08:06 --> 00:08:08 All right. 151 00:08:08 --> 00:08:11 Now I'm looking for properties, other 152 00:08:11 --> 00:08:13 properties, of that matrix. 153 00:08:13 --> 00:08:18 Let me write "2" here so that you got a spot to put it. 154 00:08:18 --> 00:08:21 What are you going to tell me next about that matrix? 155 00:08:21 --> 00:08:22 Periodic. 156 00:08:22 --> 00:08:23 Well, okay. 157 00:08:23 --> 00:08:25 Actually, that's a good question. 158 00:08:25 --> 00:08:31 Let me write periodic down here. 159 00:08:31 --> 00:08:36 You're using that word, because somehow that pattern is 160 00:08:36 --> 00:08:37 suggesting something. 161 00:08:37 --> 00:08:43 But you'll see I have a little more to add before I would 162 00:08:43 --> 00:08:44 use the word periodic. 163 00:08:44 --> 00:08:47 So that's great to see that here. 164 00:08:47 --> 00:08:47 What else? 165 00:08:47 --> 00:08:50 Somebody else was going to say something. 166 00:08:50 --> 00:08:51 Please. 167 00:08:51 --> 00:08:52 Sparse! 168 00:08:52 --> 00:08:53 Oh, very good. 169 00:08:53 --> 00:08:54 Sparse. 170 00:08:54 --> 00:08:59 That's also an obvious property that you see from 171 00:08:59 --> 00:09:01 looking at the matrix. 172 00:09:01 --> 00:09:03 What does sparse mean? 173 00:09:03 --> 00:09:05 Mostly zeros. 174 00:09:05 --> 00:09:07 Well that isn't mostly zeros, I guess. 175 00:09:07 --> 00:09:11 I mean, that's got what, out of sixteen entries, 176 00:09:11 --> 00:09:13 it's got six zeros. 177 00:09:13 --> 00:09:14 That doesn't sound like sparse. 178 00:09:14 --> 00:09:19 But when I grow the matrix -- because this is 179 00:09:19 --> 00:09:21 just a four by four. 180 00:09:21 --> 00:09:24 I would even call this one K_4. 181 00:09:24 --> 00:09:30 When the matrix grows to 100 by 100, then you really 182 00:09:30 --> 00:09:31 see it as sparse. 183 00:09:31 --> 00:09:35 So if that matrix was 100 by 100, how many 184 00:09:35 --> 00:09:37 non-zeros would it have? 185 00:09:37 --> 00:09:44 So if n is 100, then the number of non-zeros -- wow, that's the 186 00:09:44 --> 00:09:46 first MATLAB command I've written. 187 00:09:46 --> 00:09:51 A number of non-zeros of K would be -- anybody 188 00:09:51 --> 00:09:53 know what it would be? 189 00:09:53 --> 00:10:00 I'm just asking to go up to five by five. 190 00:10:00 --> 00:10:03 I'm asking you to keep that pattern alive. 191 00:10:03 --> 00:10:08 Twos on the diagonal, minus ones above and below. 192 00:10:08 --> 00:10:13 So yeah, so 298, would it be? 193 00:10:13 --> 00:10:20 A hundred diagonal entries, 99 and 99, maybe 298? 194 00:10:20 --> 00:10:27 298 out of 100 by 100 would be what? 195 00:10:27 --> 00:10:29 It's been a long summer. 196 00:10:29 --> 00:10:32 Yeah, a lot of zeros. 197 00:10:32 --> 00:10:32 A lot. 198 00:10:32 --> 00:10:33 Right. 199 00:10:33 --> 00:10:37 Because the matrix has got what 100 x 100, 10,000 entries. 200 00:10:37 --> 00:10:39 Out of 10,000. 201 00:10:39 --> 00:10:42 So that's sparse. 202 00:10:42 --> 00:10:46 But we see those all the time, and fortunately we do. 203 00:10:46 --> 00:10:48 Because, of course, this matrix, or even 100 204 00:10:48 --> 00:10:53 by 100, we could deal with if it was dense. 205 00:10:53 --> 00:10:58 But 10,000, 100,000, or 1 million, which happens 206 00:10:58 --> 00:11:02 all the time now in scientific computation. 207 00:11:02 --> 00:11:05 A million by million dense matrix is not a nice 208 00:11:05 --> 00:11:07 thing to think about. 209 00:11:07 --> 00:11:13 A million by million matrix like this is a cinch. 210 00:11:13 --> 00:11:14 OK. 211 00:11:14 --> 00:11:15 So sparse. 212 00:11:15 --> 00:11:18 What else do you want to say? 213 00:11:18 --> 00:11:19 Toeplitz. 214 00:11:19 --> 00:11:22 Holy Moses. 215 00:11:22 --> 00:11:23 Exactly right. 216 00:11:23 --> 00:11:29 But I want to say, before I use that word, so that'll be 217 00:11:29 --> 00:11:30 my second MATLAB command. 218 00:11:30 --> 00:11:31 Thanks. 219 00:11:31 --> 00:11:33 Toeplitz. 220 00:11:33 --> 00:11:35 What's that mean? 221 00:11:35 --> 00:11:40 So this matrix has a property that we see 222 00:11:40 --> 00:11:46 right away, which is? 223 00:11:46 --> 00:11:51 I want to stay with Toeplitz but everybody tell me something 224 00:11:51 --> 00:11:54 more about properties of that matrix. 225 00:11:54 --> 00:11:56 Tridiagonal. 226 00:11:56 --> 00:12:02 Tridiagonal, so that's almost a special subcase of sparse. 227 00:12:02 --> 00:12:05 It has just three diagonals. 228 00:12:05 --> 00:12:08 Tridiagonal matrices are truly important. 229 00:12:08 --> 00:12:11 They come in all the time, we'll see that they come from 230 00:12:11 --> 00:12:14 second order differential equations, which are, thanks 231 00:12:14 --> 00:12:17 to Newton, the big ones. 232 00:12:17 --> 00:12:23 Ok, now it's more than tridiagonal and what more? 233 00:12:23 --> 00:12:26 So what further, we're getting deeper now. 234 00:12:26 --> 00:12:32 What patterns do you see beyond just tridiagonal, because 235 00:12:32 --> 00:12:35 tridiagonal would allow any numbers there but those are 236 00:12:35 --> 00:12:39 not, there's more of a pattern than just three 237 00:12:39 --> 00:12:42 diagonals, what is it? 238 00:12:42 --> 00:12:45 Those diagonals are constant. 239 00:12:45 --> 00:12:48 If I run down each of those three diagonals, 240 00:12:48 --> 00:12:50 I see the same number. 241 00:12:50 --> 00:12:53 Twos, minus ones, minus ones, and that's what 242 00:12:53 --> 00:12:55 the word Toeplitz means. 243 00:12:55 --> 00:13:05 Toeplitz is constant diagonal. 244 00:13:05 --> 00:13:05 Ok. 245 00:13:05 --> 00:13:09 And that kind of matrix is so important. 246 00:13:09 --> 00:13:18 It corresponds, yeah, if we were in EE, I would use the 247 00:13:18 --> 00:13:23 words time invariant filter, linear time invariant. 248 00:13:23 --> 00:13:27 So it's linear because we're dealing with a matrix. 249 00:13:27 --> 00:13:31 And it's time invariant, shift invariant. 250 00:13:31 --> 00:13:36 I just use all these equivalent words to mean that we're seeing 251 00:13:36 --> 00:13:41 the same thing row by row, except of course, at shall 252 00:13:41 --> 00:13:44 I call that the boundary? 253 00:13:44 --> 00:13:46 That's like, the end of the system and this is like the 254 00:13:46 --> 00:13:51 other end and there it's chopped off. 255 00:13:51 --> 00:13:56 But if it was ten by ten I would see that row eight times. 256 00:13:56 --> 00:13:58 100 by 100 I'd see it 98 times. 257 00:13:58 --> 00:14:05 So it's constant diagonals and the guy who first studied 258 00:14:05 --> 00:14:08 that was Toeplitz. 259 00:14:08 --> 00:14:14 And we wouldn't need that great historical information except 260 00:14:14 --> 00:14:18 that MATLAB created a command to create that matrix. 261 00:14:18 --> 00:14:25 K, MATLAB is all set to create Toeplitz matrices. 262 00:14:25 --> 00:14:30 Yeah, so I'll have to put what MATLAB would put. 263 00:14:30 --> 00:14:37 I realize I'm already using the word MATLAB. 264 00:14:37 --> 00:14:42 I think that MATLAB language is really convenient to 265 00:14:42 --> 00:14:44 talk about linear algebra. 266 00:14:44 --> 00:14:46 And how many know MATLAB or have used it? 267 00:14:46 --> 00:14:49 Yeah. 268 00:14:49 --> 00:14:51 You know it better than I. 269 00:14:51 --> 00:14:56 I talk a good line with MATLAB but I, the code never runs. 270 00:14:56 --> 00:14:58 Never! 271 00:14:58 --> 00:15:02 I always forget some stupid semicolon. 272 00:15:02 --> 00:15:04 You may have had that experience. 273 00:15:04 --> 00:15:11 And I just want to say it now that there are other languages, 274 00:15:11 --> 00:15:14 and if you want to do homeworks and want to do your own work 275 00:15:14 --> 00:15:18 in other languages, that makes sense. 276 00:15:18 --> 00:15:23 So the older established alternatives were Mathematica 277 00:15:23 --> 00:15:29 and Maple and those two have symbolic, they can deal with 278 00:15:29 --> 00:15:33 algebra as well as numbers. 279 00:15:33 --> 00:15:34 But there are newer languages. 280 00:15:34 --> 00:15:37 I don't know if you know them. 281 00:15:37 --> 00:15:41 I just know my friends say, Yes they're terrific. 282 00:15:41 --> 00:15:46 Python is one. 283 00:15:46 --> 00:15:47 And R. 284 00:15:47 --> 00:15:51 I've just had a email saying, Tell your class about R. 285 00:15:51 --> 00:15:53 And others. 286 00:15:53 --> 00:15:59 Ok, so but we'll use MATLAB language because that's really 287 00:15:59 --> 00:16:01 a good common language. 288 00:16:01 --> 00:16:03 Ok, so what is a Toeplitz matrix? 289 00:16:03 --> 00:16:06 A Toeplitz matrix is one with constant diagonals. 290 00:16:06 --> 00:16:09 You could use the word time invariant, linear time 291 00:16:09 --> 00:16:11 invariant filter. 292 00:16:11 --> 00:16:16 And to create K, this is an 18.085 command. 293 00:16:16 --> 00:16:19 It's just set up for us. 294 00:16:19 --> 00:16:26 I can create K by telling the system the first row. 295 00:16:26 --> 00:16:32 Two, minus one, zero, zero. 296 00:16:32 --> 00:16:37 That would, then if it wasn't symmetric I would have to 297 00:16:37 --> 00:16:40 give the first column also. 298 00:16:40 --> 00:16:42 Toeplitz would be constant diagonal, it doesn't 299 00:16:42 --> 00:16:44 have to be symmetric. 300 00:16:44 --> 00:16:47 But if it's symmetric, then the first row and first column are 301 00:16:47 --> 00:16:50 the same vector, so I just have to give that vector. 302 00:16:50 --> 00:16:55 Okay, so that's the quickest way to create K. 303 00:16:55 --> 00:17:00 And of course, if it was bigger then I would, rather than 304 00:17:00 --> 00:17:08 writing 100 zeros, I could put zeros of 98 and one. 305 00:17:08 --> 00:17:09 Wouldn't I have to say that? 306 00:17:09 --> 00:17:11 Or is it one and 98? 307 00:17:11 --> 00:17:15 You see why it doesn't run. 308 00:17:15 --> 00:17:18 Well I guess I'm thinking of that as a row. 309 00:17:18 --> 00:17:19 I don't know. 310 00:17:19 --> 00:17:24 Anyway. 311 00:17:24 --> 00:17:27 I realize getting this videotaped means I'm supposed 312 00:17:27 --> 00:17:28 to get things right! 313 00:17:28 --> 00:17:31 Usually it's like, we'll get it right later. 314 00:17:31 --> 00:17:36 But anyway, that might work. 315 00:17:36 --> 00:17:37 Okay. 316 00:17:37 --> 00:17:39 So there's a command that you know. 317 00:17:39 --> 00:17:44 Zeros that creates a matrix of this size with all zeros. 318 00:17:44 --> 00:17:45 Okay. 319 00:17:45 --> 00:17:48 That would create the 100 by 100. 320 00:17:48 --> 00:17:48 Good. 321 00:17:48 --> 00:17:50 Ok. 322 00:17:50 --> 00:17:52 Oh, by the way, as long as we're speaking about 323 00:17:52 --> 00:17:55 computation I've gotta say something more. 324 00:17:55 --> 00:17:59 We said that the matrix is sparse. 325 00:17:59 --> 00:18:02 And this 100 by 100 matrix is certainly sparse. 326 00:18:02 --> 00:18:07 But if I create it this way, I've created all those zeros 327 00:18:07 --> 00:18:13 and if I ask MATLAB to work with that matrix, to square it 328 00:18:13 --> 00:18:19 or whatever, it would carry all those zeros and do all 329 00:18:19 --> 00:18:21 those zero computations. 330 00:18:21 --> 00:18:25 In other words, it would treat K like a dense matrix and it 331 00:18:25 --> 00:18:27 would just, it wouldn't know the zeros were there 332 00:18:27 --> 00:18:29 until it looked. 333 00:18:29 --> 00:18:33 So I just want to say that if you have really big systems 334 00:18:33 --> 00:18:38 Sparse MATLAB is the way to go. 335 00:18:38 --> 00:18:42 Because Sparse MATLAB keeps track only of the non-zeros. 336 00:18:42 --> 00:18:45 So it knows-- and their locations, of course. 337 00:18:45 --> 00:18:47 What the numbers are and their location. 338 00:18:47 --> 00:18:50 So I could create a sparse matrix out of that, 339 00:18:50 --> 00:18:53 like KS for K sparse. 340 00:18:53 --> 00:18:59 I think if I just did sparse(K) that would 341 00:18:59 --> 00:19:01 create a sparse matrix. 342 00:19:01 --> 00:19:07 And then if I do stuff to it, MATLAB would automatically know 343 00:19:07 --> 00:19:11 those zeros were there and not spend it's time multiplying by 344 00:19:11 --> 00:19:15 zero But of course, this isn't perfect because I've created 345 00:19:15 --> 00:19:17 the big matrix before sparsifying it. 346 00:19:17 --> 00:19:20 And better to have created it in the first place 347 00:19:20 --> 00:19:22 as a sparse matrix. 348 00:19:22 --> 00:19:27 Ok. 349 00:19:27 --> 00:19:32 So those were properties that you could see. 350 00:19:32 --> 00:19:36 Now I'm looking for little deeper. 351 00:19:36 --> 00:19:39 What's the first question I would ask about a matrix if I 352 00:19:39 --> 00:19:43 have to solve a system of equations, say KU=F 353 00:19:43 --> 00:19:46 or something. 354 00:19:46 --> 00:19:53 I got a 4 by 4 matrix, four equations, four unknowns. 355 00:19:53 --> 00:19:57 What would I want to know next? 356 00:19:57 --> 00:19:59 Is it invertible? 357 00:19:59 --> 00:20:04 Is the matrix invertible? 358 00:20:04 --> 00:20:07 And that's an important question and how do you 359 00:20:07 --> 00:20:10 recognize an invertible matrix? 360 00:20:10 --> 00:20:12 This one is invertible. 361 00:20:12 --> 00:20:15 So let me say K is invertible. 362 00:20:15 --> 00:20:17 And what does that mean? 363 00:20:17 --> 00:20:21 That means that there's another matrix, K inverse such that 364 00:20:21 --> 00:20:27 K times K inverse is the identity matrix. 365 00:20:27 --> 00:20:33 The identity matrix in MATLAB would be eye(n) and it's 366 00:20:33 --> 00:20:35 the diagonal matrix of one. 367 00:20:35 --> 00:20:39 It's the unit matrix is the matrix that doesn't do 368 00:20:39 --> 00:20:43 anything to a vector. 369 00:20:43 --> 00:20:48 So this K has an inverse. 370 00:20:48 --> 00:20:49 But how do you know? 371 00:20:49 --> 00:20:53 How can you recognize that a matrix is invertible? 372 00:20:53 --> 00:20:56 Because obviously that's a critical question and many, 373 00:20:56 --> 00:21:00 many-- since our matrices are not-- a random matrix would be 374 00:21:00 --> 00:21:06 invertible, for sure, but our matrices have patterns, they're 375 00:21:06 --> 00:21:11 created out of a problem and the question of whether that 376 00:21:11 --> 00:21:13 matrix is invertible is fundamental. 377 00:21:13 --> 00:21:18 I mean finite elements has these, zero energy modes that 378 00:21:18 --> 00:21:24 you have to watch out for because, what are they? 379 00:21:24 --> 00:21:28 They produce non-invertible stiffness matrix. 380 00:21:28 --> 00:21:28 Ok. 381 00:21:28 --> 00:21:31 So how did we know, or how could we know that 382 00:21:31 --> 00:21:34 this K is invertible? 383 00:21:34 --> 00:21:37 Somebody said invertible and I wrote it down. 384 00:21:37 --> 00:21:39 Yeah? 385 00:21:39 --> 00:21:41 Well ok. 386 00:21:41 --> 00:21:45 Now I get to make a speech about determinants. 387 00:21:45 --> 00:21:46 Don't deal with them! 388 00:21:46 --> 00:21:49 Don't touch determinants. 389 00:21:49 --> 00:21:54 I mean this particular four by four happens to have 390 00:21:54 --> 00:21:56 a nice determinant. 391 00:21:56 --> 00:21:58 I think it's five. 392 00:21:58 --> 00:22:04 But if it was a 100 by 100 how would we show that the 393 00:22:04 --> 00:22:06 matrix was invertible? 394 00:22:06 --> 00:22:10 And what I mean by this is the whole family is invertible. 395 00:22:10 --> 00:22:13 All sizes are invertible. 396 00:22:13 --> 00:22:17 K_ n is invertible for every n, not just this particular guy, 397 00:22:17 --> 00:22:20 whose determinant we could take. 398 00:22:20 --> 00:22:24 But as five by five, six by six, we would be up in the-- 399 00:22:24 --> 00:22:28 but you're completely right. 400 00:22:28 --> 00:22:33 The determinant is a test. 401 00:22:33 --> 00:22:35 Alright. 402 00:22:35 --> 00:22:40 But I guess I'm saying that it's not the test 403 00:22:40 --> 00:22:45 that I would use. 404 00:22:45 --> 00:22:49 So what I do? 405 00:22:49 --> 00:22:52 I would row reduce. 406 00:22:52 --> 00:22:58 That's the default option in linear algebra. 407 00:22:58 --> 00:23:01 If you don't know what to do with a matrix, if you want to 408 00:23:01 --> 00:23:03 see what's going on, row reduce. 409 00:23:03 --> 00:23:04 What does that mean? 410 00:23:04 --> 00:23:09 That means, shall I try it? 411 00:23:09 --> 00:23:21 So let me just start it just so I'm not using a word 412 00:23:21 --> 00:23:24 that we don't need. 413 00:23:24 --> 00:23:25 Ok. 414 00:23:25 --> 00:23:29 And actually, maybe the third lecture, maybe next Monday 415 00:23:29 --> 00:23:33 we'll come back to row reduce. 416 00:23:33 --> 00:23:38 So I won't make heavy weather of that, certainly not now. 417 00:23:38 --> 00:23:43 So what is row reduce, just so you know. 418 00:23:43 --> 00:23:46 I want to get that minus one to be a zero. 419 00:23:46 --> 00:23:50 I'm aiming for a triangular matrix. 420 00:23:50 --> 00:23:55 I want to clean out below the diagonal because if my matrix 421 00:23:55 --> 00:23:59 is triangular then I can see immediately everything. 422 00:23:59 --> 00:24:01 Right? 423 00:24:01 --> 00:24:06 Ultimately I'll reach a matrix U that'll be upper triangular 424 00:24:06 --> 00:24:11 and that first row won't change but the second row will change. 425 00:24:11 --> 00:24:13 And what does it change to? 426 00:24:13 --> 00:24:17 How do I clean out, get a zero in that where the 427 00:24:17 --> 00:24:21 minus one is right now? 428 00:24:21 --> 00:24:29 Well I want to use the first row, the first equation. 429 00:24:29 --> 00:24:32 I want to add some multiple of the first 430 00:24:32 --> 00:24:36 row to the second row. 431 00:24:36 --> 00:24:38 And what should that multiple be? 432 00:24:38 --> 00:24:41 I want to multiply that row by something. 433 00:24:41 --> 00:24:43 And I'll say "add" today. 434 00:24:43 --> 00:24:47 Later I'll say "subtract." But what shall I do? 435 00:24:47 --> 00:24:50 Just tell me what the heck to do. 436 00:24:50 --> 00:24:53 I've got that row and I want to use it, I want to take a 437 00:24:53 --> 00:24:55 combination of these two rows. 438 00:24:55 --> 00:24:59 This row and some multiple of this one that'll 439 00:24:59 --> 00:25:00 produce a zero. 440 00:25:00 --> 00:25:02 This is called the pivot. 441 00:25:02 --> 00:25:04 That's the first pivot P-I-V-O-T. 442 00:25:04 --> 00:25:07 Pivot. 443 00:25:07 --> 00:25:11 And then that's the pivot row. 444 00:25:11 --> 00:25:14 And what do I do? 445 00:25:14 --> 00:25:15 Tell me what to do. 446 00:25:15 --> 00:25:18 Add half this row to this one. 447 00:25:18 --> 00:25:21 When I add half of that row to that one, what do I get? 448 00:25:21 --> 00:25:22 I get that zero. 449 00:25:22 --> 00:25:26 What do I get here for the second pivot? 450 00:25:26 --> 00:25:27 What is it? 451 00:25:27 --> 00:25:30 1.5, 3/2. 452 00:25:30 --> 00:25:32 Because half of that is, so 3/2. 453 00:25:32 --> 00:25:39 And the rest won't change. 454 00:25:39 --> 00:25:43 So I'm happy with that zero. 455 00:25:43 --> 00:25:48 Now I've got a couple more entries below that first pivot, 456 00:25:48 --> 00:25:49 but they're already zero. 457 00:25:49 --> 00:25:52 That's where the sparseness pays off. 458 00:25:52 --> 00:25:54 The tridiagonal really pays off. 459 00:25:54 --> 00:25:59 So those zeros say the first column is finished. 460 00:25:59 --> 00:26:02 So I'm ready to go on to the second column. 461 00:26:02 --> 00:26:08 It's like I got to this smaller problem with the 3/2 here. 462 00:26:08 --> 00:26:12 And a zero there. 463 00:26:12 --> 00:26:13 What do I do now? 464 00:26:13 --> 00:26:16 There is the second pivot, 3/2. 465 00:26:16 --> 00:26:17 Below it is a non-zero. 466 00:26:17 --> 00:26:20 I gotta get rid of it. 467 00:26:20 --> 00:26:23 What do I multiply by now? 468 00:26:23 --> 00:26:24 2/3. 469 00:26:24 --> 00:26:28 2/3 of that new, second row added to the third row will 470 00:26:28 --> 00:26:30 clean out the third row. 471 00:26:30 --> 00:26:32 This was already cleaned out. 472 00:26:32 --> 00:26:34 This is already a zero. 473 00:26:34 --> 00:26:38 But I want to have 2/3 of this row added to this one so 474 00:26:38 --> 00:26:41 what's my new third row? 475 00:26:41 --> 00:26:43 Starts with zero and what's the third pivot now? 476 00:26:43 --> 00:26:46 You see the pivots appearing? 477 00:26:46 --> 00:26:51 The third pivot will be 4/3 because I've got 2/3 this 478 00:26:51 --> 00:26:56 minus one and two is 6/3 so I have 6/3. 479 00:26:56 --> 00:27:00 I'm taking 2/3 away, I get 4/3 and that minus 480 00:27:00 --> 00:27:01 one is still there. 481 00:27:01 --> 00:27:07 So you see that I'm-- this is fast. 482 00:27:07 --> 00:27:08 This is really fast. 483 00:27:08 --> 00:27:11 And the next step, maybe you can see the beautiful 484 00:27:11 --> 00:27:13 patterns that are coming. 485 00:27:13 --> 00:27:16 Do you want to just guess the fourth pivot? 486 00:27:16 --> 00:27:21 5/4, good guess, right. 487 00:27:21 --> 00:27:24 5/4. 488 00:27:24 --> 00:27:29 Now this is actually how MATLAB would find the determinant. 489 00:27:29 --> 00:27:32 It would do elimination. 490 00:27:32 --> 00:27:36 I call that elimination because it eliminated all those numbers 491 00:27:36 --> 00:27:39 below the diagonal and got zeros. 492 00:27:39 --> 00:27:42 Now what's the determinant? 493 00:27:42 --> 00:27:45 If I asked you for the determinant, and I will very 494 00:27:45 --> 00:27:51 rarely use the word determinant, but I guess I'm 495 00:27:51 --> 00:27:55 into it now, so tell me the determinant. 496 00:27:55 --> 00:27:58 Five. 497 00:27:58 --> 00:27:59 Why's that? 498 00:27:59 --> 00:28:01 I guess I did say five earlier. 499 00:28:01 --> 00:28:06 But how do you know it's five? 500 00:28:06 --> 00:28:10 Whatever the determinant of that matrix is, why is it five? 501 00:28:10 --> 00:28:12 Because it's a triangular matrix. 502 00:28:12 --> 00:28:16 Triangular matrices, you've got all these zeros. 503 00:28:16 --> 00:28:18 You can see what's happening. 504 00:28:18 --> 00:28:21 And the determinant of a triangular matrix is just the 505 00:28:21 --> 00:28:24 product down the diagonal. 506 00:28:24 --> 00:28:25 The product of these pivots. 507 00:28:25 --> 00:28:29 The determinant is the product of the pivots. 508 00:28:29 --> 00:28:32 And that's how MATLAB would compute a determinant. 509 00:28:32 --> 00:28:36 And it would take two times 3/2 times 4/3 times 5/4 and it 510 00:28:36 --> 00:28:40 would give answer five. 511 00:28:40 --> 00:28:45 My friend Alan Edelman told me something yesterday. 512 00:28:45 --> 00:28:54 MATLAB computes in floating point. 513 00:28:54 --> 00:29:02 So 4/3, that's 1.3333, etc. 514 00:29:02 --> 00:29:06 So MATLAB would not, when it does that multiplication, 515 00:29:06 --> 00:29:08 get a whole number. 516 00:29:08 --> 00:29:09 Right? 517 00:29:09 --> 00:29:14 Because in MATLAB that would be 1.333 and probably it would 518 00:29:14 --> 00:29:18 make that last pivot a decimal, a long decimal. 519 00:29:18 --> 00:29:22 And then when it multiplies that it gets whatever it gets. 520 00:29:22 --> 00:29:25 But it's not exactly five I think. 521 00:29:25 --> 00:29:30 Nevertheless MATLAB will print the answer five. 522 00:29:30 --> 00:29:31 It's cheated actually. 523 00:29:31 --> 00:29:36 It's done that calculation and I don't know if it takes the 524 00:29:36 --> 00:29:42 nearest integer when it knows that the-- I shouldn't tell 525 00:29:42 --> 00:29:46 you this, this isn't even interesting. 526 00:29:46 --> 00:29:50 If the determinant of an integer matrix, whole number is 527 00:29:50 --> 00:29:54 a whole number, so MATLAB says, Better get a whole number. 528 00:29:54 --> 00:29:58 And somehow it gets one. 529 00:29:58 --> 00:30:01 Actually, it doesn't always get the right one. 530 00:30:01 --> 00:30:09 So maybe later I'll know the matrix whose determinant 531 00:30:09 --> 00:30:11 might not come out right. 532 00:30:11 --> 00:30:15 But ours is right, five. 533 00:30:15 --> 00:30:19 Now where was this going? 534 00:30:19 --> 00:30:23 It got thrown off track by the determinant. 535 00:30:23 --> 00:30:25 What's the real test? 536 00:30:25 --> 00:30:28 Well so I said there are two ways to see that a 537 00:30:28 --> 00:30:30 matrix is invertible. 538 00:30:30 --> 00:30:32 Or not invertible. 539 00:30:32 --> 00:30:34 Here we're talking about the first way. 540 00:30:34 --> 00:30:38 How do I know that this matrix-- I've got an 541 00:30:38 --> 00:30:39 upper triangular matrix. 542 00:30:39 --> 00:30:41 When is it invertible? 543 00:30:41 --> 00:30:47 When is an upper triangular matrix invertible? 544 00:30:47 --> 00:30:48 Upper triangular is great. 545 00:30:48 --> 00:30:50 When you've got it in that form you should 546 00:30:50 --> 00:30:51 be able to see stuff. 547 00:30:51 --> 00:30:58 So this key question of invertible, which is not 548 00:30:58 --> 00:31:04 obvious for a typical matrix is obvious for 549 00:31:04 --> 00:31:06 a triangular matrix. 550 00:31:06 --> 00:31:06 And why? 551 00:31:06 --> 00:31:10 What's the test? 552 00:31:10 --> 00:31:12 Well, we could do the determinant but we can say it 553 00:31:12 --> 00:31:15 without using that long word. 554 00:31:15 --> 00:31:18 The diagonal is non-zero. 555 00:31:18 --> 00:31:22 K as invertible because the diagonal-- no, it's got 556 00:31:22 --> 00:31:24 a full set of pivots. 557 00:31:24 --> 00:31:26 It's got four non-zero pivots. 558 00:31:26 --> 00:31:28 That's what it takes. 559 00:31:28 --> 00:31:31 That's what it's going to take to solve systems. 560 00:31:31 --> 00:31:33 So this is the first step in solving this system. 561 00:31:33 --> 00:31:38 In other words, to decide if a matrix is invertible, you 562 00:31:38 --> 00:31:41 just go ahead and use it. 563 00:31:41 --> 00:31:45 You don't stop first necessarily to check 564 00:31:45 --> 00:31:46 invertibility. 565 00:31:46 --> 00:31:49 You go forward, you get to this point and you see non-zeros 566 00:31:49 --> 00:31:53 there and then you're practically got to 567 00:31:53 --> 00:31:55 the answer here. 568 00:31:55 --> 00:32:00 I'll leave for another day the final back to going back 569 00:32:00 --> 00:32:03 upwards that gives you the answer. 570 00:32:03 --> 00:32:05 So K is invertible. 571 00:32:05 --> 00:32:15 That means full set of pivots. n non-zero pivots. 572 00:32:15 --> 00:32:20 And here they are, two, 3/2, 4/3 and 5/4. 573 00:32:20 --> 00:32:23 Worth knowing because this matrix K is so important. 574 00:32:23 --> 00:32:24 We'll see it over and over again. 575 00:32:24 --> 00:32:33 Part of my purpose today is to give some matrices a name 576 00:32:33 --> 00:32:35 because we'll see them again and you'll know them and 577 00:32:35 --> 00:32:38 you'll recognize them. 578 00:32:38 --> 00:32:44 While I'm on this invertible or not invertible business I 579 00:32:44 --> 00:32:48 want to ask you to change K. 580 00:32:48 --> 00:32:52 To make it not invertible. 581 00:32:52 --> 00:32:54 Change that matrix. 582 00:32:54 --> 00:32:56 How could I change that matrix? 583 00:32:56 --> 00:32:58 Well, of course, many ways. 584 00:32:58 --> 00:33:01 But I'm interested in another matrix and this'll be 585 00:33:01 --> 00:33:04 among my special matrices. 586 00:33:04 --> 00:33:07 And it will start out the same. 587 00:33:07 --> 00:33:14 It'll have these same diagonals. 588 00:33:14 --> 00:33:16 It'll be Toeplitz. 589 00:33:16 --> 00:33:23 I'm going to call it C and I want to say the reason I'm 590 00:33:23 --> 00:33:25 talking about it now is that it's not going to 591 00:33:25 --> 00:33:29 be invertible. 592 00:33:29 --> 00:33:38 And I'm going to tell you a C and see if you can tell me 593 00:33:38 --> 00:33:40 why it is not invertible. 594 00:33:40 --> 00:33:42 So here's the difference; I'm going to put minus 595 00:33:42 --> 00:33:45 one in the corners. 596 00:33:45 --> 00:33:49 Still zeros there. 597 00:33:49 --> 00:33:56 So that matrix C still has that pattern. 598 00:33:56 --> 00:33:58 It's still a Toeplitz matrix, actually. 599 00:33:58 --> 00:34:04 That would still be the matrix Toeplitz of two, minus 600 00:34:04 --> 00:34:04 one, zero, minus one. 601 00:34:04 --> 00:34:07 602 00:34:07 --> 00:34:14 I claim that matrix is not invertible and I claim that we 603 00:34:14 --> 00:34:19 can see that without computing determinants, we can see it 604 00:34:19 --> 00:34:22 without doing elimination, too. 605 00:34:22 --> 00:34:24 MATLAB would see it by doing elimination. 606 00:34:24 --> 00:34:30 We can see it by just human intelligence. 607 00:34:30 --> 00:34:33 Now why? 608 00:34:33 --> 00:34:39 How do I recognize a matrix that's not invertible? 609 00:34:39 --> 00:34:44 And then, by converse, how a matrix that is invertible. 610 00:34:44 --> 00:34:49 I claim-- and let may say first, let me say 611 00:34:49 --> 00:34:51 why that letter C. 612 00:34:51 --> 00:35:00 That letter C stands for circulant. it's because this 613 00:35:00 --> 00:35:03 word circulant, why circulant, it's because that diagonal 614 00:35:03 --> 00:35:09 which only had three guys circled around to the fourth. 615 00:35:09 --> 00:35:12 This diagonal that only had three entries circled around 616 00:35:12 --> 00:35:14 to the fourth entry. 617 00:35:14 --> 00:35:16 This diagonal with two zeros circled around to 618 00:35:16 --> 00:35:17 the other two zeros. 619 00:35:17 --> 00:35:22 The diagonal are not only constant, they loop around. 620 00:35:22 --> 00:35:24 And you use the word periodic. 621 00:35:24 --> 00:35:29 Now for me, that's the periodic matrix. 622 00:35:29 --> 00:35:35 See, a circulant matrix comes from a periodic problem. 623 00:35:35 --> 00:35:38 Because it loops around. 624 00:35:38 --> 00:35:42 It brings numbers, zero is the same as number 625 00:35:42 --> 00:35:45 four or something. 626 00:35:45 --> 00:35:51 And why is that not invertible? 627 00:35:51 --> 00:35:55 The thing is can you find a vector? 628 00:35:55 --> 00:35:57 Because matrices multiply vectors, that's 629 00:35:57 --> 00:35:59 their whole point. 630 00:35:59 --> 00:36:03 Can you see a vector that it takes to zero? 631 00:36:03 --> 00:36:05 Can you see a solution to Cu=0? 632 00:36:06 --> 00:36:11 I'm looking for a u with four entries so that 633 00:36:11 --> 00:36:18 I get four zeros. 634 00:36:18 --> 00:36:20 Do you see it? 635 00:36:20 --> 00:36:21 All ones. 636 00:36:21 --> 00:36:23 All ones. 637 00:36:23 --> 00:36:25 That will do it. 638 00:36:25 --> 00:36:33 So that's a nice, natural entry, a constant. 639 00:36:33 --> 00:36:37 And do you see why when I-- we haven't spoken about 640 00:36:37 --> 00:36:42 multiplying matrices times vectors. 641 00:36:42 --> 00:36:44 And most people will do it this way. 642 00:36:44 --> 00:36:46 And let's do this one this way. 643 00:36:46 --> 00:36:48 You take row one times that, you get two, minus 644 00:36:48 --> 00:36:49 one, zero, minus one. 645 00:36:51 --> 00:36:53 You get the zero because of that new number. 646 00:36:53 --> 00:36:58 Here we always got zero from the all ones vector and now 647 00:36:58 --> 00:37:04 over here that minus one, you see it's just right. 648 00:37:04 --> 00:37:09 If all the rows add to zero then this vector of all ones 649 00:37:09 --> 00:37:14 will be, I would use the word in the null space if you 650 00:37:14 --> 00:37:18 wanted a fancy word, a linear algebra word. 651 00:37:18 --> 00:37:19 What does that mean? 652 00:37:19 --> 00:37:21 It solves Cu=0. 653 00:37:21 --> 00:37:24 654 00:37:24 --> 00:37:29 And why does that show that the matrix isn't invertible? 655 00:37:29 --> 00:37:31 Because that's our point here. 656 00:37:31 --> 00:37:32 I have a solution to Cu=0. 657 00:37:35 --> 00:37:39 I claim that the existence of such a solution has wiped out 658 00:37:39 --> 00:37:45 the possibility that the matrix is invertible because if it 659 00:37:45 --> 00:37:49 was invertible, what would this lead to? 660 00:37:49 --> 00:37:56 If invertible, if C inverse exists what would I do to that 661 00:37:56 --> 00:38:04 equation that would show me that C inverse can't exist? 662 00:38:04 --> 00:38:08 Multiply both sides by C inverse. 663 00:38:08 --> 00:38:11 So you're seeing, just this first day you're seeing some 664 00:38:11 --> 00:38:14 of the natural steps of linear algebra. 665 00:38:14 --> 00:38:17 Row reduction, multiply when you want to see what's 666 00:38:17 --> 00:38:21 happening, multiply both sides by C inverse. 667 00:38:21 --> 00:38:25 That's the same as in ordinary language, Do the same thing 668 00:38:25 --> 00:38:27 to all the equations. 669 00:38:27 --> 00:38:30 So I multiply both sides by the same matrix. 670 00:38:30 --> 00:38:31 And here I would get (C inverse)(Cu)=(C inverse)(0). 671 00:38:31 --> 00:38:36 672 00:38:36 --> 00:38:40 So what does that tell me? 673 00:38:40 --> 00:38:43 I made it long, I threw in this extra step. 674 00:38:43 --> 00:38:51 You were going to jump immediately to C inverse C is I 675 00:38:51 --> 00:38:54 is the identity matrix and when the identity matrix multiplies 676 00:38:54 --> 00:38:57 a vector u, you get u. 677 00:38:57 --> 00:39:00 And on the right side, C inverse, whatever it is if 678 00:39:00 --> 00:39:05 it existed, times zero would have to be zero. 679 00:39:05 --> 00:39:09 So this would say that if C inverse exists, then the only 680 00:39:09 --> 00:39:13 solution is u equals u. 681 00:39:13 --> 00:39:15 That's a good way to recognize invertible matrices. 682 00:39:15 --> 00:39:20 If it is invertible then the only solution to Cu=0 u=0. 683 00:39:21 --> 00:39:24 And that wasn't true here. 684 00:39:24 --> 00:39:28 So we conclude C is not invertible. 685 00:39:28 --> 00:39:32 C is therefore not invertible. 686 00:39:32 --> 00:39:36 Now can I even jump in. 687 00:39:36 --> 00:39:38 I've got two more matrices that I want to tell you 688 00:39:38 --> 00:39:43 about that are also close cousins of K and C. 689 00:39:43 --> 00:39:50 But let me just explain physically a little 690 00:39:50 --> 00:39:54 bit about where these matrices are coming from. 691 00:39:54 --> 00:39:59 So maybe next to K-- so I'm not going to put periodic there. 692 00:39:59 --> 00:40:01 Right? 693 00:40:01 --> 00:40:03 That's the one that I would call periodic. 694 00:40:03 --> 00:40:08 This one is fixed at the ends. 695 00:40:08 --> 00:40:13 Can I draw a little picture that aims to show that? 696 00:40:13 --> 00:40:18 Aims to show where this is coming from. 697 00:40:18 --> 00:40:22 It's coming from I think of this as controlling 698 00:40:22 --> 00:40:23 like four masses. 699 00:40:23 --> 00:40:28 Mass one, mass two, mass three and mass four with springs 700 00:40:28 --> 00:40:40 attached and with endpoints fixed. 701 00:40:40 --> 00:40:47 So if I put some weights on those masses-- we'll do this; 702 00:40:47 --> 00:40:51 masses and springs is going to be the very first application 703 00:40:51 --> 00:40:55 and it will connect to all these matrices. 704 00:40:55 --> 00:41:05 And all I'm doing now is just asking to draw the system. 705 00:41:05 --> 00:41:06 Draw the mechanical system. 706 00:41:06 --> 00:41:09 Actually I'll usually draw it vertically. 707 00:41:09 --> 00:41:14 But anyway, it's got four masses and the fact that this 708 00:41:14 --> 00:41:19 minus one here got chopped off, what would I call that end? 709 00:41:19 --> 00:41:21 I'd call that a fixed end. 710 00:41:21 --> 00:41:25 So this is a fixed, fixed matrix. 711 00:41:25 --> 00:41:28 Both ends or fixed. 712 00:41:28 --> 00:41:32 And it's the matrix that would govern and the springs and 713 00:41:32 --> 00:41:36 masses all the same is what tells me that the 714 00:41:36 --> 00:41:38 thing is Toeplitz. 715 00:41:38 --> 00:41:42 Now what's the picture that goes with C? 716 00:41:42 --> 00:41:46 What's the picture with C? 717 00:41:46 --> 00:41:49 Do you have an instinct of that? 718 00:41:49 --> 00:41:52 So C is periodic. 719 00:41:52 --> 00:41:57 So again we've got four masses connected by springs. 720 00:41:57 --> 00:42:03 But what's up with those masses to make the problem cyclic, 721 00:42:03 --> 00:42:07 periodic, circular, whatever word you like. 722 00:42:07 --> 00:42:13 They're arranged in a ring. 723 00:42:13 --> 00:42:16 The fourth guy comes back to the first one. 724 00:42:16 --> 00:42:22 So the four masses would be, so in some kind of a ring, the 725 00:42:22 --> 00:42:27 springs would connect them. 726 00:42:27 --> 00:42:31 I don't know if that's suggestive, but I hope so. 727 00:42:31 --> 00:42:37 And what's the point of, can we just speak about 728 00:42:37 --> 00:42:39 mechanics one moment? 729 00:42:39 --> 00:42:46 How does that system differ from this fixed system? 730 00:42:46 --> 00:42:53 Here the whole system can't move, right? 731 00:42:53 --> 00:42:55 If there no force, then nothing can happen. 732 00:42:55 --> 00:43:00 Here the whole system can turn. 733 00:43:00 --> 00:43:03 They can all displace the same amount and just turn without 734 00:43:03 --> 00:43:06 any compression of the springs, without any force 735 00:43:06 --> 00:43:08 having to do anything. 736 00:43:08 --> 00:43:12 And that's why the solution that kills this matrix 737 00:43:12 --> 00:43:13 is one, one, one, one. 738 00:43:15 --> 00:43:19 So one, one, one, one would describe a case where all the 739 00:43:19 --> 00:43:21 displacements were equal. 740 00:43:21 --> 00:43:25 In a way it's like the arbitrary constant in calculus. 741 00:43:25 --> 00:43:29 You're always adding plus C. 742 00:43:29 --> 00:43:35 So here we've got a solution of all ones that produces zero the 743 00:43:35 --> 00:43:38 way the derivative of a constant function is 744 00:43:38 --> 00:43:41 the zero function. 745 00:43:41 --> 00:43:49 So this is just like an indication. 746 00:43:49 --> 00:43:51 Yes, perfect. 747 00:43:51 --> 00:43:52 I've got two more matrices. 748 00:43:52 --> 00:43:58 Are you okay for two more? 749 00:43:58 --> 00:44:03 Yes okay, what are they? 750 00:44:03 --> 00:44:10 Okay a different blackboard for the last two. 751 00:44:10 --> 00:44:17 So one of them is going to come by freeing up this end. 752 00:44:17 --> 00:44:24 So I'm going to take that support away. 753 00:44:24 --> 00:44:29 And you might imagine like a tower oscillating up and down 754 00:44:29 --> 00:44:34 or you might turn it upside down and like a hanging spring, 755 00:44:34 --> 00:44:39 or rather four springs with four masses hanging onto them. 756 00:44:39 --> 00:44:43 But this end is fixed and this is not fixed anymore, 757 00:44:43 --> 00:44:46 this is now free. 758 00:44:46 --> 00:44:50 And can I tell you the matrix, the free-fixed matrix. 759 00:44:50 --> 00:44:53 Free-fixed. 760 00:44:53 --> 00:44:56 Because it's the top end that I changed, I'm 761 00:44:56 --> 00:44:58 going to call it T. 762 00:44:58 --> 00:45:10 So all the other guys are going to be the same but the top one, 763 00:45:10 --> 00:45:15 the top row, the boundary row, boundary conditions are always 764 00:45:15 --> 00:45:20 the tough part, the tricky part, the key part of a model, 765 00:45:20 --> 00:45:23 and here the natural boundary condition is 766 00:45:23 --> 00:45:25 to have a one there. 767 00:45:25 --> 00:45:34 That two changed to a one. 768 00:45:34 --> 00:45:37 Now if I asked you for the properties of that matrix-- 769 00:45:37 --> 00:45:41 so that's the third. shall I do the fourth one? 770 00:45:41 --> 00:45:44 So you have them all, you'll have the whole picture. 771 00:45:44 --> 00:45:45 The fourth one, well you can guess. 772 00:45:45 --> 00:45:48 What's the fourth? 773 00:45:48 --> 00:45:51 What am I going to do? 774 00:45:51 --> 00:45:53 Free up the other end. 775 00:45:53 --> 00:45:59 So this guy had one free end and the other guy 776 00:45:59 --> 00:46:01 has B for both ends. 777 00:46:01 --> 00:46:04 B for both ends are going to be free. 778 00:46:04 --> 00:46:06 So this is free-fixed. 779 00:46:06 --> 00:46:08 This'll be free-free. 780 00:46:08 --> 00:46:13 So that means I have this free end, the usual stuff in the 781 00:46:13 --> 00:46:23 middle, no change, and the last row is what? 782 00:46:23 --> 00:46:25 What am I going to put in the last row? 783 00:46:25 --> 00:46:26 Minus one, one. 784 00:46:26 --> 00:46:26 Minus one, one. 785 00:46:26 --> 00:46:29 786 00:46:29 --> 00:46:34 So I've changed the diagonal. 787 00:46:34 --> 00:46:38 There I put a single one in because I freed up one end. 788 00:46:38 --> 00:46:41 With B I freed both ends and I got two minus ones. 789 00:46:41 --> 00:46:44 Now what do you think? 790 00:46:44 --> 00:46:52 So we've drawn the free-fixed one and what's your guess? 791 00:46:52 --> 00:46:55 They're all symmetric. 792 00:46:55 --> 00:46:57 That's no accident. 793 00:46:57 --> 00:47:00 They're all tridiagonal, no accident again. 794 00:47:00 --> 00:47:02 Why are they tridiagonal? 795 00:47:02 --> 00:47:06 Physically they're tridiagonal because that mass is only 796 00:47:06 --> 00:47:08 connected to it's two neighbors, it's not 797 00:47:08 --> 00:47:10 connected to that mass. 798 00:47:10 --> 00:47:16 That's why we get a zero in the two, four position. 799 00:47:16 --> 00:47:19 Because two is not connected to four. 800 00:47:19 --> 00:47:21 So it's tridiagonal. 801 00:47:21 --> 00:47:25 And it's not Toeplitz anymore, right? 802 00:47:25 --> 00:47:27 Toeplitz says constant diagonals and these are 803 00:47:27 --> 00:47:29 not quite constant. 804 00:47:29 --> 00:47:34 I would create K, I would take T equal K if I was going to 805 00:47:34 --> 00:47:37 create this matrix and then I would say T of one, 806 00:47:37 --> 00:47:40 one equal one. 807 00:47:40 --> 00:47:49 That command would fix up the first entry. 808 00:47:49 --> 00:47:50 Yeah, that's a serious question. 809 00:47:50 --> 00:47:53 Maybe, can I hang on until Friday, and 810 00:47:53 --> 00:47:54 even maybe next week. 811 00:47:54 --> 00:47:56 Because it's very important. 812 00:47:56 --> 00:48:00 When I said boundary conditions are the key to 813 00:48:00 --> 00:48:02 problems, I'm serious. 814 00:48:02 --> 00:48:07 If I had to think okay, what do people come in my office ask 815 00:48:07 --> 00:48:09 about questions, I say right away, What's the 816 00:48:09 --> 00:48:10 boundary condition? 817 00:48:10 --> 00:48:12 Because I know that's where the problem is. 818 00:48:12 --> 00:48:16 And so here we'll see these guys clearly. 819 00:48:16 --> 00:48:23 Fixed and free, very important. 820 00:48:23 --> 00:48:26 But also let me say two more words, I never can resist. 821 00:48:26 --> 00:48:30 So fixed means the displacement is zero. 822 00:48:30 --> 00:48:32 Something was set to zero. 823 00:48:32 --> 00:48:36 The fifth guy, the fifth over here, that fifth column 824 00:48:36 --> 00:48:39 was knocked out. 825 00:48:39 --> 00:48:44 Free means that in here it could mean that the fifth guy 826 00:48:44 --> 00:48:49 is the same as the fourth. 827 00:48:49 --> 00:48:52 The slope is zero. 828 00:48:52 --> 00:48:55 Fixed is u is zero. 829 00:48:55 --> 00:48:59 Free is slope is zero. 830 00:48:59 --> 00:49:05 So here I have a slope of zero at that end, here 831 00:49:05 --> 00:49:05 I have it at both ends. 832 00:49:05 --> 00:49:09 So maybe that's a sort of part answer. 833 00:49:09 --> 00:49:12 Now I wanted to get to the difference between 834 00:49:12 --> 00:49:15 these two matrices. 835 00:49:15 --> 00:49:19 And the main properties. 836 00:49:19 --> 00:49:19 So what are we see? 837 00:49:19 --> 00:49:23 Symmetric again, tridiagonal again, not quite Toeplitz, 838 00:49:23 --> 00:49:27 but almost, sort of morally Toeplitz. 839 00:49:27 --> 00:49:32 But then the key question was invertible or not. 840 00:49:32 --> 00:49:34 Key question was invertible or not. 841 00:49:34 --> 00:49:35 Right. 842 00:49:35 --> 00:49:37 And what's your guess on these two? 843 00:49:37 --> 00:49:41 Do you think that one's invertible or not? 844 00:49:41 --> 00:49:41 Make a guess. 845 00:49:41 --> 00:49:46 You're allowed to guess. 846 00:49:46 --> 00:49:47 Yeah it is. 847 00:49:47 --> 00:49:48 Why's that? 848 00:49:48 --> 00:49:52 Because this thing has still got a support. 849 00:49:52 --> 00:49:56 It's not free to shift forever. 850 00:49:56 --> 00:49:57 It's held in there. 851 00:49:57 --> 00:50:01 So that gives you a hint about this guy. 852 00:50:01 --> 00:50:04 Invertible or not for B? 853 00:50:04 --> 00:50:06 No. 854 00:50:06 --> 00:50:09 And now prove that it's not. 855 00:50:09 --> 00:50:14 Physically you were saying, well this free guy with this 856 00:50:14 --> 00:50:19 thing gone now, this is now free-free. 857 00:50:19 --> 00:50:21 Physically we're saying the whole thing can move, 858 00:50:21 --> 00:50:24 there's nothing holding it. 859 00:50:24 --> 00:50:27 But now, for linear algebra, that's not the proper language. 860 00:50:27 --> 00:50:31 You have to say something about that matrix. 861 00:50:31 --> 00:50:37 Maybe tell me something about Bu=0. u What are 862 00:50:37 --> 00:50:38 you going to take for u? 863 00:50:38 --> 00:50:39 Yeah. 864 00:50:39 --> 00:50:41 Same u. 865 00:50:41 --> 00:50:45 We're lucky in this course, u equal is the 866 00:50:45 --> 00:50:48 guilty main vector many times. 867 00:50:48 --> 00:50:55 Because again the rows are all adding to zero and the all ones 868 00:50:55 --> 00:51:02 vector is in the null space. 869 00:51:02 --> 00:51:06 If I could just close with one more word. 870 00:51:06 --> 00:51:07 Because it's the most important. 871 00:51:07 --> 00:51:10 Two words, two words. 872 00:51:10 --> 00:51:12 Because they're the most important words, they're the 873 00:51:12 --> 00:51:15 words that we're leading to in this chapter. 874 00:51:15 --> 00:51:18 And I'm assuming that for most people they will be 875 00:51:18 --> 00:51:21 new words, but not for all. 876 00:51:21 --> 00:51:24 It's a further property of this matrix. 877 00:51:24 --> 00:51:25 So we've got, how many? 878 00:51:25 --> 00:51:27 Four properties, or five? 879 00:51:27 --> 00:51:29 I'm going to go for one more. 880 00:51:29 --> 00:51:33 And I'm just going to say that name first so 881 00:51:33 --> 00:51:36 you know it's coming. 882 00:51:36 --> 00:51:38 And then I'll say, I can't resist saying 883 00:51:38 --> 00:51:41 a tiny bit about it. 884 00:51:41 --> 00:51:45 I'll use a whole blackboard for this. 885 00:51:45 --> 00:51:54 So I'm going to say that K and T are, here it comes, take 886 00:51:54 --> 00:52:07 a breath; positive definite matrices. 887 00:52:07 --> 00:52:10 So if you don't know what that means, I'm happy. 888 00:52:10 --> 00:52:10 Right? 889 00:52:10 --> 00:52:14 Because well, I can tell you one way to recognize a 890 00:52:14 --> 00:52:16 positive definite matrix. 891 00:52:16 --> 00:52:21 And while we're at it, let me tell you about C and B. 892 00:52:21 --> 00:52:30 Those are positive semi-definite because 893 00:52:30 --> 00:52:32 they hit zero somehow. 894 00:52:32 --> 00:52:35 Positive means up there, greater than zero. 895 00:52:35 --> 00:52:40 And what is greater than zero that we've already seen? 896 00:52:40 --> 00:52:42 And we'll say more. 897 00:52:42 --> 00:52:44 The pivots were. 898 00:52:44 --> 00:52:50 So if I have a symmetric matrix and the pivots are all positive 899 00:52:50 --> 00:52:55 then that matrix is not only invertible, because I'm in good 900 00:52:55 --> 00:52:59 shape, the determinant isn't zero, I can go backwards and do 901 00:52:59 --> 00:53:03 everything, those positive numbers are telling me that 902 00:53:03 --> 00:53:07 more than that, the matrix is positive definite. 903 00:53:07 --> 00:53:11 So that's a test. 904 00:53:11 --> 00:53:14 We'll say more about positive definite, but one way to 905 00:53:14 --> 00:53:18 recognize it is compute the pivots by elimination. 906 00:53:18 --> 00:53:20 Are they positive? 907 00:53:20 --> 00:53:23 We'll see that all the eigenvalues are positive. 908 00:53:23 --> 00:53:27 The word positive definite just brings the whole of 909 00:53:27 --> 00:53:29 linear algebra together. 910 00:53:29 --> 00:53:33 It connects to pivots, it connects to eigenvalues, it 911 00:53:33 --> 00:53:36 connects to least squares, it's all over the place. 912 00:53:36 --> 00:53:39 Determinants too. 913 00:53:39 --> 00:53:42 Questions or discussion. 914 00:53:42 --> 00:53:44 It's a big class and we're just meeting for the first time 915 00:53:44 --> 00:53:49 but there's lots of time to, chance to ask me. 916 00:53:49 --> 00:53:52 I'll always be here after class. 917 00:53:52 --> 00:53:53 So shall we stop today? 918 00:53:53 --> 00:53:58 I'll see you Friday or this afternoon. 919 00:53:58 --> 00:54:03 If this wasn't familiar, this afternoon would be a good idea. 920 00:54:03 --> 00:54:05 Thank you.