1 00:00:00,090 --> 00:00:02,490 The following content is provided under a Creative 2 00:00:02,490 --> 00:00:04,030 Commons license. 3 00:00:04,030 --> 00:00:06,330 Your support will help MIT OpenCourseWare 4 00:00:06,330 --> 00:00:10,690 continue to offer high-quality educational resources for free. 5 00:00:10,690 --> 00:00:13,320 To make a donation or view additional materials 6 00:00:13,320 --> 00:00:17,270 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,270 --> 00:00:18,200 at ocw.mit.edu. 8 00:00:32,518 --> 00:00:33,870 HERBERT GROSS: Hi. 9 00:00:33,870 --> 00:00:37,710 Welcome to our lecture today in Calculus Revisited. 10 00:00:37,710 --> 00:00:41,400 And our topic is going to be the dot product. 11 00:00:41,400 --> 00:00:44,550 And by way of a preamble, let me point out 12 00:00:44,550 --> 00:00:49,020 that in our development of vector spaces in this block, up 13 00:00:49,020 --> 00:00:52,800 until now, we have deliberately not mentioned 14 00:00:52,800 --> 00:00:56,610 the concept of the dot product, even though dot product plays 15 00:00:56,610 --> 00:01:01,470 a very prominent role in the usual vector algebra of two- 16 00:01:01,470 --> 00:01:03,060 and three-dimensional space. 17 00:01:03,060 --> 00:01:04,890 And the reason for this is that we 18 00:01:04,890 --> 00:01:08,580 wanted to emphasize the structure of a vector space. 19 00:01:08,580 --> 00:01:12,540 In a manner of speaking, mathematical structures 20 00:01:12,540 --> 00:01:15,030 are pretty much like an anatomy-type course, 21 00:01:15,030 --> 00:01:18,690 that one starts with a basic structure, exploits it, 22 00:01:18,690 --> 00:01:22,950 sees what ramifications it has, and then gradually adds 23 00:01:22,950 --> 00:01:28,710 on more sophisticated layers of nerves, nerve centers, skin 24 00:01:28,710 --> 00:01:30,390 grafts, and what have you. 25 00:01:30,390 --> 00:01:33,450 And ultimately, builds up a superstructure 26 00:01:33,450 --> 00:01:38,820 based on top of an underlying basic fundamental model. 27 00:01:38,820 --> 00:01:41,560 Now, without going into that in more detail right now, 28 00:01:41,560 --> 00:01:44,520 let it suffice to say that I call today's lecture 29 00:01:44,520 --> 00:01:45,870 "Dot Products." 30 00:01:45,870 --> 00:01:50,310 And by way of review, notice that in the usual two- 31 00:01:50,310 --> 00:01:54,460 and three-dimensional case by the dot product of two vectors, 32 00:01:54,460 --> 00:01:56,520 we mean a real number. 33 00:01:56,520 --> 00:01:59,370 And if we want to keep this thing vague 34 00:01:59,370 --> 00:02:03,030 and not indicate the cosine of the angle between two 35 00:02:03,030 --> 00:02:05,670 vectors, et cetera, one usually says 36 00:02:05,670 --> 00:02:08,610 that a dot product is a mapping that 37 00:02:08,610 --> 00:02:13,350 carries ordered pairs of vectors into the real numbers. 38 00:02:13,350 --> 00:02:15,720 This is the mathematical way of talking 39 00:02:15,720 --> 00:02:18,630 about what a dot product is without referring 40 00:02:18,630 --> 00:02:20,250 to angles or lines. 41 00:02:20,250 --> 00:02:24,270 It's a function which maps ordered pairs of vectors 42 00:02:24,270 --> 00:02:25,980 into real numbers. 43 00:02:25,980 --> 00:02:30,450 In particular, the mapping must have certain properties 44 00:02:30,450 --> 00:02:32,040 to be called a dot product. 45 00:02:32,040 --> 00:02:35,460 And again, emphasizing mathematical structure 46 00:02:35,460 --> 00:02:38,430 based on what we already know is the case from the two- 47 00:02:38,430 --> 00:02:40,650 and three-dimensional situation, what 48 00:02:40,650 --> 00:02:43,380 properties do we want the dot product to have? 49 00:02:43,380 --> 00:02:47,075 Well, we would like alpha dot beta to equal beta dot alpha, 50 00:02:47,075 --> 00:02:48,450 because that's certainly happened 51 00:02:48,450 --> 00:02:50,970 in the two- and three-dimensional case. 52 00:02:50,970 --> 00:02:54,150 We would like alpha dot beta plus gamma 53 00:02:54,150 --> 00:02:58,740 to be alpha dot beta plus alpha dot gamma. 54 00:02:58,740 --> 00:03:02,880 And if we have a scalar multiple of alpha dotted with beta, 55 00:03:02,880 --> 00:03:07,960 we would like that to be the scalar c times the number alpha 56 00:03:07,960 --> 00:03:09,090 dot beta. 57 00:03:09,090 --> 00:03:12,600 Or if we wanted to rewrite this in vector form, 58 00:03:12,600 --> 00:03:16,740 that the c could also be used as a scale of multiple of beta. 59 00:03:16,740 --> 00:03:21,660 That c alpha dot theta is the same as alpha dot c beta. 60 00:03:21,660 --> 00:03:23,830 And where did these three rules come from? 61 00:03:23,830 --> 00:03:26,970 They came from the properties that the ordinary dot product 62 00:03:26,970 --> 00:03:27,900 has. 63 00:03:27,900 --> 00:03:30,450 By the way, a dot product becomes 64 00:03:30,450 --> 00:03:34,230 known as a Euclidean dot product if, in addition 65 00:03:34,230 --> 00:03:36,720 to the given three properties, we also 66 00:03:36,720 --> 00:03:39,900 know that the dot product of a vector and itself, 67 00:03:39,900 --> 00:03:44,130 alpha dot alpha, is a non-negative real number. 68 00:03:44,130 --> 00:03:46,440 And that in particular, alpha dot alpha 69 00:03:46,440 --> 00:03:50,130 can equal 0, if and only if alpha equals 0. 70 00:03:50,130 --> 00:03:53,910 By the way, let me point out that this fourth condition is 71 00:03:53,910 --> 00:03:56,130 independent of the other three. 72 00:03:56,130 --> 00:04:00,300 That when one studies vector spaces in more abstract detail, 73 00:04:00,300 --> 00:04:02,880 one usually defines a dot product 74 00:04:02,880 --> 00:04:04,530 to have just the first three properties 75 00:04:04,530 --> 00:04:05,790 that we talked about. 76 00:04:05,790 --> 00:04:08,700 And if the fourth property happens to be present, 77 00:04:08,700 --> 00:04:13,200 then one talks about a positive-definite dot product. 78 00:04:13,200 --> 00:04:16,140 But not every dot product has to be positive definite, 79 00:04:16,140 --> 00:04:18,630 except that if we're trying to imitate the distance 80 00:04:18,630 --> 00:04:22,230 property of real vector spaces, then we 81 00:04:22,230 --> 00:04:24,390 add the positive-definite criterion. 82 00:04:24,390 --> 00:04:27,270 And that's explained in more detail in the study guide 83 00:04:27,270 --> 00:04:29,130 as we do the exercises. 84 00:04:29,130 --> 00:04:30,840 But I did want to simply point out 85 00:04:30,840 --> 00:04:34,560 that once you know that you have a positive-definite form, 86 00:04:34,560 --> 00:04:37,030 that alpha dot alpha is non-negative, 87 00:04:37,030 --> 00:04:40,170 then it makes sense to abstractly define 88 00:04:40,170 --> 00:04:44,790 the norm or the length of alpha to be the positive square root 89 00:04:44,790 --> 00:04:46,210 of alpha dot alpha. 90 00:04:46,210 --> 00:04:47,850 You see, this makes sense, because you 91 00:04:47,850 --> 00:04:50,820 know that alpha dot alpha is a non-negative real number. 92 00:04:50,820 --> 00:04:53,400 Hence, it has a unique positive square root. 93 00:04:53,400 --> 00:04:56,880 And this is how we abstract the usual distance function. 94 00:04:56,880 --> 00:05:00,210 At any rate, to show you how this works in abstract form, 95 00:05:00,210 --> 00:05:02,100 let's take a three-dimensional case. 96 00:05:02,100 --> 00:05:04,740 Let's suppose v is a three-dimensional vector space, 97 00:05:04,740 --> 00:05:10,230 that a particularly-chosen basis for v is u1, u2, and u3, 98 00:05:10,230 --> 00:05:15,000 and let's suppose that we have a positive-definite dot product. 99 00:05:15,000 --> 00:05:18,480 And I'll just make this up at random or semi-random here. 100 00:05:18,480 --> 00:05:20,790 Let's suppose that I know what the dot 101 00:05:20,790 --> 00:05:24,720 product looks like on all of the pairs of basis elements. 102 00:05:24,720 --> 00:05:28,980 In other words, I know what u1 dot u1 is, what u2 dot u1 is, 103 00:05:28,980 --> 00:05:30,840 what u3 dot u1 is. 104 00:05:30,840 --> 00:05:34,600 And by the way, by the property, that u2 dot u1 105 00:05:34,600 --> 00:05:38,500 must equal u1 dot u2, et cetera, by the commutative property. 106 00:05:38,500 --> 00:05:41,890 Notice that once I know what u2 dot u1 is, 107 00:05:41,890 --> 00:05:44,260 I also know what u1 dot u2 is. 108 00:05:44,260 --> 00:05:47,110 Once I know what u3 dot u1 is, I know 109 00:05:47,110 --> 00:05:49,690 what u1 dot u3 is, et cetera. 110 00:05:49,690 --> 00:05:52,930 At any rate, there are nine possible combinations here. 111 00:05:52,930 --> 00:05:55,330 And just try to fix these in your mind, 112 00:05:55,330 --> 00:05:58,420 because I'll be referring to these as we go along. 113 00:05:58,420 --> 00:06:00,250 u1 dot u1 is 3. 114 00:06:00,250 --> 00:06:02,110 u1 dot u2 is 4. 115 00:06:02,110 --> 00:06:04,120 u1 dot u3 is 5. 116 00:06:04,120 --> 00:06:06,190 u2 dot u2 is 6. 117 00:06:06,190 --> 00:06:08,110 u2 dot u3 is 7. 118 00:06:08,110 --> 00:06:10,360 And u3 dot u3 is 9. 119 00:06:10,360 --> 00:06:12,490 I just happen to know that particular amount. 120 00:06:12,490 --> 00:06:15,490 Let's also suppose that I now want 121 00:06:15,490 --> 00:06:19,900 to find the dot product of any two vectors in my space, 122 00:06:19,900 --> 00:06:23,800 v. My first claim is-- and again, this is an overview. 123 00:06:23,800 --> 00:06:26,260 I'll do this in more abstract detail 124 00:06:26,260 --> 00:06:28,870 in the exercises in the study guide. 125 00:06:28,870 --> 00:06:31,210 My first point, though, is that once we 126 00:06:31,210 --> 00:06:34,660 know what the dot product is for each pair of basis vectors, 127 00:06:34,660 --> 00:06:38,380 we know what the dot product is for all vectors and pairs 128 00:06:38,380 --> 00:06:39,610 of vectors in the space. 129 00:06:39,610 --> 00:06:41,955 That in particular, the four rules-- 130 00:06:41,955 --> 00:06:43,330 actually, the three rules that we 131 00:06:43,330 --> 00:06:47,523 have for defining a dot product tells us 132 00:06:47,523 --> 00:06:49,690 how to compute the dot product for any two elements. 133 00:06:49,690 --> 00:06:53,350 For example, suppose I have one 3 tuple, 134 00:06:53,350 --> 00:06:56,830 and just suppose I have two vectors-- x1u1 plus x2u2 135 00:06:56,830 --> 00:07:02,890 plus x3u3, and y1u1 plus y2u2 plus y3u3. 136 00:07:02,890 --> 00:07:05,960 Let's suppose I want to find that dot product. 137 00:07:05,960 --> 00:07:09,730 Notice that my distributive rule for multiplication, coupled 138 00:07:09,730 --> 00:07:12,550 with the commutative properties and how I can factor out 139 00:07:12,550 --> 00:07:16,180 scale in multiples, tells me that I multiply these two 140 00:07:16,180 --> 00:07:19,720 trinomials the same way as I would have had these 141 00:07:19,720 --> 00:07:21,610 been algebraic expressions. 142 00:07:21,610 --> 00:07:25,820 In other words, I dot x1u1 with each of these three terms, 143 00:07:25,820 --> 00:07:29,290 I dot x2u2 with each of these three terms, 144 00:07:29,290 --> 00:07:32,950 and I dot x3u3 with each of these three terms. 145 00:07:32,950 --> 00:07:39,130 For example, this will give me x1y1 times u1 dot u1. 146 00:07:39,130 --> 00:07:42,910 But if you recall, I said that u1 dot u1 is 3, 147 00:07:42,910 --> 00:07:45,580 so this becomes 3x1y1. 148 00:07:45,580 --> 00:07:49,990 And in a similar way, when I dot x1u1 with y2u2, 149 00:07:49,990 --> 00:07:54,730 I get x1y2, u1 dot u2. 150 00:07:54,730 --> 00:07:58,750 But I gave the fact that u1 dot u2 was 4, 151 00:07:58,750 --> 00:08:02,070 so this term becomes 4x1y2. 152 00:08:02,070 --> 00:08:04,270 And in a similar way, going through this, 153 00:08:04,270 --> 00:08:07,780 I get-- a term will be 5x1y3. 154 00:08:07,780 --> 00:08:11,230 Then, dotting x2u2 with each of these three terms, 155 00:08:11,230 --> 00:08:12,850 I get three more terms-- 156 00:08:12,850 --> 00:08:19,660 4x2y1 plus 6x2y2 plus 7x2y3. 157 00:08:19,660 --> 00:08:24,100 Then I dot x3u3 with each of these three terms. 158 00:08:24,100 --> 00:08:25,700 Again, just to do this thing quickly. 159 00:08:25,700 --> 00:08:35,350 x3u3 dotted with y1u1 is x3y1 times u3 dotted with u1. 160 00:08:35,350 --> 00:08:38,890 And u3 dotted with u1 was given to be 5. 161 00:08:38,890 --> 00:08:42,600 In fact, it was the same as u1 dotted with u3 162 00:08:42,600 --> 00:08:45,190 by commutativity, you see, which was also 5. 163 00:08:45,190 --> 00:08:46,510 At any rate, I get what? 164 00:08:46,510 --> 00:08:52,990 5x3y1 plus 7x3y2 plus 9x3y3. 165 00:08:52,990 --> 00:08:55,570 By the way, at this particular stage, 166 00:08:55,570 --> 00:08:57,880 I would like to pause for a moment. 167 00:08:57,880 --> 00:09:01,210 And some of you may have picked this up better than others, 168 00:09:01,210 --> 00:09:03,730 but at this stage of the game, maybe you've 169 00:09:03,730 --> 00:09:07,690 already learned how to look at a system like this 170 00:09:07,690 --> 00:09:10,140 and write it in matrix form. 171 00:09:10,140 --> 00:09:13,720 Now, notice that these are what we call quadratic forms. 172 00:09:13,720 --> 00:09:17,740 Each term involves an x multiplied by a y. 173 00:09:17,740 --> 00:09:20,080 I've written this so that the x's are always 174 00:09:20,080 --> 00:09:23,980 the left side factor and the y's are always on the right. 175 00:09:23,980 --> 00:09:28,820 Notice that there is a matrix of coefficients-- namely 3, 4, 5, 176 00:09:28,820 --> 00:09:31,790 4, 6, 7, 5, 7, 9. 177 00:09:31,790 --> 00:09:37,570 And that consequently, I can write this array of nine terms 178 00:09:37,570 --> 00:09:39,850 simply as the following product. 179 00:09:39,850 --> 00:09:43,840 I write the row matrix, x1 x2 x3. 180 00:09:43,840 --> 00:09:46,690 Multiply that by the 3 by 3 matrix-- 181 00:09:46,690 --> 00:09:52,720 3 4 5, 4 6 7, 5 7 9, and multiply that, in turn, 182 00:09:52,720 --> 00:09:56,740 by the column matrix, y1 y2 y3. 183 00:09:56,740 --> 00:10:00,370 As a quick check, notice that this is a 1 by 3 matrix. 184 00:10:00,370 --> 00:10:01,940 This is 3 by 3. 185 00:10:01,940 --> 00:10:04,360 This is 3 by 1. 186 00:10:04,360 --> 00:10:08,980 And consequently, this will be a 1 by 1 matrix, or a number. 187 00:10:08,980 --> 00:10:10,400 What number will it be? 188 00:10:10,400 --> 00:10:12,260 It will be this number here. 189 00:10:12,260 --> 00:10:13,870 And notice what I'm saying-- 190 00:10:13,870 --> 00:10:20,590 that x1, x2, x3, y1, y2, y3, are given numbers. 191 00:10:20,590 --> 00:10:23,620 And consequently, the dot product in question 192 00:10:23,620 --> 00:10:27,850 is determined as soon as I know these coefficients, which 193 00:10:27,850 --> 00:10:31,580 happen to be the dot products of the basis elements with one 194 00:10:31,580 --> 00:10:32,520 another. 195 00:10:32,520 --> 00:10:34,440 So this shows two things. 196 00:10:34,440 --> 00:10:37,890 One, how we write this in matrix form. 197 00:10:37,890 --> 00:10:43,220 And two, how the dot product of each two vectors in v 198 00:10:43,220 --> 00:10:45,350 is determined by just knowing what 199 00:10:45,350 --> 00:10:48,890 happens to the pairs of basis vectors when we dot them. 200 00:10:48,890 --> 00:10:50,870 And this gives us still another way 201 00:10:50,870 --> 00:10:54,990 of seeing how matrices are used as a coding system. 202 00:10:54,990 --> 00:10:56,765 In other words, every symmetric-- 203 00:10:56,765 --> 00:10:58,490 remember what symmetric means? 204 00:10:58,490 --> 00:11:01,700 You get the same matrix when you interchange rows and columns. 205 00:11:01,700 --> 00:11:07,370 Every symmetric n by n matrix codes a dot product. 206 00:11:07,370 --> 00:11:10,190 And conversely, every dot product 207 00:11:10,190 --> 00:11:14,120 is coded by a symmetric n by n matrix. 208 00:11:14,120 --> 00:11:16,760 You see, what does this matrix tell me? 209 00:11:16,760 --> 00:11:19,400 This is the entry in the i-- 210 00:11:19,400 --> 00:11:21,980 the entry in the i-th row, j-th column 211 00:11:21,980 --> 00:11:26,570 is what happens when u sub i is dotted with u sub j. 212 00:11:26,570 --> 00:11:28,640 Without meaning to belabor this point, 213 00:11:28,640 --> 00:11:30,890 let us notice that the array here-- 214 00:11:30,890 --> 00:11:37,700 3 4 5, 4 6 7, 5 7 9-- is precisely the array that we had 215 00:11:37,700 --> 00:11:38,480 over here-- 216 00:11:38,480 --> 00:11:42,150 3 4 5, 4 6 7, 5 7 9. 217 00:11:42,150 --> 00:11:44,090 This is what we mean when we talk 218 00:11:44,090 --> 00:11:46,940 about the matrix of the dot product. 219 00:11:46,940 --> 00:11:52,010 It's the array of how the basis vectors look when you dot them 220 00:11:52,010 --> 00:11:53,570 with one another. 221 00:11:53,570 --> 00:11:56,540 By the way, as another aside, we agreed 222 00:11:56,540 --> 00:11:59,480 that when a vector with itself, that 223 00:11:59,480 --> 00:12:03,680 can be identified in a way with the length of the vector. 224 00:12:03,680 --> 00:12:06,140 In particular, going back to this example 225 00:12:06,140 --> 00:12:10,190 here, if the x's and the y's are chosen to be equal-- 226 00:12:10,190 --> 00:12:15,260 if I let x1 equal y1, x2 equal y2, and x3 equals y3, 227 00:12:15,260 --> 00:12:20,930 this problem here reduces to this problem. 228 00:12:20,930 --> 00:12:23,510 Which in turn, just combining what I had above-- 229 00:12:23,510 --> 00:12:26,450 in other words, with the x's and the y's equal-- 230 00:12:26,450 --> 00:12:30,320 gives me that to find the square of the length of a vector, 231 00:12:30,320 --> 00:12:33,770 I have to solve this particular algebraic equation. 232 00:12:33,770 --> 00:12:38,300 And this algebraic equation is called a quadratic form. 233 00:12:38,300 --> 00:12:43,370 You see, notice that each of the variables, x1, x2, x3, 234 00:12:43,370 --> 00:12:48,240 appears in each term here as a second-degree variable. 235 00:12:48,240 --> 00:12:49,820 See, here's an x1 squared. 236 00:12:49,820 --> 00:12:51,590 Here's x1 times x2. 237 00:12:51,590 --> 00:12:52,820 You see two factors-- 238 00:12:52,820 --> 00:12:54,500 x1 times x3. 239 00:12:54,500 --> 00:12:56,660 And what I'm driving at, again, is aside 240 00:12:56,660 --> 00:12:59,280 from any other connections, notice 241 00:12:59,280 --> 00:13:04,370 that to solve for the length of a particular vector, one 242 00:13:04,370 --> 00:13:08,360 has to wind up solving what we call quadratic forms. 243 00:13:08,360 --> 00:13:10,400 And these, for two and three dimensions, 244 00:13:10,400 --> 00:13:13,400 can be identified with certain curves in two- 245 00:13:13,400 --> 00:13:14,990 or three-dimensional space. 246 00:13:14,990 --> 00:13:17,870 But I don't want to belabor that point too much here, either. 247 00:13:17,870 --> 00:13:19,730 I just want you to get a buckshot 248 00:13:19,730 --> 00:13:24,110 overview of how one works abstractly with the dot product 249 00:13:24,110 --> 00:13:25,010 idea. 250 00:13:25,010 --> 00:13:27,740 By the way, some new terminology. 251 00:13:27,740 --> 00:13:33,350 If v is an n dimensional vector space with basis u1 up to u n, 252 00:13:33,350 --> 00:13:37,670 we call the basis u1 up to u n an orthogonal basis 253 00:13:37,670 --> 00:13:41,540 relative to a given dot product if the dot 254 00:13:41,540 --> 00:13:45,860 product of any two different members of the basis is 0. 255 00:13:45,860 --> 00:13:48,260 In other words, if u sub i dot u sub 256 00:13:48,260 --> 00:13:51,380 j is 0 for all i unequal to j, we 257 00:13:51,380 --> 00:13:55,250 call this an orthogonal basis, for the usual reason 258 00:13:55,250 --> 00:13:56,450 of what orthogonal means. 259 00:13:56,450 --> 00:13:59,150 When the dot product of two arrows was zero, 260 00:13:59,150 --> 00:14:01,220 we said the arrows were perpendicular, et cetera. 261 00:14:01,220 --> 00:14:03,440 This is just a carryover from that. 262 00:14:03,440 --> 00:14:06,590 In particular, if in addition, the dot product 263 00:14:06,590 --> 00:14:09,410 of any vector with itself happens to be 1, 264 00:14:09,410 --> 00:14:13,850 then the basis is called an orthonormal basis. 265 00:14:13,850 --> 00:14:18,410 And this whole idea is modeled after the usual example 266 00:14:18,410 --> 00:14:20,420 that in three-dimensional space, i 267 00:14:20,420 --> 00:14:25,100 j k forms an orthonormal basis, relative to the usual dot 268 00:14:25,100 --> 00:14:25,620 product. 269 00:14:25,620 --> 00:14:31,880 In other words, i dot j, i dot k, and j dot k are all 0. 270 00:14:31,880 --> 00:14:36,590 But i dot i, j dot j, and k dot k are all 1. 271 00:14:36,590 --> 00:14:39,620 Now, the interesting point is that in most textbooks, 272 00:14:39,620 --> 00:14:43,340 one always defines the dot product in much the same way 273 00:14:43,340 --> 00:14:45,290 as we do i, j, and k. 274 00:14:45,290 --> 00:14:47,750 One always assumes that somehow or other, 275 00:14:47,750 --> 00:14:50,060 we have an orthonormal basis. 276 00:14:50,060 --> 00:14:51,950 Now, obviously in the example I'm 277 00:14:51,950 --> 00:14:53,960 motivating today's lecture on, we 278 00:14:53,960 --> 00:14:55,770 don't have an orthonormal basis. 279 00:14:55,770 --> 00:14:59,940 Remember, we had something like u1 dot u2 was 4. 280 00:14:59,940 --> 00:15:03,520 Well, see, to be orthonormal, u1 dot u2, first of all, 281 00:15:03,520 --> 00:15:07,540 would have to be 0, and u1 dot u1 would have to be 1. 282 00:15:07,540 --> 00:15:10,990 But in our example, u1 dot u1 is 3. 283 00:15:10,990 --> 00:15:14,170 So the question that comes up is, do we always 284 00:15:14,170 --> 00:15:15,760 have an orthonormal basis? 285 00:15:15,760 --> 00:15:17,620 And the answer is yes. 286 00:15:17,620 --> 00:15:21,610 And in fact, if you like big words, the process by which we 287 00:15:21,610 --> 00:15:25,420 construct an orthonormal basis-- 288 00:15:25,420 --> 00:15:28,000 in fact, the process is easier than the name. 289 00:15:28,000 --> 00:15:31,240 The name is called the Gram-Schmidt orthogonalization 290 00:15:31,240 --> 00:15:32,320 process. 291 00:15:32,320 --> 00:15:35,620 And geometrically, what the method boils down to 292 00:15:35,620 --> 00:15:38,230 is that if I'm given two non-parallel vectors, 293 00:15:38,230 --> 00:15:43,120 u1 and u2, to find the space spanned by u1 and u2, 294 00:15:43,120 --> 00:15:47,470 I can replace u2 by the component of u2, which 295 00:15:47,470 --> 00:15:49,300 is perpendicular to u1. 296 00:15:49,300 --> 00:15:52,030 Let's call that component u2 star. 297 00:15:52,030 --> 00:15:55,030 What I'm saying is that u1 and u2 star 298 00:15:55,030 --> 00:15:58,600 span the same space as u1 and u2. 299 00:15:58,600 --> 00:16:00,640 But obviously, geometrically, it's 300 00:16:00,640 --> 00:16:04,480 easy to see that u1 and u2 star are orthogonal. 301 00:16:04,480 --> 00:16:06,160 Now, the question is, how could we 302 00:16:06,160 --> 00:16:10,150 get the same result without having to rely on geometry? 303 00:16:10,150 --> 00:16:12,040 And the answer is, we say, look it. 304 00:16:12,040 --> 00:16:14,650 We know already in our course that in terms 305 00:16:14,650 --> 00:16:16,780 of spanning vectors, if you replace 306 00:16:16,780 --> 00:16:22,600 a vector by itself plus or minus a scalar multiple of another, 307 00:16:22,600 --> 00:16:24,400 you don't change the space spanned 308 00:16:24,400 --> 00:16:26,510 by the given set of vectors. 309 00:16:26,510 --> 00:16:33,010 So why not write u2 star to be u2 minus some suitable multiple 310 00:16:33,010 --> 00:16:33,940 of u1? 311 00:16:33,940 --> 00:16:37,120 See, u2 minus xu1, where we'll try 312 00:16:37,120 --> 00:16:42,760 to determine x in such a way that u2 star, dotted with u1, 313 00:16:42,760 --> 00:16:43,770 will be 0. 314 00:16:43,770 --> 00:16:45,610 You see, what we're going to try to do 315 00:16:45,610 --> 00:16:49,780 is replace u2 by an equivalent vector, 316 00:16:49,780 --> 00:16:53,890 meaning that u1 and u2 star will span the same space 317 00:16:53,890 --> 00:16:57,340 as u1 and u2, but with the additional property 318 00:16:57,340 --> 00:17:00,670 that u1 and u2 star will be orthogonal. 319 00:17:00,670 --> 00:17:04,359 Whereas, u1 and u2 might not have been orthogonal. 320 00:17:04,359 --> 00:17:07,510 And now you see, we can proceed axiomatically. 321 00:17:07,510 --> 00:17:09,040 Namely, we say, OK. 322 00:17:09,040 --> 00:17:12,160 What we want to look at is u2 star dot u1. 323 00:17:12,160 --> 00:17:16,210 So let's start both sides of this equation with u1. 324 00:17:16,210 --> 00:17:20,150 You see, the left-hand side will become u2 star dot u1. 325 00:17:20,150 --> 00:17:25,690 The right-hand side becomes u2 minus xu1, 1 dotted with u1. 326 00:17:25,690 --> 00:17:27,859 But by our distributive property, 327 00:17:27,859 --> 00:17:33,100 this is the same as u2 dot u1 minus xu1 dot u1. 328 00:17:33,100 --> 00:17:35,440 Now, we didn't want x to be any old number. 329 00:17:35,440 --> 00:17:38,800 We wanted x to be that number, if one exists, 330 00:17:38,800 --> 00:17:42,580 that makes u2 star dot u1 equal to 0. 331 00:17:42,580 --> 00:17:48,580 Remember, u2 dot u1 and u1 dot u1 are numbers. 332 00:17:48,580 --> 00:17:53,050 Consequently, I treat this as an ordinary algebraic equation. 333 00:17:53,050 --> 00:17:55,430 Namely, this is a number. 334 00:17:55,430 --> 00:17:56,650 This is a number. 335 00:17:56,650 --> 00:17:59,650 I equate this to 0 and solve for x. 336 00:17:59,650 --> 00:18:05,350 In other words, x is u2 dot u1 divided by u1 dot u1. 337 00:18:05,350 --> 00:18:08,620 And by the way, just as an aside, many mathematicians, 338 00:18:08,620 --> 00:18:10,480 as an abbreviation when they want 339 00:18:10,480 --> 00:18:13,330 to write the dot product of a vector and itself, 340 00:18:13,330 --> 00:18:16,420 they write it as the square of the given vector. 341 00:18:16,420 --> 00:18:20,950 Obviously, this cannot mean the usual squaring operation, 342 00:18:20,950 --> 00:18:23,830 because we don't multiply vectors by themselves. 343 00:18:23,830 --> 00:18:26,640 This obviously refers to the dot product. 344 00:18:26,640 --> 00:18:28,690 And I mention this in passing, simply 345 00:18:28,690 --> 00:18:30,580 so that if you see this in the literature, 346 00:18:30,580 --> 00:18:32,290 you will not be confused by it. 347 00:18:32,290 --> 00:18:36,520 I prefer to write u1 dot u1 rather than u1 squared, 348 00:18:36,520 --> 00:18:39,460 but we'd like you to see this notation anyway. 349 00:18:39,460 --> 00:18:42,770 At any rate, now knowing what x is, 350 00:18:42,770 --> 00:18:46,960 I replace x here into this equation, 351 00:18:46,960 --> 00:18:49,780 and I wind up with that u2 star is 352 00:18:49,780 --> 00:18:56,410 u2 minus u2 dot u1 over u1 dot u1 times u1. 353 00:18:56,410 --> 00:18:58,590 And if you want to check this thing geometrically, 354 00:18:58,590 --> 00:19:01,330 the u2 star that I've computed this way 355 00:19:01,330 --> 00:19:05,920 is precisely the u2 star that I get geometrically. 356 00:19:05,920 --> 00:19:08,620 The beauty of this technique is that it did not 357 00:19:08,620 --> 00:19:10,270 require geometry. 358 00:19:10,270 --> 00:19:12,550 And consequently, structurally, I 359 00:19:12,550 --> 00:19:15,560 should be able to use this in higher dimensions. 360 00:19:15,560 --> 00:19:18,470 And that's exactly what the Gram-Schmidt orthogonalization 361 00:19:18,470 --> 00:19:19,600 process is. 362 00:19:19,600 --> 00:19:23,200 What you do is is you start with a basis. 363 00:19:23,200 --> 00:19:25,420 You take the first vector in that basis 364 00:19:25,420 --> 00:19:27,655 and dot it with the second vector. 365 00:19:27,655 --> 00:19:30,670 If that dot product is 0, those two vectors 366 00:19:30,670 --> 00:19:34,810 are both eligible to be part of an orthogonal basis. 367 00:19:34,810 --> 00:19:37,180 Now, if that dot product isn't 0, 368 00:19:37,180 --> 00:19:40,180 you use the Gram-Schmidt orthogonalization process 369 00:19:40,180 --> 00:19:43,660 to replace the second vector, u2, by u2 star, 370 00:19:43,660 --> 00:19:47,080 where u1 dot u2 star is 0. 371 00:19:47,080 --> 00:19:50,470 Now you take that new basis, which spans the same space 372 00:19:50,470 --> 00:19:54,370 as the original u1 and u2, and compare that 373 00:19:54,370 --> 00:19:57,220 or take that in connection with u3 374 00:19:57,220 --> 00:20:00,360 and see if those three vectors form an orthogonal basis-- 375 00:20:00,360 --> 00:20:03,840 or orthogonal, meaning their dot product of different ones is 0. 376 00:20:03,840 --> 00:20:05,280 And if it is, fine. 377 00:20:05,280 --> 00:20:07,650 And if it isn't, you just inductively keep 378 00:20:07,650 --> 00:20:10,280 using the Gram-Schmidt orthogonalization process. 379 00:20:10,280 --> 00:20:12,900 Now, to show you what this means in a specific case, 380 00:20:12,900 --> 00:20:15,690 let's suppose I have the following situation. 381 00:20:15,690 --> 00:20:17,760 Let's suppose I have a four-dimensional vector 382 00:20:17,760 --> 00:20:23,130 space for which a basis is u1, u2, u3, and u4. 383 00:20:23,130 --> 00:20:28,740 Suppose I also know that u1, u2, and u3 form an orthogonal set. 384 00:20:28,740 --> 00:20:33,180 That means I know that u1 dot u2, u1 dot u3, 385 00:20:33,180 --> 00:20:36,770 and u2 dot u3 are all 0. 386 00:20:36,770 --> 00:20:40,430 But I don't know that u4 dot u1, u4 dot u2, 387 00:20:40,430 --> 00:20:43,063 u4 dot u3 are 0 or anything like this. 388 00:20:43,063 --> 00:20:44,480 And the question that comes up now 389 00:20:44,480 --> 00:20:49,160 is, I would like to replace u4 by an equivalent vector, which 390 00:20:49,160 --> 00:20:54,950 I'll call u sub 4 star, such that u1, u2, u3, and u sub 391 00:20:54,950 --> 00:20:59,170 4 star span the same space, v. But 392 00:20:59,170 --> 00:21:02,480 that now with u4 replaced by u4 star, 393 00:21:02,480 --> 00:21:06,300 this becomes an orthogonal basis. 394 00:21:06,300 --> 00:21:09,500 Well, the way I proceed is I use the fact 395 00:21:09,500 --> 00:21:13,760 that whenever I replace a vector by itself, plus or minus 396 00:21:13,760 --> 00:21:16,580 a linear combination of the other vectors, 397 00:21:16,580 --> 00:21:19,400 I don't change the space spanned by the vectors. 398 00:21:19,400 --> 00:21:23,450 So what I say is, let u4 star be u4 399 00:21:23,450 --> 00:21:28,490 minus x1u1 minus x2u2 minus x3u3. 400 00:21:28,490 --> 00:21:30,620 You see, I'm modeling this after what I did 401 00:21:30,620 --> 00:21:33,020 in the two-dimensional case. 402 00:21:33,020 --> 00:21:37,130 What I want to do is determine x1, x2, and x3 403 00:21:37,130 --> 00:21:41,840 such that u4 star dot u1, u4 star dot u2, 404 00:21:41,840 --> 00:21:45,080 and u4 star dot u3 will all be 0. 405 00:21:45,080 --> 00:21:46,610 And the trick is simply this. 406 00:21:46,610 --> 00:21:49,100 Let me focus my attention on one of these coefficients, 407 00:21:49,100 --> 00:21:52,410 because the technique is the same for all the others. 408 00:21:52,410 --> 00:21:55,280 Let's suppose I want to find out what x3 is. 409 00:21:55,280 --> 00:22:00,830 The trick is, I dot both sides of this equation with u3. 410 00:22:00,830 --> 00:22:02,460 Now, why do I do that? 411 00:22:02,460 --> 00:22:06,560 Remember, u1, u2, and u3 are an orthogonal set. 412 00:22:06,560 --> 00:22:11,000 Consequently, when I dot u1 with u3 and u2 with u3, 413 00:22:11,000 --> 00:22:13,940 those terms will be 0, and they will drop out. 414 00:22:13,940 --> 00:22:18,290 In other words, dotting both sides of this equation with u3 415 00:22:18,290 --> 00:22:22,040 and using the fact that the dot product is distributive, 416 00:22:22,040 --> 00:22:26,480 I get u4 star dot u3 is u4 dot u3 417 00:22:26,480 --> 00:22:34,940 minus x1u1 dot u3 minus x2u2 dot u3 minus x3u3 dot u3. 418 00:22:34,940 --> 00:22:40,250 And the key point is that by orthogonality, these two things 419 00:22:40,250 --> 00:22:41,810 here are 0. 420 00:22:41,810 --> 00:22:50,420 Consequently, u4 star dot u3 is u4 dot u3 minus x3u3 dot u3. 421 00:22:50,420 --> 00:22:52,310 Remember, these are numbers. 422 00:22:52,310 --> 00:22:56,930 I would like u4 star dot u3 to be 0, so all I have to do now 423 00:22:56,930 --> 00:23:00,230 is solve this algebraic equation for x3. 424 00:23:00,230 --> 00:23:03,680 And notice that x3 algebraically simply turns out 425 00:23:03,680 --> 00:23:07,500 to be u4 dot u3 over u3 dot u3. 426 00:23:07,500 --> 00:23:10,370 Again, the only thing I have to be careful about 427 00:23:10,370 --> 00:23:12,230 is that the denominator not be 0. 428 00:23:12,230 --> 00:23:14,400 And notice that by positive definiteness, 429 00:23:14,400 --> 00:23:20,900 u3 dot u3 can only equal 0 if u3 is the 0 vector. 430 00:23:20,900 --> 00:23:25,520 But if u3 had been the 0 vector, it couldn't be part of a basis. 431 00:23:25,520 --> 00:23:27,930 So you get the idea of how this works. 432 00:23:27,930 --> 00:23:31,280 This is the same Gram-Schmidt orthogonalization process 433 00:23:31,280 --> 00:23:33,110 that we used in two dimensions. 434 00:23:33,110 --> 00:23:36,470 Because once we know that all the vectors up to u sub 4 435 00:23:36,470 --> 00:23:39,290 are orthogonal, whenever we dot one of them 436 00:23:39,290 --> 00:23:41,210 with the other three, we essentially 437 00:23:41,210 --> 00:23:44,120 reduce this to a two-dimensional problem, 438 00:23:44,120 --> 00:23:47,210 because all the other dot products drop out. 439 00:23:47,210 --> 00:23:49,130 In a similar way, you see, I could 440 00:23:49,130 --> 00:23:53,270 have found x sub 2 and x sub 1, leaving the details to you. 441 00:23:53,270 --> 00:23:55,250 The easiest way to remember this is 442 00:23:55,250 --> 00:23:57,800 every place I see a subscript 3 over here, 443 00:23:57,800 --> 00:23:59,960 let me replace it by a 2. 444 00:23:59,960 --> 00:24:03,770 And then every place I see the 3, let me replace it by a 1. 445 00:24:03,770 --> 00:24:07,700 And what I get is that u4 star is u4. 446 00:24:07,700 --> 00:24:10,760 And then I subtract off these scalar multiples. 447 00:24:10,760 --> 00:24:12,410 What scalar multiples are they? 448 00:24:12,410 --> 00:24:14,000 Notice that it's what? 449 00:24:14,000 --> 00:24:21,380 The coefficient of u1 is u4 dot u1 over u1 dot u1. 450 00:24:21,380 --> 00:24:26,240 The coefficient of u2 has as its denominator u2 dot u2. 451 00:24:26,240 --> 00:24:29,430 The numerator is u4 dotted with u2. 452 00:24:29,430 --> 00:24:32,280 In other words, in a sense, it's like the projection of u4 453 00:24:32,280 --> 00:24:33,530 onto u2. 454 00:24:33,530 --> 00:24:38,780 And the coefficient of u3 has as its denominator u3 dot u3, 455 00:24:38,780 --> 00:24:42,530 and its denominator is u4 dot u3. 456 00:24:42,530 --> 00:24:44,660 Now, to show you how this works, let's 457 00:24:44,660 --> 00:24:46,850 come back to our original problem, which 458 00:24:46,850 --> 00:24:48,390 I've reproduced over here. 459 00:24:48,390 --> 00:24:49,265 Remember what we had. 460 00:24:49,265 --> 00:24:53,600 We had that u1 dot u1 is 3, u1 dot u2 is 4, 461 00:24:53,600 --> 00:24:56,090 u1 dot u3 is 5, et cetera. 462 00:24:56,090 --> 00:24:57,980 I'll refer to this as I need it. 463 00:24:57,980 --> 00:25:01,100 Let me show you how the Gram-Schmidt process allows 464 00:25:01,100 --> 00:25:04,520 me to find, from this, an orthogonal basis. 465 00:25:04,520 --> 00:25:07,040 The first thing is that obviously, u1 466 00:25:07,040 --> 00:25:10,490 is not the 0 vector, because if it were the 0 vector, 467 00:25:10,490 --> 00:25:13,460 u1 dot u1 would be 0, not 3. 468 00:25:13,460 --> 00:25:16,070 So the first vector in my orthogonal basis, 469 00:25:16,070 --> 00:25:20,340 which I'll call u1 star, can be u1 itself. 470 00:25:20,340 --> 00:25:22,460 How do I find the second vector? 471 00:25:22,460 --> 00:25:25,640 According to the Gram-Schmidt orthogonalization process, 472 00:25:25,640 --> 00:25:30,140 I replace u2 by u2 minus a suitable scalar 473 00:25:30,140 --> 00:25:32,480 multiple of u1 star. 474 00:25:32,480 --> 00:25:35,210 And what scalar multiple is that? 475 00:25:35,210 --> 00:25:38,700 The denominator will be u1 star dot u1 star, 476 00:25:38,700 --> 00:25:42,330 and the numerator will be u2 dot u1 star. 477 00:25:42,330 --> 00:25:45,120 Well, remember, u1 star is equal to u1. 478 00:25:45,120 --> 00:25:48,780 So right from this array here, what do I have? 479 00:25:48,780 --> 00:25:52,560 u2 dot u1 star is u2 dot u1. 480 00:25:52,560 --> 00:25:55,410 u2 dot u1 is 4. 481 00:25:55,410 --> 00:25:59,400 u1 star dot u1 star is u1 dot u1. 482 00:25:59,400 --> 00:26:01,440 u1 dot u1 is 3. 483 00:26:01,440 --> 00:26:02,700 So this is simply what? 484 00:26:02,700 --> 00:26:06,600 Minus 4 over 3 times u1. 485 00:26:06,600 --> 00:26:12,060 In other words, u2 star is simply u2 minus 4/3 u1. 486 00:26:12,060 --> 00:26:15,420 And by the way, as a check, what was this supposed to do? 487 00:26:15,420 --> 00:26:18,990 If I replaced u2 by u2 star, these two vectors 488 00:26:18,990 --> 00:26:21,030 here should now be orthogonal. 489 00:26:21,030 --> 00:26:21,790 Are they? 490 00:26:21,790 --> 00:26:22,830 Let's start them. 491 00:26:22,830 --> 00:26:25,230 If I dot these two, I get what? 492 00:26:25,230 --> 00:26:30,390 u1 dot u2 minus 4/3 u1 dot u1. 493 00:26:30,390 --> 00:26:33,630 Notice that u1 dot u2 is 4. 494 00:26:33,630 --> 00:26:37,050 u1 dot u1 is 3. 495 00:26:37,050 --> 00:26:39,750 So coming down here as a check, u1 star 496 00:26:39,750 --> 00:26:48,390 dot u2 star is u1 minus u2, which is 4, minus 4/3 u1 dot 497 00:26:48,390 --> 00:26:49,920 u1, which is 3. 498 00:26:49,920 --> 00:26:51,100 The 3's cancel. 499 00:26:51,100 --> 00:26:53,460 4 minus 4 is 0. 500 00:26:53,460 --> 00:26:55,620 This checks out fine. 501 00:26:55,620 --> 00:26:57,930 How do I find u3 star? 502 00:26:57,930 --> 00:27:02,430 Well, I subtract from u3 suitable scalar multiples 503 00:27:02,430 --> 00:27:04,440 of u1 star and u2 star. 504 00:27:04,440 --> 00:27:05,670 You see what I've done. 505 00:27:05,670 --> 00:27:08,840 I orthogonalized one vector at a time. 506 00:27:08,840 --> 00:27:10,950 See, first I got u2 star. 507 00:27:10,950 --> 00:27:13,230 Now I know that u1 star and u2 star 508 00:27:13,230 --> 00:27:16,950 are orthogonal, using u1 star and u2 star, 509 00:27:16,950 --> 00:27:21,050 the multiples that I subtract from u3 are what? 510 00:27:21,050 --> 00:27:25,670 For u1 star, my denominator is u1 star dot u1 star, 511 00:27:25,670 --> 00:27:30,060 and my numerator is u3 dot u1 star. 512 00:27:30,060 --> 00:27:34,370 And for u2 star, my denominator is u2 star dot u2 star, 513 00:27:34,370 --> 00:27:37,980 and my numerator is u3 dot u2 star. 514 00:27:37,980 --> 00:27:42,300 And now how do I evaluate what this thing really means? 515 00:27:42,300 --> 00:27:46,710 Well, first of all, I know what u3 dot u1 star is. 516 00:27:46,710 --> 00:27:52,160 That's just u3 dot u1, which happens to be 5. 517 00:27:52,160 --> 00:27:54,930 And I also know what u1 star dot u1 star is. 518 00:27:54,930 --> 00:27:57,600 That's u1 dot u1, which is 3. 519 00:27:57,600 --> 00:27:59,360 I now have to compute what? 520 00:27:59,360 --> 00:28:05,490 u3 dot u2 star, and I have to compute u2 dot u2 star. 521 00:28:05,490 --> 00:28:09,540 Keep in mind that I've already computed that u2 star is 522 00:28:09,540 --> 00:28:12,390 u2 minus 4/3 u1. 523 00:28:12,390 --> 00:28:20,160 Consequently, u3 dot u2 star is u3 dot u2 minus 4/3 u1. 524 00:28:20,160 --> 00:28:23,700 And now you see the mechanical algebra takes over again. 525 00:28:23,700 --> 00:28:26,640 I know what my axioms for dot products are. 526 00:28:26,640 --> 00:28:32,680 This gives me u3 dot u2, minus 4/3 u3 dot u1. 527 00:28:32,680 --> 00:28:35,830 I'm given that u3 dot u2 is 7. 528 00:28:35,830 --> 00:28:38,610 I'm given that u3 dot u1 is 5. 529 00:28:38,610 --> 00:28:43,260 So u3 dot u2 star is 7 minus 4/3 of 5. 530 00:28:43,260 --> 00:28:49,110 That's 1/3-- 21 minus 20 over 3, which is 1/3. 531 00:28:49,110 --> 00:28:52,680 So let's see what I know now. 532 00:28:52,680 --> 00:28:56,930 I know now how to handle this numerator. 533 00:28:56,930 --> 00:29:00,180 u3 dot u2 star is 1/3. 534 00:29:00,180 --> 00:29:03,740 How do I handle u2 star dot u2 star? 535 00:29:03,740 --> 00:29:07,480 u2 star is u2 minus 4/3 u1. 536 00:29:07,480 --> 00:29:13,080 So to find u2 star dot u2 star, I have to dot u2 minus 4/3 u1 537 00:29:13,080 --> 00:29:14,370 with itself. 538 00:29:14,370 --> 00:29:16,710 If I do that-- and again, notice how 539 00:29:16,710 --> 00:29:19,650 these rules work the same way as for ordinary algebra. 540 00:29:19,650 --> 00:29:28,500 I get u2 dot u2 minus 8/3 u1 dot u2 plus 16/9 u1 dot u1. 541 00:29:28,500 --> 00:29:31,170 Well, I know that u2 dot u2 is 6. 542 00:29:31,170 --> 00:29:32,310 That was given. 543 00:29:32,310 --> 00:29:37,950 I know that u2 dot u2 is 6, u1 dot u2 is 4, 544 00:29:37,950 --> 00:29:40,590 and u1 dot u1 is 3. 545 00:29:40,590 --> 00:29:44,040 So evaluating this, I wind up with the fact 546 00:29:44,040 --> 00:29:51,130 that u2 star dot u2 star is 6 minus 16/3, or 2/3. 547 00:29:51,130 --> 00:29:56,020 Consequently, going back here, you see my numerator is 1/3. 548 00:29:56,020 --> 00:30:01,150 My denominator is 2/3, so the fraction is 1/2. 549 00:30:01,150 --> 00:30:03,110 In other words, substituting into here, 550 00:30:03,110 --> 00:30:07,100 I get that u3 star is u3 minus 5/3 551 00:30:07,100 --> 00:30:11,440 u1 minus 1/2 u2 minus 4/3 u1. 552 00:30:11,440 --> 00:30:14,560 And retabulating my results to write them 553 00:30:14,560 --> 00:30:18,250 in the usual order of u1, u2, and u3 components, 554 00:30:18,250 --> 00:30:21,340 notice that u1 star is u1. 555 00:30:21,340 --> 00:30:25,210 u2 star is minus 4/3 u1 plus u2. 556 00:30:25,210 --> 00:30:30,430 u3 star is minus u1 minus 1/2 u2 plus u3. 557 00:30:30,430 --> 00:30:33,370 Now, what property does this set of vectors have? 558 00:30:33,370 --> 00:30:36,430 First of all, my claim is that they're orthogonal. 559 00:30:36,430 --> 00:30:40,840 And secondly, they span the same space as u1, u2, and u3. 560 00:30:40,840 --> 00:30:45,170 And it's clear why they span the same space as u1, u2, and u3. 561 00:30:45,170 --> 00:30:48,430 Namely, they're essentially suitable scalar multiples 562 00:30:48,430 --> 00:30:49,540 of u1, u2-- 563 00:30:49,540 --> 00:30:54,315 linear combinations of scalar multiples of u1, u2, and u3. 564 00:30:54,315 --> 00:30:55,690 As a quick check-- and I'll leave 565 00:30:55,690 --> 00:30:57,340 some of the other details to you. 566 00:30:57,340 --> 00:31:01,990 Let's actually check that u1 star dot u3 star is really 0. 567 00:31:01,990 --> 00:31:05,740 If I dot u1 star with u3 star, look at what I get. 568 00:31:05,740 --> 00:31:14,650 I get minus u1 dot u1 minus 1/2 u1 dot u2 plus u1 dot u3. 569 00:31:14,650 --> 00:31:16,720 u1 dot u1 is 3. 570 00:31:16,720 --> 00:31:18,880 u1 dot u2 is 4. 571 00:31:18,880 --> 00:31:22,420 So this is minus 3 minus 1/2 of 4. 572 00:31:22,420 --> 00:31:23,740 That's minus 2. 573 00:31:23,740 --> 00:31:28,900 u1 dot u3 is 5, minus 3 minus 2 plus 5 is 0. 574 00:31:28,900 --> 00:31:34,630 A similar check will show that u2 star dot u3 star is 0. 575 00:31:34,630 --> 00:31:37,900 We should also check to see what the lengths of u1 star, 576 00:31:37,900 --> 00:31:39,820 u2 star, and u3 star are. 577 00:31:39,820 --> 00:31:43,750 In other words, u1 star dot u1 star is what? 578 00:31:43,750 --> 00:31:46,330 That's u1 dot u1, which is 3. 579 00:31:46,330 --> 00:31:48,580 u2 star dot u2 star-- 580 00:31:48,580 --> 00:31:51,140 well, we just found that over here. 581 00:31:51,140 --> 00:31:53,230 That's 2/3. 582 00:31:53,230 --> 00:31:55,540 u3 star dot u3 star-- 583 00:31:55,540 --> 00:32:00,900 well, u3 star was minus u1 minus 1/2 u2 plus u3. 584 00:32:00,900 --> 00:32:02,620 Dotting that with itself-- and notice 585 00:32:02,620 --> 00:32:05,500 I used the square notation here to save space 586 00:32:05,500 --> 00:32:07,270 and not have this run too long. 587 00:32:07,270 --> 00:32:09,880 But dotting this, the same as I would 588 00:32:09,880 --> 00:32:12,490 the square of any trinomial, I get what? 589 00:32:12,490 --> 00:32:21,160 u1 dot u1 plus 1/4 u2 dot u2 plus u3 dot u3 plus u1 dot u2 590 00:32:21,160 --> 00:32:26,590 minus twice u1 dot u3 minus u2 dot u3. 591 00:32:26,590 --> 00:32:29,350 Putting in the values for u1 dot u1, 592 00:32:29,350 --> 00:32:37,540 for u2 dot u2, u3 dot u3, u1 dot u2, u1 dot u3, and u2 dot u3-- 593 00:32:37,540 --> 00:32:40,420 putting in these values, I find that u3 star 594 00:32:40,420 --> 00:32:42,760 dot u3 star is 1/2. 595 00:32:42,760 --> 00:32:46,030 So I have an orthogonal basis. 596 00:32:46,030 --> 00:32:48,610 It's not orthonormal, but notice that it's 597 00:32:48,610 --> 00:32:54,100 easy to fix up the normality part, because all I have to do 598 00:32:54,100 --> 00:32:56,950 is divide each of these vectors by the square root 599 00:32:56,950 --> 00:33:00,430 of this number, and that will make the dot product 1. 600 00:33:00,430 --> 00:33:03,190 For example, if I were to replace u1 star 601 00:33:03,190 --> 00:33:05,830 by u1 star over the square root of 3, 602 00:33:05,830 --> 00:33:07,540 then when I dotted these two vectors, 603 00:33:07,540 --> 00:33:10,300 I would have 3 divided by the square root 604 00:33:10,300 --> 00:33:13,180 of 3 times the square root of 3, which is 3. 605 00:33:13,180 --> 00:33:14,960 And 3 over 3 is 1. 606 00:33:14,960 --> 00:33:17,170 So getting the lengths to be one is quite easy. 607 00:33:17,170 --> 00:33:19,960 The hard part is this Gram-Schmidt orthogonalization 608 00:33:19,960 --> 00:33:24,520 process to orthogonalize what's going on. 609 00:33:24,520 --> 00:33:27,440 Now, this may seem rather complicated, 610 00:33:27,440 --> 00:33:30,670 but it's really one long piece of computation. 611 00:33:30,670 --> 00:33:33,470 We'll do this more slowly in the exercises. 612 00:33:33,470 --> 00:33:34,930 But the whole idea is what? 613 00:33:34,930 --> 00:33:38,560 I successively use the Gram-Schmidt orthogonalization 614 00:33:38,560 --> 00:33:42,460 process to replace each vector by 615 00:33:42,460 --> 00:33:46,960 a suitable linear combination of itself and the preceding ones 616 00:33:46,960 --> 00:33:49,630 to make sure that the new replacement is 617 00:33:49,630 --> 00:33:51,640 orthogonal to the remaining ones, 618 00:33:51,640 --> 00:33:54,880 and that doesn't change the space that I already have. 619 00:33:54,880 --> 00:33:56,590 By the way, if you want to see what 620 00:33:56,590 --> 00:34:00,280 this means in terms of matrix multiplication, 621 00:34:00,280 --> 00:34:04,740 notice that we saw earlier that to dot a vector with itself, 622 00:34:04,740 --> 00:34:08,080 you write the vector as a row vector, 623 00:34:08,080 --> 00:34:11,020 then write down the matrix of coefficients for the basis 624 00:34:11,020 --> 00:34:15,370 elements, then write down that same vector as a column vector. 625 00:34:15,370 --> 00:34:21,790 The fact that u1 star is u1 plus 0 u2 plus 0 u3 626 00:34:21,790 --> 00:34:26,750 means that as a 3 tuple, it would be 1 0 0. 627 00:34:26,750 --> 00:34:31,210 It's transpose would be 1 0 0-- the column vector. 628 00:34:31,210 --> 00:34:34,270 And so when we take this vector and dot it 629 00:34:34,270 --> 00:34:36,790 with this vector, that should be the entry, 630 00:34:36,790 --> 00:34:40,420 u1 dot u1, in this particular matrix here. 631 00:34:40,420 --> 00:34:43,270 If we take this times this, this should 632 00:34:43,270 --> 00:34:48,280 be u1 star dotted with u2 star. 633 00:34:48,280 --> 00:34:51,580 And since u1 star and u2 star are orthogonal, 634 00:34:51,580 --> 00:34:54,100 that had better come out to be 0. 635 00:34:54,100 --> 00:34:56,350 And going on in this way, what I claim 636 00:34:56,350 --> 00:35:02,410 is, if you now replace u1 star, u2 star, and u3 star 637 00:35:02,410 --> 00:35:05,800 by what 3 tuples they are, then also replace them 638 00:35:05,800 --> 00:35:10,780 as column vectors here, you will get a new 3 by 3 matrix which 639 00:35:10,780 --> 00:35:12,640 will be diagonal. 640 00:35:12,640 --> 00:35:18,670 In other words, if we change our basis from u1, u2, and u3-- 641 00:35:18,670 --> 00:35:26,350 if we change that basis to u1 star, u2 star, u3 star, 642 00:35:26,350 --> 00:35:30,660 notice that the new matrix that we get is a diagonal matrix. 643 00:35:30,660 --> 00:35:33,940 It's our way of saying that when you dot two different vectors, 644 00:35:33,940 --> 00:35:35,640 the dot product is 0. 645 00:35:35,640 --> 00:35:38,080 The diagonal elements tell you what 646 00:35:38,080 --> 00:35:43,180 u1 star dot u1 star are-- u2 star dot u2 star, 647 00:35:43,180 --> 00:35:45,850 u3 star dot u3 star. 648 00:35:45,850 --> 00:35:49,570 You see, by using the diagonal matrix over here, 649 00:35:49,570 --> 00:35:52,180 this is a much simpler model to use. 650 00:35:52,180 --> 00:35:56,170 u1 star, u2 star, u3 star is a much nicer basis 651 00:35:56,170 --> 00:35:59,560 to use than u1, u2, and u3 in order 652 00:35:59,560 --> 00:36:02,770 to determine what dot products are. 653 00:36:02,770 --> 00:36:05,860 I'll emphasize that in more detail later. 654 00:36:05,860 --> 00:36:09,460 Let me make one remark before I get to our concluding remarks. 655 00:36:09,460 --> 00:36:14,500 And that is, have you begun to notice how complicated matrices 656 00:36:14,500 --> 00:36:16,690 are from the point of view that they 657 00:36:16,690 --> 00:36:19,870 are such a wonderful coding device that we use them 658 00:36:19,870 --> 00:36:21,710 to code many different things? 659 00:36:21,710 --> 00:36:23,440 For example, we have used matrices 660 00:36:23,440 --> 00:36:28,670 to code a linear transformation relative to various bases. 661 00:36:28,670 --> 00:36:32,230 We're now using matrices to code the same dot product 662 00:36:32,230 --> 00:36:33,880 relative to different bases. 663 00:36:33,880 --> 00:36:39,790 We have used matrices to code equivalent row-reduced bases 664 00:36:39,790 --> 00:36:42,460 as bases for the same vector space. 665 00:36:42,460 --> 00:36:45,790 That at the end of this unit, what I will do 666 00:36:45,790 --> 00:36:47,950 is try to give you a summary in the study 667 00:36:47,950 --> 00:36:50,862 guide of the different kinds of matrix equivalents 668 00:36:50,862 --> 00:36:51,820 and where they're used. 669 00:36:51,820 --> 00:36:53,650 Because after a while, you tend to get 670 00:36:53,650 --> 00:36:57,520 mixed up if we don't see these pieces one bit at a time. 671 00:36:57,520 --> 00:37:01,360 But to help you see what all this problem means, 672 00:37:01,360 --> 00:37:03,400 I have cheated-- 673 00:37:03,400 --> 00:37:06,850 that the place I got the data from for the u1, u2, 674 00:37:06,850 --> 00:37:09,070 and u3 in this problem was that I 675 00:37:09,070 --> 00:37:11,470 thought of the following three vectors-- 676 00:37:11,470 --> 00:37:17,770 i plus j plus k, 2i plus j plus k, 2i plus j plus 2k-- 677 00:37:17,770 --> 00:37:22,450 and used this to make up the data for u1, u2, and u3 that 678 00:37:22,450 --> 00:37:25,330 was in this exercise. 679 00:37:25,330 --> 00:37:27,940 What you may find informative, and what 680 00:37:27,940 --> 00:37:30,250 I will do as a learning exercise, 681 00:37:30,250 --> 00:37:32,620 is have you go through the same exercise 682 00:37:32,620 --> 00:37:35,410 that we have just done as today's lesson 683 00:37:35,410 --> 00:37:40,300 and have you verify that this checks out with the equivalent 684 00:37:40,300 --> 00:37:42,250 geometric construction-- that one 685 00:37:42,250 --> 00:37:44,290 of the exercises in the homework will 686 00:37:44,290 --> 00:37:48,580 be to show what the Gram-Schmidt orthogonalization process means 687 00:37:48,580 --> 00:37:51,370 geometrically for these three vectors. 688 00:37:51,370 --> 00:37:54,520 By the way, rather than to belabor that point for now, 689 00:37:54,520 --> 00:37:56,680 let me make one other aside. 690 00:37:56,680 --> 00:37:59,080 And that is, I had mentioned before that 691 00:37:59,080 --> 00:38:02,650 to find the dot product of two vectors, 692 00:38:02,650 --> 00:38:04,840 it may be more advantageous to use 693 00:38:04,840 --> 00:38:06,850 one basis rather than another. 694 00:38:06,850 --> 00:38:10,240 Remember, when we wanted to dot a vector with itself, 695 00:38:10,240 --> 00:38:13,570 using u1, u2, and u3 as a basis, remember 696 00:38:13,570 --> 00:38:15,880 we got six different terms? 697 00:38:15,880 --> 00:38:21,070 We had an x1 squared, x2 squared, x3 squared, an x1x2, 698 00:38:21,070 --> 00:38:24,790 x1x3, and x2x3 term. 699 00:38:24,790 --> 00:38:27,430 Notice that when you pick an orthogonal basis, 700 00:38:27,430 --> 00:38:29,350 you only get three terms. 701 00:38:29,350 --> 00:38:33,310 Because once the basis is orthogonal, all you have to do 702 00:38:33,310 --> 00:38:36,040 is dot corresponding entries. 703 00:38:36,040 --> 00:38:38,800 Because you see, if I dot u1 star 704 00:38:38,800 --> 00:38:41,890 with a term involving u2 star or u3 star, 705 00:38:41,890 --> 00:38:44,170 that will be 0 by orthogonality. 706 00:38:44,170 --> 00:38:46,270 If I didn't have an orthogonal basis, 707 00:38:46,270 --> 00:38:48,500 I couldn't neglect those terms. 708 00:38:48,500 --> 00:38:51,520 So for example, to dot a vector with itself 709 00:38:51,520 --> 00:38:54,640 in this particular example, using u1 star, u2 star, 710 00:38:54,640 --> 00:38:57,880 and u3 star as a basis, notice that what I wind up with 711 00:38:57,880 --> 00:38:58,810 is what? 712 00:38:58,810 --> 00:39:03,610 x1 times x1. x1 squared times u1 star 713 00:39:03,610 --> 00:39:06,130 dot u1 star, which is 3, et cetera. 714 00:39:06,130 --> 00:39:11,200 Meaning 3x1 squared plus 2/3 x2 squared plus 1/2 x3 715 00:39:11,200 --> 00:39:13,540 squared, which is the square of the length of the vector 716 00:39:13,540 --> 00:39:14,830 that I'm looking for. 717 00:39:14,830 --> 00:39:17,350 If I put everything over a common denominator, 718 00:39:17,350 --> 00:39:21,400 I get a very nice equation for an ellipsoid over here. 719 00:39:21,400 --> 00:39:24,250 You see a very nice equation with no mixed terms-- 720 00:39:24,250 --> 00:39:27,190 just perfect squares appearing over here. 721 00:39:27,190 --> 00:39:30,250 The mixed terms drop out, and that's one reason 722 00:39:30,250 --> 00:39:32,740 why we like orthogonal bases. 723 00:39:32,740 --> 00:39:35,710 Still another reason that we like orthogonal bases-- 724 00:39:35,710 --> 00:39:37,720 and I'll conclude with this example-- 725 00:39:37,720 --> 00:39:41,230 is that if we know that the vectors u1 up to u n 726 00:39:41,230 --> 00:39:45,640 are orthonormal, say, and that x1u1 plus et cetera x n u 727 00:39:45,640 --> 00:39:51,090 n is 0, then we automatically know that the u's are-- 728 00:39:51,090 --> 00:39:53,820 that x1 up to x n are 0-- 729 00:39:53,820 --> 00:39:55,740 that these are linearly independent. 730 00:39:55,740 --> 00:39:57,360 And the proof is quite simple. 731 00:39:57,360 --> 00:40:00,750 Namely, all we do is we dot both sides of this equation 732 00:40:00,750 --> 00:40:02,460 with, for example, u sub 1. 733 00:40:02,460 --> 00:40:04,590 We could've used any one of the u's that we wanted, 734 00:40:04,590 --> 00:40:07,170 but using u sub 1, we dot u sub 1 735 00:40:07,170 --> 00:40:08,970 with both sides of this equation. 736 00:40:08,970 --> 00:40:13,230 Notice that this term gives me x1u1 dot u1. 737 00:40:13,230 --> 00:40:14,730 The next terms are what? 738 00:40:14,730 --> 00:40:18,270 u1 dot u2 up to u1 dot u n. 739 00:40:18,270 --> 00:40:20,610 By the property of being orthogonal, 740 00:40:20,610 --> 00:40:22,530 all of these products are 0. 741 00:40:22,530 --> 00:40:26,490 Consequently, this scalar times x1 must be 0. 742 00:40:26,490 --> 00:40:30,060 Consequently, x1 itself must be 0. 743 00:40:30,060 --> 00:40:32,430 I've deliberately gone very rapidly here 744 00:40:32,430 --> 00:40:34,860 to give you an overview, and to also 745 00:40:34,860 --> 00:40:37,470 have you see, as I'm talking, what 746 00:40:37,470 --> 00:40:39,120 these computations look like. 747 00:40:39,120 --> 00:40:42,790 I think by hearing this, as complicated as it may sound, 748 00:40:42,790 --> 00:40:46,020 I think when you now read the solutions to the exercises, 749 00:40:46,020 --> 00:40:49,060 you will have a better feeling for what's going on. 750 00:40:49,060 --> 00:40:50,730 And as I say, in the exercises, I 751 00:40:50,730 --> 00:40:53,130 will develop these topics much more slowly, 752 00:40:53,130 --> 00:40:55,530 and you can read these solutions as we go along. 753 00:40:55,530 --> 00:40:59,430 At any rate, this presents our overview of dot products. 754 00:40:59,430 --> 00:41:01,695 And so until next time, then, goodbye. 755 00:41:04,640 --> 00:41:07,040 Funding for the publication of this video 756 00:41:07,040 --> 00:41:11,920 was provided by the Gabriella and Paul Rosenbaum Foundation. 757 00:41:11,920 --> 00:41:16,060 Help OCW continue to provide free and open access to MIT 758 00:41:16,060 --> 00:41:21,495 courses by making a donation at ocw.mit.edu/donate.