WEBVTT 00:00:00.130 --> 00:00:02.490 The following content is provided under a Creative 00:00:02.490 --> 00:00:03.650 Commons license. 00:00:03.650 --> 00:00:06.920 Your support will help MIT OpenCourseWare continue offer 00:00:06.920 --> 00:00:10.030 high quality educational resources for free. 00:00:10.030 --> 00:00:12.780 To make a donation or to view additional materials from 00:00:12.780 --> 00:00:16.560 hundreds of MIT courses, visit MIT OpenCourseWare at 00:00:16.560 --> 00:00:19.270 ocw.mit.edu. 00:00:19.270 --> 00:00:26.460 PROFESSOR: I just started talking a little bit last time 00:00:26.460 --> 00:00:33.230 about viewing L2, namely this set of functions that are 00:00:33.230 --> 00:00:37.190 square integrable as a vector space. 00:00:37.190 --> 00:00:39.230 And I want to reveal a little bit about why 00:00:39.230 --> 00:00:41.370 we want to do that. 00:00:41.370 --> 00:00:45.320 Because after spending so long worrying about all these 00:00:45.320 --> 00:00:49.620 questions of measurability and all that, you must wonder, why 00:00:49.620 --> 00:00:53.460 do we want to look at it a different way now. 00:00:53.460 --> 00:00:59.590 Well, a part of it is we want to be able to look at 00:00:59.590 --> 00:01:02.820 orthogonal expansions geometrically. 00:01:02.820 --> 00:01:04.300 In other words, we would like to be able to 00:01:04.300 --> 00:01:06.390 draw pictures of them. 00:01:06.390 --> 00:01:09.190 You can draw pictures of a function, but you're drawing a 00:01:09.190 --> 00:01:10.860 picture of one function. 00:01:10.860 --> 00:01:14.420 And you can't draw anything about the relationship between 00:01:14.420 --> 00:01:17.180 different functions, except by drawing the 00:01:17.180 --> 00:01:18.360 two different functions. 00:01:18.360 --> 00:01:20.990 You get all the detail there, and you can't 00:01:20.990 --> 00:01:22.590 abstract it at all. 00:01:22.590 --> 00:01:25.980 So somehow we want to start to abstract some of this 00:01:25.980 --> 00:01:27.740 information. 00:01:27.740 --> 00:01:30.650 And we want to be able to draw pictures which look more like 00:01:30.650 --> 00:01:33.490 vector pictures than like functions. 00:01:33.490 --> 00:01:37.250 And you'll see why that becomes important in a while. 00:01:37.250 --> 00:01:40.090 The other thing is, when you draw a function as a function 00:01:40.090 --> 00:01:44.520 of time, the only thing you see it how it behaves in time. 00:01:44.520 --> 00:01:47.490 When you take the Fourier transform and you draw it in 00:01:47.490 --> 00:01:49.960 frequency, the only thing you see is how 00:01:49.960 --> 00:01:51.780 it behaves in frequency. 00:01:51.780 --> 00:01:55.110 And, again, you don't see what the relationship is between 00:01:55.110 --> 00:01:56.310 different functions. 00:01:56.310 --> 00:01:58.120 So you lose all of that. 00:01:58.120 --> 00:02:01.780 When you take this signal space viewpoint, what you're 00:02:01.780 --> 00:02:08.890 trying to do there is to not stress time or frequency so 00:02:08.890 --> 00:02:11.270 much, but more to look at the relationship 00:02:11.270 --> 00:02:13.030 between different functions. 00:02:13.030 --> 00:02:14.780 Why do we want to do that? 00:02:14.780 --> 00:02:17.390 Well, because as soon as we start looking at noise and 00:02:17.390 --> 00:02:20.620 things like that, we want to be able to tell something 00:02:20.620 --> 00:02:24.340 about how distinguishable different functions are from 00:02:24.340 --> 00:02:25.180 each other. 00:02:25.180 --> 00:02:29.320 So the critical question there you want to ask, when you ask 00:02:29.320 --> 00:02:32.380 how different functions are, is how do you look at two 00:02:32.380 --> 00:02:34.870 functions both at the same time. 00:02:34.870 --> 00:02:39.000 So, again, that's -- 00:02:39.000 --> 00:02:41.450 all of this is coming back to the same thing. 00:02:41.450 --> 00:02:45.050 So we want to be able to draw pictures of functions which 00:02:45.050 --> 00:02:48.270 show how functions are related, rather than show all 00:02:48.270 --> 00:02:50.910 the individual detail of function. 00:02:50.910 --> 00:02:55.640 Finally, what we'll see at the end of today that it gives us 00:02:55.640 --> 00:02:59.230 a lot of capability for understanding much, much 00:02:59.230 --> 00:03:03.040 better what's going on when we look at conversions of 00:03:03.040 --> 00:03:06.170 orthogonal series. 00:03:06.170 --> 00:03:09.040 This is something we haven't had any way to look at before 00:03:09.040 --> 00:03:14.500 we talked about the Fourier series, the discrete time 00:03:14.500 --> 00:03:17.510 Fourier transform, and things like this. 00:03:17.510 --> 00:03:22.280 But we haven't been able to really say something general. 00:03:22.280 --> 00:03:24.330 Which, again, is what we want to do now. 00:03:28.020 --> 00:03:31.630 I'm going to go very quickly through these axioms of a 00:03:31.630 --> 00:03:34.410 vector space. 00:03:34.410 --> 00:03:37.490 Most of you have seen them before. 00:03:37.490 --> 00:03:41.430 Those of you who haven't seen axiomatic treatments of 00:03:41.430 --> 00:03:44.540 various things in mathematics are going to be 00:03:44.540 --> 00:03:46.020 puzzled by it anyway. 00:03:46.020 --> 00:03:49.510 And you're going to have to sit at home or somewhere on 00:03:49.510 --> 00:03:52.690 your own and puzzle this out. 00:03:52.690 --> 00:03:55.720 But I just wanted to put them up so I could start to say 00:03:55.720 --> 00:04:00.440 what it is that we're trying to do with these axioms. 00:04:00.440 --> 00:04:05.700 What we're trying to do is to say everything we know about 00:04:05.700 --> 00:04:11.430 n-tuples, which we've been using all of our lives. 00:04:11.430 --> 00:04:15.580 All of these tricks that we use, most of them we can use 00:04:15.580 --> 00:04:18.110 to deal with functions also. 00:04:18.110 --> 00:04:20.550 The pictures, we can use. 00:04:20.550 --> 00:04:23.900 All of the ideas about dimension, all of the ideas 00:04:23.900 --> 00:04:25.880 about expansions. 00:04:25.880 --> 00:04:29.660 All of this stuff becomes useful again. 00:04:29.660 --> 00:04:33.910 But, there are a lot of things that are true about functions 00:04:33.910 --> 00:04:36.740 that aren't true about vectors. 00:04:36.740 --> 00:04:42.150 There are lots of things which are true about n-dimensional 00:04:42.150 --> 00:04:44.750 vectors that aren't true about functions. 00:04:44.750 --> 00:04:48.750 And you want to be able to go back to these axioms when you 00:04:48.750 --> 00:04:53.530 have to, and say, well, is this property we're looking at 00:04:53.530 --> 00:04:56.160 something which is a consequence of the axioms. 00:04:56.160 --> 00:05:00.850 Namely, is this an inherent property of vector spaces, or 00:05:00.850 --> 00:05:05.830 is it in fact something else which is just because of the 00:05:05.830 --> 00:05:08.770 particular kind of thing we're looking at. 00:05:08.770 --> 00:05:10.180 So, vector spaces. 00:05:10.180 --> 00:05:13.690 Important thing is, there's an addition operation. 00:05:13.690 --> 00:05:16.100 You can add any two vectors. 00:05:16.100 --> 00:05:18.000 You can't multiply them, by the way. 00:05:18.000 --> 00:05:20.230 You can only add them. 00:05:20.230 --> 00:05:22.850 You can multiply by scalars, which we'll talk about in the 00:05:22.850 --> 00:05:24.210 next slide. 00:05:24.210 --> 00:05:28.360 But you can only add the vectors themselves. 00:05:28.360 --> 00:05:31.090 And the addition is commutative, just like 00:05:31.090 --> 00:05:34.730 ordinary addition of real numbers or complex numbers is. 00:05:34.730 --> 00:05:41.630 It's associative, which says v, u plus w in parentheses. 00:05:41.630 --> 00:05:44.670 Now, you see why we're doing this is, we've said by 00:05:44.670 --> 00:05:50.990 definition that for any two vectors, u and w, there's 00:05:50.990 --> 00:05:54.300 another vector, which is called u plus w. 00:05:54.300 --> 00:05:57.700 In other words, this axiomatic system says, whenever you add 00:05:57.700 --> 00:06:00.110 two vectors you have to get a vector. 00:06:00.110 --> 00:06:01.750 There's no way around that. 00:06:01.750 --> 00:06:04.720 So u plus w has to be a vector. 00:06:04.720 --> 00:06:09.430 And that says that v plus u plus w has to be a vector. 00:06:09.430 --> 00:06:13.520 Associativity says that you get the same vector if you 00:06:13.520 --> 00:06:16.690 look at it the other way, if first adding v and u 00:06:16.690 --> 00:06:18.190 and then adding w. 00:06:18.190 --> 00:06:21.940 So, anything what you call a vector space has 00:06:21.940 --> 00:06:22.935 to have this property. 00:06:22.935 --> 00:06:24.965 Of course, the real numbers have this property. 00:06:24.965 --> 00:06:27.340 The complex numbers have this property. 00:06:27.340 --> 00:06:29.240 All the n-tuples you deal with all the 00:06:29.240 --> 00:06:30.930 time have this property. 00:06:30.930 --> 00:06:32.780 Functions have this property. 00:06:32.780 --> 00:06:36.780 Sequences, infinite length sequences have this property, 00:06:36.780 --> 00:06:38.370 so there's no big deal about it. 00:06:38.370 --> 00:06:40.500 But that is one of the axioms. 00:06:40.500 --> 00:06:44.330 There's a unique vector 0, such that when you add 0 to a 00:06:44.330 --> 00:06:48.300 vector, you get the same vector again. 00:06:48.300 --> 00:06:52.380 Now, this is not the 0 in the real number system. 00:06:52.380 --> 00:06:56.100 It's not the 0 in a complex number system. 00:06:56.100 --> 00:07:00.040 In terms of vectors, if you're thinking of n-tuples, this is 00:07:00.040 --> 00:07:02.590 the n-tuple which is all 0's. 00:07:02.590 --> 00:07:05.840 In terms of other vectors, spaces, it 00:07:05.840 --> 00:07:06.820 might be other things. 00:07:06.820 --> 00:07:11.480 Like, in terms of functions, 0 means the function which is 0 00:07:11.480 --> 00:07:13.850 everywhere. 00:07:13.850 --> 00:07:17.400 And finally, there's the unique vector minus v, which 00:07:17.400 --> 00:07:20.620 has the property that v plus minus v -- in other words, v 00:07:20.620 --> 00:07:23.250 minus v -- is equal to 0. 00:07:23.250 --> 00:07:25.440 And in a sense that defines 00:07:25.440 --> 00:07:27.430 subtraction as well as addition. 00:07:27.430 --> 00:07:30.710 So we have addition and subtraction, but we don't have 00:07:30.710 --> 00:07:31.960 multiplication. 00:07:38.480 --> 00:07:40.130 And there's scalar multiplication. 00:07:40.130 --> 00:07:44.070 Which means you can multiply a vector by a scalar. 00:07:44.070 --> 00:07:50.330 And the vector spaces we're looking at here only have two 00:07:50.330 --> 00:07:51.400 kinds of scalars. 00:07:51.400 --> 00:07:54.730 One is real scalars and the other is complex scalars. 00:07:54.730 --> 00:07:57.790 The two are quite different, as you know. 00:07:57.790 --> 00:08:01.580 And so when we're talking about a vector space we have 00:08:01.580 --> 00:08:06.340 to say what the scalar field is that we're talking about. 00:08:06.340 --> 00:08:10.100 So, for every vector. 00:08:10.100 --> 00:08:13.800 And also for every scalar, there's another vector which 00:08:13.800 --> 00:08:15.620 is the scalar times the vector. 00:08:15.620 --> 00:08:18.300 Which means, you have scalar multiplication. 00:08:18.300 --> 00:08:24.510 You can multiply vectors by scalars in terms of n-tuples. 00:08:24.510 --> 00:08:27.750 When you're multiplying a scalar by an n-tuple, you just 00:08:27.750 --> 00:08:30.100 multiply every component by that scalar. 00:08:32.840 --> 00:08:37.430 When you take the scalar 1, and in a field there's always 00:08:37.430 --> 00:08:40.070 an element 1, there's always an element 0. 00:08:40.070 --> 00:08:43.180 When you take the element 1, and you multiply it by a 00:08:43.180 --> 00:08:45.770 vector, you got the same vector. 00:08:45.770 --> 00:08:48.230 Which, of course, is what you would expect if you were 00:08:48.230 --> 00:08:49.920 looking at functions. 00:08:49.920 --> 00:08:52.290 You multiply a function. 00:08:52.290 --> 00:08:56.100 For every t you multiply it by 1, and you get the same thing 00:08:56.100 --> 00:08:57.150 back again. 00:08:57.150 --> 00:09:00.910 For an n-tuple, you multiply every component by 1, you get 00:09:00.910 --> 00:09:02.480 the same thing back again. 00:09:02.480 --> 00:09:04.860 So that's -- 00:09:04.860 --> 00:09:06.730 well, it's just another axiom. 00:09:06.730 --> 00:09:09.500 Then we have these distributive laws. 00:09:09.500 --> 00:09:11.130 And I won't read them out to you, they're 00:09:11.130 --> 00:09:12.330 just what they say. 00:09:12.330 --> 00:09:14.710 I want to talk about them in a second a little bit. 00:09:18.500 --> 00:09:22.630 And as we said, the simplistic example of a vector space, and 00:09:22.630 --> 00:09:27.160 the one you've been using for years, partly because it saves 00:09:27.160 --> 00:09:28.840 you a lot of notation. 00:09:28.840 --> 00:09:32.240 Incidentally, one of the reasons I didn't list for 00:09:32.240 --> 00:09:37.800 talking about L2 as a vector space is, it saves you a lot 00:09:37.800 --> 00:09:38.320 of writing. 00:09:38.320 --> 00:09:41.720 Also, just like when you're talking about n-tuples it 00:09:41.720 --> 00:09:42.670 saves you writing. 00:09:42.670 --> 00:09:46.640 It's a nice, convenient notational device. 00:09:46.640 --> 00:09:50.700 I don't think any part of mathematics becomes popular 00:09:50.700 --> 00:09:55.920 with everyone, unless it saves notation as well as well as 00:09:55.920 --> 00:09:59.200 simplifying things. 00:09:59.200 --> 00:10:02.870 So we have these n-tuples, which is what we mean by r sub 00:10:02.870 --> 00:10:04.210 n or c sub n. 00:10:04.210 --> 00:10:07.770 In other words, when I talk about r sub n, I don't mean 00:10:07.770 --> 00:10:10.320 just a vector space of dimension n. 00:10:10.320 --> 00:10:13.430 I haven't even defined dimension yet. 00:10:13.430 --> 00:10:18.960 And when you talk in this generality, you have to think 00:10:18.960 --> 00:10:21.000 a little bit about what it means. 00:10:21.000 --> 00:10:25.670 So we're just talking about n-tuples now. r sub n is 00:10:25.670 --> 00:10:29.090 n-tuples of real numbers. c sub n is 00:10:29.090 --> 00:10:32.080 n-tuples of complex numbers. 00:10:32.080 --> 00:10:34.980 Addition and scalar multiplication are component 00:10:34.980 --> 00:10:36.915 by component, which we just said. 00:10:36.915 --> 00:10:42.450 In other words, w equals u plus v means this, for all i. 00:10:42.450 --> 00:10:45.350 Scalar multiplication means this. 00:10:45.350 --> 00:10:51.780 The unit vector, e sub i, is a 1 in the i'th position, a 0 00:10:51.780 --> 00:10:53.880 everywhere else. 00:10:53.880 --> 00:10:57.820 And what that means is that any vector v which is an 00:10:57.820 --> 00:11:02.800 n-tuple, n rn or n cn, can also be expressed as the 00:11:02.800 --> 00:11:06.040 equals summation of these coefficients, 00:11:06.040 --> 00:11:08.430 times these unit factors. 00:11:08.430 --> 00:11:12.230 That looks like we're giving up the simple notation we have 00:11:12.230 --> 00:11:14.650 and going to more complex notation. 00:11:14.650 --> 00:11:18.390 This is a very useful idea here. 00:11:18.390 --> 00:11:21.050 That you can take even something as simple as 00:11:21.050 --> 00:11:25.820 n-tuples and express every vector as a sum of these 00:11:25.820 --> 00:11:27.240 simple vectors. 00:11:27.240 --> 00:11:30.020 And when you start drawing pictures, you start to see why 00:11:30.020 --> 00:11:31.090 this makes sense. 00:11:31.090 --> 00:11:33.930 I'm going to do this on the next slide. 00:11:33.930 --> 00:11:35.950 So let's do it. 00:11:44.810 --> 00:11:47.990 And on the slide here, what I've done is to draw a diagram 00:11:47.990 --> 00:11:50.940 which is one you've seen many times, I'm sure. 00:11:50.940 --> 00:11:53.190 Except for the particular things on it. 00:11:53.190 --> 00:11:56.700 Of two-dimensional space, where you take a 2-tuple and 00:11:56.700 --> 00:12:00.930 you think of the first element in the 2-tuple, v1, a being 00:12:00.930 --> 00:12:02.500 the horizontal axis. 00:12:02.500 --> 00:12:07.040 The second element, v2, as being the vertical axis. 00:12:07.040 --> 00:12:12.250 And then you can draw any 2-tuple by going over v1 and 00:12:12.250 --> 00:12:16.680 then up v2, which is what you've done all your lives. 00:12:16.680 --> 00:12:19.800 The reason I'm drawing this is to show you that you can take 00:12:19.800 --> 00:12:21.380 any two vectors. 00:12:21.380 --> 00:12:26.670 First vector, v. Second vector, u. 00:12:26.670 --> 00:12:29.070 One thing that -- 00:12:29.070 --> 00:12:32.380 I'm sure you all traditionally do is you view a 00:12:32.380 --> 00:12:34.470 vector in two ways. 00:12:34.470 --> 00:12:38.610 One, you view it as a point in two-dimensional space. 00:12:38.610 --> 00:12:45.890 And the other is you view it as a line from 0 up to v. And 00:12:45.890 --> 00:12:49.030 when you put a little arrow on it, it looks like a vector, 00:12:49.030 --> 00:12:50.250 and we call it a vector. 00:12:50.250 --> 00:12:52.880 So it must be a vector, OK? 00:12:52.880 --> 00:12:56.470 So either the point or the line represents a vector. 00:12:56.470 --> 00:12:58.780 Here's another vector, u. 00:12:58.780 --> 00:13:01.650 I can take the difference between two vectors, u minus 00:13:01.650 --> 00:13:07.560 v, just by drawing a line from v up to u. 00:13:07.560 --> 00:13:10.530 And thinking of that as a vector also. 00:13:10.530 --> 00:13:16.790 So this vector really means a vector starting here, going 00:13:16.790 --> 00:13:20.610 parallel to this, up to this point, which is what w is. 00:13:20.610 --> 00:13:23.610 But it's very convenient to draw it this way, which lets 00:13:23.610 --> 00:13:24.750 you know what's going on. 00:13:24.750 --> 00:13:30.950 Namely, you represent u as the sum of v plus w. 00:13:30.950 --> 00:13:35.420 Or you represent w as a difference between u and v. 00:13:35.420 --> 00:13:37.030 And all of this is trivial. 00:13:37.030 --> 00:13:41.090 I just want to say it explicitly so you start to see 00:13:41.090 --> 00:13:46.200 what the connection is between these axioms, which if I don't 00:13:46.200 --> 00:13:48.850 talk about them a little bit, I'm sure you're just going to 00:13:48.850 --> 00:13:53.340 blow them off and decide, eh, I know all of that, I can look 00:13:53.340 --> 00:13:55.750 at things in two dimensions and I don't have to think 00:13:55.750 --> 00:13:57.010 about them at all. 00:13:57.010 --> 00:14:03.340 And then, in a few days, when we're doing the same thing for 00:14:03.340 --> 00:14:06.460 functions, you're suddenly going to become very confused. 00:14:06.460 --> 00:14:09.230 So you ought to think about it in these simple terms. 00:14:09.230 --> 00:14:14.730 Now, when I take a scalar multiple of the vector, in 00:14:14.730 --> 00:14:18.740 terms of these diagrams, what we're doing is taking 00:14:18.740 --> 00:14:21.060 something on this same line here. 00:14:21.060 --> 00:14:24.410 I can take scalar multiples which go all the way up here, 00:14:24.410 --> 00:14:25.900 all the way down there. 00:14:25.900 --> 00:14:29.130 I can take scalar multiples of u which go up here. 00:14:29.130 --> 00:14:31.610 And down here. 00:14:31.610 --> 00:14:37.560 The interesting thing here is when I scale v down by alpha 00:14:37.560 --> 00:14:42.970 and I scale u down by alpha, I can also scale w down by alpha 00:14:42.970 --> 00:14:47.000 and I connect it just like that, from alpha v to alpha u. 00:14:47.000 --> 00:14:48.250 Which axiom is that? 00:14:54.360 --> 00:14:57.960 I mean, this is the canonic example of 00:14:57.960 --> 00:15:01.360 one of those axioms. 00:15:01.360 --> 00:15:01.990 AUDIENCE: [UNINTELLIGIBLE] 00:15:01.990 --> 00:15:03.370 PROFESSOR: What? 00:15:03.370 --> 00:15:04.010 AUDIENCE: Distributed. 00:15:04.010 --> 00:15:05.110 PROFESSOR: Distributed, good. 00:15:05.110 --> 00:15:10.030 This is saying that alpha times the quantity u minus v 00:15:10.030 --> 00:15:15.770 is equal to alpha u minus alpha v. That's so trivial 00:15:15.770 --> 00:15:18.220 that it's hard to see it, but that's what it's saying. 00:15:18.220 --> 00:15:22.540 So that the distributive law really says the triangles 00:15:22.540 --> 00:15:23.790 maintain their shape. 00:15:26.200 --> 00:15:27.860 Maybe it's easier to just say distributive 00:15:27.860 --> 00:15:29.110 law, I don't know. 00:15:35.960 --> 00:15:40.960 So, for the space of L2 complex functions, we're going 00:15:40.960 --> 00:15:46.790 to define u plus v as w, where w of t equals u 00:15:46.790 --> 00:15:48.100 of t plus v of t. 00:15:48.100 --> 00:15:51.470 Now, I've done a bunch of things here all at once. 00:15:51.470 --> 00:15:56.350 One of them is to say what we used to call a function, u of 00:15:56.350 --> 00:16:02.250 t, we're now referring to with a single letter, boldface u. 00:16:02.250 --> 00:16:04.320 What's the advantage of that? 00:16:04.320 --> 00:16:08.460 Well, one advantage of it is when you talk about a function 00:16:08.460 --> 00:16:11.090 as u of t, you're really talking about 00:16:11.090 --> 00:16:12.630 two different things. 00:16:12.630 --> 00:16:16.530 One, you're talking about the value of the function at a 00:16:16.530 --> 00:16:19.800 particular argument, t. 00:16:19.800 --> 00:16:20.950 And the other is, you're talking 00:16:20.950 --> 00:16:22.860 about the whole function. 00:16:22.860 --> 00:16:24.530 I mean, sometimes you want to say a 00:16:24.530 --> 00:16:26.130 function has some property. 00:16:26.130 --> 00:16:27.770 A function is L2. 00:16:27.770 --> 00:16:30.970 Well, u of t at a particular t is just a number. 00:16:30.970 --> 00:16:33.220 It's not a function. 00:16:33.220 --> 00:16:36.780 So this gives us a nice way of distinguishing between 00:16:36.780 --> 00:16:40.060 functions and between the value of functions for 00:16:40.060 --> 00:16:42.310 particular arguments. 00:16:42.310 --> 00:16:46.060 So that's one more notational advantage you get by talking 00:16:46.060 --> 00:16:49.310 about vectors here. 00:16:49.310 --> 00:16:54.250 We're going to define the sum of two functions just as the 00:16:54.250 --> 00:16:55.340 point y sum. 00:16:55.340 --> 00:16:57.530 In other words, these two functions are 00:16:57.530 --> 00:16:59.450 defined at each t. 00:16:59.450 --> 00:17:04.390 w defined at a t is the sum of this and that. 00:17:04.390 --> 00:17:09.680 The scalar multiplication is just defined by, at every t, u 00:17:09.680 --> 00:17:12.990 of t is equal to alpha times v of t. 00:17:12.990 --> 00:17:14.810 Just what you'd expect. 00:17:14.810 --> 00:17:16.510 There's nothing strange here. 00:17:16.510 --> 00:17:20.600 I just want to be explicit in saying how everything follows 00:17:20.600 --> 00:17:21.890 from these axioms here. 00:17:21.890 --> 00:17:24.260 And I won't say all of it because 00:17:24.260 --> 00:17:27.720 there's too much of it. 00:17:27.720 --> 00:17:32.700 With this addition and scalar multiplication, L2, the set of 00:17:32.700 --> 00:17:36.390 finite energy measurable functions is in fact the 00:17:36.390 --> 00:17:38.390 complex vector space. 00:17:38.390 --> 00:17:40.950 And it's trivial to go back and check through all the 00:17:40.950 --> 00:17:43.520 axioms with what we said here, and I'm not 00:17:43.520 --> 00:17:45.610 going to do it now. 00:17:45.610 --> 00:17:48.510 There's only one of those axioms which is a bit 00:17:48.510 --> 00:17:49.670 questionable. 00:17:49.670 --> 00:17:54.490 And that is, when you add up two functions which are square 00:17:54.490 --> 00:17:57.880 integrable, is the sum going to be square integrable also. 00:18:00.600 --> 00:18:04.060 Well, you nod and say yes, but it's worthwhile proving it 00:18:04.060 --> 00:18:06.240 once in your lives. 00:18:06.240 --> 00:18:09.790 So the question is, is this function here less than 00:18:09.790 --> 00:18:16.040 infinity if the integral of u of t squared and the integral 00:18:16.040 --> 00:18:18.700 of v of t squared are both integrable. 00:18:18.700 --> 00:18:22.340 Well, it's a useful inequality, which looks like a 00:18:22.340 --> 00:18:26.260 very weak inequality but it, in fact, is not weak. 00:18:26.260 --> 00:18:28.660 It says that u of t plus v of t. 00:18:28.660 --> 00:18:30.975 This is just at a particular value of t. 00:18:30.975 --> 00:18:34.320 So this is just an inequality for real 00:18:34.320 --> 00:18:36.360 numbers and complex numbers. 00:18:36.360 --> 00:18:40.860 It says that this u of t plus v of t, quantity squared, 00:18:40.860 --> 00:18:44.370 magnitude squared, is less than or equal to 2 times u of 00:18:44.370 --> 00:18:47.780 t squared plus 2 times v of t squared. 00:18:47.780 --> 00:18:50.990 You wonder, at first, what's the 2 doing in there. 00:18:50.990 --> 00:18:55.310 But then think of making v of t equal to u of t. 00:18:55.310 --> 00:18:59.740 You have 2 times u of t squared, which is 4 00:18:59.740 --> 00:19:01.400 times u of t squared. 00:19:01.400 --> 00:19:03.460 Well, that's what you need here to make this true. 00:19:03.460 --> 00:19:06.260 So, in that example this inequality is 00:19:06.260 --> 00:19:09.350 satisfied with equality. 00:19:09.350 --> 00:19:12.110 To verify the inequality in general, you just 00:19:12.110 --> 00:19:13.240 multiply this out. 00:19:13.240 --> 00:19:17.790 It's u of t squared plus v of t squared plus two cross-terms 00:19:17.790 --> 00:19:19.970 and it all works. 00:19:23.250 --> 00:19:27.300 This vector space here is not quite the vector space we want 00:19:27.300 --> 00:19:28.900 to talk about. 00:19:28.900 --> 00:19:31.800 But let's put off that question for a while and we'll 00:19:31.800 --> 00:19:34.000 come to it later. 00:19:34.000 --> 00:19:37.290 We will come up with a vector space which is just slightly 00:19:37.290 --> 00:19:38.540 different than that. 00:19:41.650 --> 00:19:46.620 The main thing you can do with vector spaces is talk about 00:19:46.620 --> 00:19:48.760 their dimension. 00:19:48.760 --> 00:19:50.660 Well, there are a lot of other things you can do, but this is 00:19:50.660 --> 00:19:53.460 one of the main things we can do. 00:19:53.460 --> 00:19:57.080 And it's an important thing which we have to talk about 00:19:57.080 --> 00:20:01.000 before going into inner product spaces, which is what 00:20:01.000 --> 00:20:04.400 we're really interested in. 00:20:04.400 --> 00:20:06.620 So we need a bunch of definitions here. 00:20:06.620 --> 00:20:10.120 All of this, I'm sure is familiar to most of you. 00:20:10.120 --> 00:20:13.370 For those of you that it's not familiar to, you just have to 00:20:13.370 --> 00:20:15.480 spend a little longer with it. 00:20:15.480 --> 00:20:19.650 There's not a whole lot involved here. 00:20:19.650 --> 00:20:25.000 If you have a set of vectors, which are in a vector space, 00:20:25.000 --> 00:20:28.940 you say that they span the vector space if in fact every 00:20:28.940 --> 00:20:32.590 vector in this vector space is a linear 00:20:32.590 --> 00:20:36.870 combination of those vectors. 00:20:36.870 --> 00:20:42.890 In other words, any vector, u, can be made up as some sum of 00:20:42.890 --> 00:20:45.420 alpha i times v sub i. 00:20:45.420 --> 00:20:50.980 Now, notice we've gone a long way here beyond those axioms. 00:20:50.980 --> 00:20:55.160 Because we're talking about scalar multiplications. 00:20:55.160 --> 00:20:58.340 And then we're talking about a sum of a lot of scalar 00:20:58.340 --> 00:20:59.860 multiplications. 00:20:59.860 --> 00:21:02.000 Each scalar multiplication is a vector. 00:21:02.000 --> 00:21:05.330 The sum of a bunch of vectors, by the associative law is, in 00:21:05.330 --> 00:21:06.670 fact, another vector. 00:21:06.670 --> 00:21:11.710 So every one of these sums is a vector. 00:21:11.710 --> 00:21:17.110 And by definition, a set of vectors spans a vector space 00:21:17.110 --> 00:21:21.180 if every vector can be represented as some linear 00:21:21.180 --> 00:21:22.356 combination of them. 00:21:22.356 --> 00:21:22.530 In 00:21:22.530 --> 00:21:26.720 other words, there isn't something outside of here 00:21:26.720 --> 00:21:29.000 sitting there waiting to be discovered later. 00:21:29.000 --> 00:21:32.570 You really understand everything that's there. 00:21:32.570 --> 00:21:36.230 And we say that v is finite dimensional if it is spanned 00:21:36.230 --> 00:21:38.730 by a finite set of vectors. 00:21:38.730 --> 00:21:40.060 So that's another definition. 00:21:40.060 --> 00:21:42.390 That's what you mean by finite dimensional. 00:21:42.390 --> 00:21:46.030 You have to be able to find a finite set of vectors such 00:21:46.030 --> 00:21:49.020 that linear combinations of those vectors gives you 00:21:49.020 --> 00:21:50.720 everything. 00:21:50.720 --> 00:21:52.780 It doesn't mean you have a finite set of vectors. 00:21:52.780 --> 00:21:55.530 You have an infinite set of vectors because you have an 00:21:55.530 --> 00:21:57.480 infinite set of scalars. 00:21:57.480 --> 00:21:59.320 In fact, you'd have an uncountably infinite set of 00:21:59.320 --> 00:22:01.820 vectors because of these scalars. 00:22:04.330 --> 00:22:05.300 Second definition. 00:22:05.300 --> 00:22:09.940 The vector v1 to vn are linearly independent -- and 00:22:09.940 --> 00:22:15.150 this is a mouthful -- if u equals the sum of alpha sub i 00:22:15.150 --> 00:22:20.280 v sub i equals 0, only for alpha sub i equal 00:22:20.280 --> 00:22:22.780 to 0 for each i. 00:22:22.780 --> 00:22:26.900 In other words, you can't take any linear combination of the 00:22:26.900 --> 00:22:32.130 v sub i's and get 0 unless that linear combination is 00:22:32.130 --> 00:22:34.230 using all 0's. 00:22:34.230 --> 00:22:37.330 You can't, in any non-trivial way, add up a bunch of 00:22:37.330 --> 00:22:39.870 vectors and get 0. 00:22:39.870 --> 00:22:43.660 To put it another way, none of these basis vectors is a 00:22:43.660 --> 00:22:45.290 linear combination of the others. 00:22:45.290 --> 00:22:50.130 That's usually a more convenient way to put it. 00:22:50.130 --> 00:22:52.180 Although it takes more writing. 00:22:52.180 --> 00:22:55.820 Now, we say that a set of vectors are a basis for v if 00:22:55.820 --> 00:22:59.530 they're both linearly independent and if they span 00:22:59.530 --> 00:23:02.850 v. When we first talked about spanning, we didn't say 00:23:02.850 --> 00:23:06.740 anything about these vectors being linearly independent, so 00:23:06.740 --> 00:23:10.110 we might have had many more of them than we needed. 00:23:10.110 --> 00:23:14.360 Now, when we're talking about a basis, we restrict ourselves 00:23:14.360 --> 00:23:18.330 to just the set we need to span the space. 00:23:18.330 --> 00:23:20.920 And then the theorem, which I'm not going to prove, but 00:23:20.920 --> 00:23:28.590 it's standard Theorem One of any linear algebra book -- 00:23:28.590 --> 00:23:31.110 well, it's probably Theorem One, Two, Three and Four all 00:23:31.110 --> 00:23:36.990 put together -- but anyway, if it says if v1 and v sub n span 00:23:36.990 --> 00:23:42.100 v, then a subset of these vectors is the basis of b. 00:23:42.100 --> 00:23:45.180 In other words, if you have a set of vectors which span a 00:23:45.180 --> 00:23:50.460 space, you can find the basis by systematically eliminating 00:23:50.460 --> 00:23:53.470 vectors which are linear combinations of the others, 00:23:53.470 --> 00:23:57.040 until you get to a point where you can't do that any further. 00:23:57.040 --> 00:24:00.110 So this theorem has an algorithm tied into it. 00:24:00.110 --> 00:24:03.280 Given any set of vectors which span a space, you can find the 00:24:03.280 --> 00:24:06.840 basis from it by perhaps throwing out 00:24:06.840 --> 00:24:09.510 some of those vectors. 00:24:09.510 --> 00:24:14.920 The next part of it is, if v is a finite dimensional space, 00:24:14.920 --> 00:24:17.880 then every basis has the same size. 00:24:17.880 --> 00:24:20.030 This, in fact, is a thing which takes a little bit of 00:24:20.030 --> 00:24:21.940 work proving it. 00:24:21.940 --> 00:24:25.460 And, also, any linearly independent set, v1 to v sub 00:24:25.460 --> 00:24:28.420 n, is part of the basis. 00:24:28.420 --> 00:24:31.150 In other words, here's another algorithm you can use. 00:24:31.150 --> 00:24:35.230 You have this big, finite dimensional space. 00:24:35.230 --> 00:24:38.960 You have a few vectors which are linearly independent. 00:24:38.960 --> 00:24:43.230 You can build a basis starting with these, and you just 00:24:43.230 --> 00:24:43.870 experiment. 00:24:43.870 --> 00:24:48.250 You experiment to find new vectors, which are not linear 00:24:48.250 --> 00:24:50.200 combinations of that set. 00:24:50.200 --> 00:24:52.780 As soon as you find one, you add it to the basis. 00:24:52.780 --> 00:24:54.200 You keep on going. 00:24:54.200 --> 00:24:56.450 And the theorem says, by time you get to n of 00:24:56.450 --> 00:24:58.380 them, you're done. 00:24:58.380 --> 00:25:05.540 So that, in a sense, spanning sets are too big. 00:25:05.540 --> 00:25:08.110 Linearly independent sets are too small. 00:25:08.110 --> 00:25:11.720 And what you want to do is add the linearly independent sets, 00:25:11.720 --> 00:25:15.150 shrink the spanning sets, and come up with a bases. 00:25:15.150 --> 00:25:19.240 And all bases have the same number of vectors. 00:25:19.240 --> 00:25:22.420 There many different bases you come up with, but they all 00:25:22.420 --> 00:25:24.080 have the same number of vectors. 00:25:26.980 --> 00:25:30.060 Not going to talk at all about infinite dimensional spaces 00:25:30.060 --> 00:25:32.750 until the last slide today. 00:25:32.750 --> 00:25:36.330 Because the only way I know to understand infinite 00:25:36.330 --> 00:25:42.170 dimensional vector spaces is by thinking of them, in some 00:25:42.170 --> 00:25:46.350 sort of limiting way, as finite dimensional spaces. 00:25:46.350 --> 00:25:50.630 And I think that's the only way you can do it. 00:25:50.630 --> 00:25:57.260 A vector space in itself has no sense of distance 00:25:57.260 --> 00:25:59.320 or angles in it. 00:25:59.320 --> 00:26:02.880 When I drew that picture before for you -- let me put 00:26:02.880 --> 00:26:04.130 it up again -- 00:26:11.820 --> 00:26:19.020 it almost looked like there was some sense 00:26:19.020 --> 00:26:20.250 of distance in here. 00:26:20.250 --> 00:26:23.800 Because when we took scalar multiples, we scaled these 00:26:23.800 --> 00:26:26.400 things down. 00:26:26.400 --> 00:26:29.540 When I took these triangles, we scaled down the triangle, 00:26:29.540 --> 00:26:31.590 and the triangle was maintained. 00:26:31.590 --> 00:26:35.880 But, in fact, I could do this just as well if I took v and 00:26:35.880 --> 00:26:40.000 moved it down here almost down on this axis, and if I took u 00:26:40.000 --> 00:26:42.770 and moved that almost down on the axis. 00:26:42.770 --> 00:26:48.040 I have the same picture, the same kind of distributive law. 00:26:48.040 --> 00:26:49.040 And everything else. 00:26:49.040 --> 00:26:50.700 You can't -- 00:26:50.700 --> 00:26:55.030 I mean, one of the troubles with n-dimensional space, to 00:26:55.030 --> 00:26:58.670 study what a vector space is about, is it's very hard to 00:26:58.670 --> 00:27:04.650 think of n-tuples without thinking of distance. 00:27:04.650 --> 00:27:07.760 And without thinking of things being orthogonal to each 00:27:07.760 --> 00:27:09.600 other, and of all of these things. 00:27:09.600 --> 00:27:14.520 None of that is talked about at all in any of these axioms. 00:27:14.520 --> 00:27:17.940 And, in fact, you need some new axioms to be able to talk 00:27:17.940 --> 00:27:23.370 about ideas of distance, or angle, or any of these things. 00:27:23.370 --> 00:27:25.000 And that's what we want to add here. 00:27:32.800 --> 00:27:34.860 So we need some new axioms. 00:27:34.860 --> 00:27:38.870 And what we need is a new operation on the vector space, 00:27:38.870 --> 00:27:41.730 before the only -- the only operations we have are 00:27:41.730 --> 00:27:43.960 addition and scalar multiplication. 00:27:43.960 --> 00:27:46.880 So that vector spaces are really 00:27:46.880 --> 00:27:48.760 incredibly simple animals. 00:27:48.760 --> 00:27:51.050 There's very little you can do with them. 00:27:51.050 --> 00:27:55.120 And this added thing is called an inner product. 00:27:55.120 --> 00:27:58.110 An inner product is a scalar valued 00:27:58.110 --> 00:28:00.330 function of two vectors. 00:28:00.330 --> 00:28:05.580 And it's represented by these little brackets. 00:28:05.580 --> 00:28:11.600 And the axioms that these inner products have to satisfy 00:28:11.600 --> 00:28:15.330 is, if you're dealing with a complex vector space. 00:28:15.330 --> 00:28:18.500 In other words, where the scalars are complex numbers, 00:28:18.500 --> 00:28:24.530 then this inner product, when you switch it around, you have 00:28:24.530 --> 00:28:25.340 Hermitian symmetry. 00:28:25.340 --> 00:28:31.460 Which means that this inner product is equal to u v 00:28:31.460 --> 00:28:33.910 complex conjugate. 00:28:33.910 --> 00:28:35.500 We've already seen that sort of thing in 00:28:35.500 --> 00:28:36.840 taking Fourier series. 00:28:36.840 --> 00:28:42.420 And the fact that when you're dealing with complex numbers, 00:28:42.420 --> 00:28:46.340 symmetry doesn't usually hold, and you usually need some kind 00:28:46.340 --> 00:28:51.720 of Hermitian symmetry in most of the things you do. 00:28:51.720 --> 00:28:56.450 The next axiom is something called bilinearity. 00:28:56.450 --> 00:28:59.920 It says that if you take a vector which is alpha times a 00:28:59.920 --> 00:29:04.450 vector v, plus beta times a vector u, take the inner 00:29:04.450 --> 00:29:09.210 product of that with w, it splits up as alpha times v w 00:29:09.210 --> 00:29:11.600 plus beta times u w. 00:29:11.600 --> 00:29:13.830 How about if I do it the other way? 00:29:13.830 --> 00:29:19.300 See if you understand what I'm talking about at all here. 00:29:19.300 --> 00:29:30.850 Suppose we take w alpha u plus beta v. What's that equal to? 00:29:36.380 --> 00:29:51.180 Well, it's equal to alpha something w u plus beta w v. 00:29:51.180 --> 00:29:54.450 Except that's wrong, It's right for real vector spaces, 00:29:54.450 --> 00:29:56.780 it's wrong for complex vector spaces. 00:29:56.780 --> 00:30:00.030 What am I missing here? 00:30:00.030 --> 00:30:04.770 I need complex conjugates here and complex conjugates here. 00:30:04.770 --> 00:30:07.530 I wanted to talk about that, because when you're dealing 00:30:07.530 --> 00:30:10.760 with inner products, I don't know whether you're like me, 00:30:10.760 --> 00:30:13.990 but every time I start dealing with inner products and I get 00:30:13.990 --> 00:30:16.880 in a hurry writing things down, I forgot to put those 00:30:16.880 --> 00:30:20.110 damn complex conjugates in them. 00:30:20.110 --> 00:30:22.150 And, just be careful. 00:30:22.150 --> 00:30:23.430 Because you need them. 00:30:23.430 --> 00:30:27.110 At least go back after you're all done and put the complex 00:30:27.110 --> 00:30:27.940 conjugates in. 00:30:27.940 --> 00:30:31.160 If you're dealing with real vectors, of course you don't 00:30:31.160 --> 00:30:33.510 need to worry about any of that. 00:30:33.510 --> 00:30:36.720 And all the pictures you draw are always of real vectors. 00:30:40.340 --> 00:30:42.840 Think of trying to draw this picture. 00:30:42.840 --> 00:30:45.520 This is the simplest picture you can draw. 00:30:45.520 --> 00:30:46.890 Of two-dimensional vectors. 00:30:46.890 --> 00:30:52.450 This is really a picture of a vector space of dimension two, 00:30:52.450 --> 00:30:54.550 for real vectors. 00:30:54.550 --> 00:30:58.700 What happens if you try to draw a picture of complex 00:30:58.700 --> 00:31:00.460 two-dimensional vector space? 00:31:04.070 --> 00:31:06.680 Well, it becomes very difficult to do. 00:31:06.680 --> 00:31:17.270 Because you're really talking about the real part of, if 00:31:17.270 --> 00:31:22.050 you're dealing with a basis which consists of two complex 00:31:22.050 --> 00:31:28.550 vectors, then instead of v1, you need a real part of v1 and 00:31:28.550 --> 00:31:30.560 imaginary part of v1. 00:31:30.560 --> 00:31:33.800 Instead of v2, you need real part of v2 and 00:31:33.800 --> 00:31:35.650 imaginary part of v2. 00:31:35.650 --> 00:31:39.090 And you can always draw this in four dimensions. 00:31:39.090 --> 00:31:41.590 And I even have trouble drawing in three dimensions, 00:31:41.590 --> 00:31:44.990 because somehow my pen doesn't make marks in three 00:31:44.990 --> 00:31:46.060 dimensions. 00:31:46.060 --> 00:31:52.520 And in four dimensions, I'm a blinking 12. 00:31:52.520 --> 00:31:56.650 And have no idea of what to do. 00:31:56.650 --> 00:31:58.440 So you have to remember this. 00:32:01.040 --> 00:32:07.070 If you're dealing with rn or cn, almost always you define 00:32:07.070 --> 00:32:12.090 the inner product of v and u as the sum of the components 00:32:12.090 --> 00:32:14.900 with the second component complex conjugated. 00:32:14.900 --> 00:32:19.690 This is just a standard thing that we do all the time. 00:32:19.690 --> 00:32:23.690 When we do this, and we use unit vectors, the inner 00:32:23.690 --> 00:32:29.510 product of v with the i'th unit vector is just v sub i. 00:32:29.510 --> 00:32:31.110 That's what this formula says. 00:32:31.110 --> 00:32:37.020 Because e sub i is this vector u, in which there's a 1 only 00:32:37.020 --> 00:32:41.220 in the i'th position, and a 0 everywhere else. 00:32:41.220 --> 00:32:46.020 So v e i is always the v i, and e i v is always v i 00:32:46.020 --> 00:32:47.240 complex conjugate. 00:32:47.240 --> 00:32:52.160 Again, this Hermitian nonsense that comes up to 00:32:52.160 --> 00:32:54.720 plague us all the time. 00:32:54.720 --> 00:32:59.180 And from that, if you make v equal to e sub j or e sub i, 00:32:59.180 --> 00:33:04.060 you get the inner product of two of these basis vectors is 00:33:04.060 --> 00:33:07.050 equal to 0 for i unequal to j. 00:33:07.050 --> 00:33:11.240 In other words, the standard way of drawing pictures when 00:33:11.240 --> 00:33:16.060 you make it into an inner product space, those unit 00:33:16.060 --> 00:33:19.200 vectors become orthonormal. 00:33:19.200 --> 00:33:20.980 Because that's the way you like to draw things. 00:33:20.980 --> 00:33:23.230 You like to draw one here and one there. 00:33:23.230 --> 00:33:27.680 And that's what we mean by perpendicular, which the 00:33:27.680 --> 00:33:32.160 two-dimensional or three-dimensional word for 00:33:32.160 --> 00:33:33.410 orthogonal. 00:33:36.600 --> 00:33:40.010 So we have a couple of definitions. 00:33:40.010 --> 00:33:45.160 The inner product of v with itself is called inner product 00:33:45.160 --> 00:33:51.110 v squared, which is called the squared norm of the vector. 00:33:51.110 --> 00:33:56.080 The squared norm has to be non-negative, by axiom. 00:33:56.080 --> 00:34:00.650 It has to be greater than 0, unless this vector, v, is in 00:34:00.650 --> 00:34:04.330 fact a 0 vector. 00:34:04.330 --> 00:34:07.050 And the length is just the square 00:34:07.050 --> 00:34:09.530 root of the norm squared. 00:34:09.530 --> 00:34:12.020 In other words, the length and the norm are the same thing. 00:34:12.020 --> 00:34:15.850 The norm of a vector is the length of the vector. 00:34:15.850 --> 00:34:18.570 I've always called it the length, but a lot of people 00:34:18.570 --> 00:34:21.190 like to call it the norm. 00:34:21.190 --> 00:34:26.530 v and u are orthogonal if the inner product of v and u is 00:34:26.530 --> 00:34:27.900 equal to 0. 00:34:27.900 --> 00:34:29.850 How did I get that? 00:34:29.850 --> 00:34:31.730 I defined it that why. 00:34:31.730 --> 00:34:34.440 Everybody defines it that way. 00:34:34.440 --> 00:34:37.100 That's what you mean by orthogonality. 00:34:37.100 --> 00:34:41.100 Now we have to go back and see if it makes any sense in terms 00:34:41.100 --> 00:34:43.920 of these nice simple diagrams. 00:34:43.920 --> 00:34:47.690 But first I'm going to do something called the 00:34:47.690 --> 00:34:50.820 one-dimensional projection theorem. 00:34:50.820 --> 00:34:53.220 Which is something you all know but you probably have 00:34:53.220 --> 00:34:56.150 never thought about. 00:34:56.150 --> 00:35:01.670 And what it says is, if you have to vectors, v and u, you 00:35:01.670 --> 00:35:05.140 can always break v up into two parts. 00:35:05.140 --> 00:35:09.530 One of which is on the same line with u. 00:35:09.530 --> 00:35:12.060 In other words, is colinear with u. 00:35:12.060 --> 00:35:15.330 I'm drawing a picture here for real spaces, but when I say 00:35:15.330 --> 00:35:20.270 colinear, when I'm dealing with complex spaces, I mean 00:35:20.270 --> 00:35:24.140 it's u times some scalar, which could be complex. 00:35:24.140 --> 00:35:26.840 So it's somewhere on this line. 00:35:26.840 --> 00:35:32.270 And the other part is perpendicular to this line. 00:35:32.270 --> 00:35:35.620 And this theorem says in any old inner product space at 00:35:35.620 --> 00:35:39.810 all, no matter how many dimensions you have, infinite 00:35:39.810 --> 00:35:42.980 dimensional, finite dimensional, anything, if it's 00:35:42.980 --> 00:35:46.880 an inner product space on either the scalars r or the 00:35:46.880 --> 00:35:52.120 scalars c, you can always take any old two vectors at all. 00:35:52.120 --> 00:35:55.630 And you can break one vector up into a part that's colinear 00:35:55.630 --> 00:35:59.730 with the other, and another part which is orthogonal. 00:35:59.730 --> 00:36:02.150 And you can always draw a picture of it. 00:36:02.150 --> 00:36:06.210 If you don't mind just drawing the picture for real vectors 00:36:06.210 --> 00:36:08.420 instead of complex vectors. 00:36:08.420 --> 00:36:11.520 This is an important idea. 00:36:11.520 --> 00:36:15.050 Because what we're going to use it for in a while, is to 00:36:15.050 --> 00:36:18.450 be able to talk about functions which are these 00:36:18.450 --> 00:36:21.160 incredibly complicated objects. 00:36:21.160 --> 00:36:23.760 And we're going to talk about two different functions. 00:36:23.760 --> 00:36:27.160 And we want to be able to draw a picture in which those two 00:36:27.160 --> 00:36:30.990 functions are represented just as points in a 00:36:30.990 --> 00:36:33.350 two-dimensional picture. 00:36:33.350 --> 00:36:36.060 And we're going to do that by saying, OK, I take one of 00:36:36.060 --> 00:36:37.380 those functions. 00:36:37.380 --> 00:36:42.110 And I can represent it as partly being colinear with 00:36:42.110 --> 00:36:43.420 this other function. 00:36:43.420 --> 00:36:47.980 And partly being orthogonal to the other function. 00:36:47.980 --> 00:36:51.660 Which says, you can forget about all of these functions 00:36:51.660 --> 00:36:57.010 which extend to infinity, in time extend to infinity, in 00:36:57.010 --> 00:36:59.370 frequency and everything else. 00:36:59.370 --> 00:37:02.400 And, so long as you're only interested in some small set 00:37:02.400 --> 00:37:06.310 of functions, you can just deal with them as a finite 00:37:06.310 --> 00:37:08.330 dimensional vector space. 00:37:08.330 --> 00:37:12.810 You can get rid of all the mess, and just think of them 00:37:12.810 --> 00:37:14.570 in this very simple sense. 00:37:14.570 --> 00:37:19.000 That's really why this vector space idea, which is called 00:37:19.000 --> 00:37:22.070 signal space, is so popular among engineers. 00:37:22.070 --> 00:37:25.420 It lets than get rid of all the mess and think in terms of 00:37:25.420 --> 00:37:27.900 very, very simple things. 00:37:27.900 --> 00:37:32.710 So let's see why this complicated 00:37:32.710 --> 00:37:37.870 theorem is in fact true. 00:37:37.870 --> 00:37:39.360 Let me state the theorem first. 00:37:39.360 --> 00:37:43.800 It says for any inner products space, v. And any vectors, u 00:37:43.800 --> 00:37:48.350 and v in v, with u unequal to 0 -- 00:37:48.350 --> 00:37:53.400 I hope I said that in the notes -- the vector v can be 00:37:53.400 --> 00:37:59.720 broken up into a colinear term plus an orthogonal term, where 00:37:59.720 --> 00:38:04.140 the colinear term is equal to a scalar times u. 00:38:04.140 --> 00:38:05.980 That's what we mean by colinear. 00:38:05.980 --> 00:38:09.070 That's just the definition of colinear. 00:38:09.070 --> 00:38:13.210 And the other vector is orthogonal to u. 00:38:13.210 --> 00:38:18.770 And alpha is uniquely given by the inner product v u divided 00:38:18.770 --> 00:38:22.280 by the norm squared of u. 00:38:22.280 --> 00:38:24.400 Now, there's one thing ugly about this. 00:38:24.400 --> 00:38:28.830 You see that norm squared, you say, what is that doing there. 00:38:28.830 --> 00:38:31.730 It just looks like it's making the formula complicated. 00:38:31.730 --> 00:38:33.900 Forgot about that for the time being. 00:38:33.900 --> 00:38:37.480 We will get into it in a minute and explain why we have 00:38:37.480 --> 00:38:39.190 that problem. 00:38:39.190 --> 00:38:41.670 But, for the moment, let's just prove this 00:38:41.670 --> 00:38:43.540 and see what it says. 00:38:43.540 --> 00:38:48.040 So what we're going to do is say, well, if I don't look at 00:38:48.040 --> 00:38:51.810 what the theorem is saying, what I'd like to do is look at 00:38:51.810 --> 00:39:00.520 some generic element which is a scalar multiple times u. 00:39:00.520 --> 00:39:06.780 So I'll say, OK let v parallel to u be alpha times u. 00:39:06.780 --> 00:39:09.510 I don't know what alpha is yet, but alpha's going to be 00:39:09.510 --> 00:39:11.340 whatever it has to be. 00:39:11.340 --> 00:39:15.570 We're going to choose alpha so that this other vector, v 00:39:15.570 --> 00:39:19.960 minus v u, is a thing we're calling v perp. 00:39:19.960 --> 00:39:22.680 So that that, the inner product of that and u, is 00:39:22.680 --> 00:39:23.890 equal to 0. 00:39:23.890 --> 00:39:26.010 So what I'm trying to do is to find a vector -- 00:39:29.570 --> 00:39:35.070 strategy here, is to take any old vector along this line and 00:39:35.070 --> 00:39:38.810 try to choose the scalar alpha in such a way that the 00:39:38.810 --> 00:39:43.610 difference between this point and this point is orthogonal 00:39:43.610 --> 00:39:45.370 to this line. 00:39:45.370 --> 00:39:47.930 That's why I started out with alpha unknown. 00:39:47.930 --> 00:39:50.140 Alpha unknown just says we have any 00:39:50.140 --> 00:39:51.760 point along this line. 00:39:51.760 --> 00:39:54.660 Now I'm going to find out what alpha has to be. 00:39:54.660 --> 00:39:58.180 I hope I will find out that it has to be only one thing, and 00:39:58.180 --> 00:40:00.980 it's uniquely chosen. 00:40:00.980 --> 00:40:04.550 And that's what we're going to find. 00:40:04.550 --> 00:40:10.230 So v minus this projection term, this is called a 00:40:10.230 --> 00:40:20.260 projection of v on u, is equal to v u minus a projection 00:40:20.260 --> 00:40:22.410 inner product with u. 00:40:22.410 --> 00:40:24.330 So it's equal to this difference here. 00:40:24.330 --> 00:40:27.870 This is equal to the inner product. 00:40:27.870 --> 00:40:32.230 Since we have chosen this term to be alpha times u, we can 00:40:32.230 --> 00:40:33.410 bring the alpha out. 00:40:33.410 --> 00:40:37.180 So it's alpha times the inner product of u with itself. 00:40:37.180 --> 00:40:39.020 That's the norm squared of alpha. 00:40:39.020 --> 00:40:42.850 So this is inner product of v and u minus alpha times the 00:40:42.850 --> 00:40:44.100 norm squared. 00:40:46.550 --> 00:40:50.870 This is 0 if and only if you set this equal to 0. 00:40:50.870 --> 00:40:54.720 And the only value alpha can have to make this 0 is the 00:40:54.720 --> 00:41:01.000 inner product of v and u divided by the norm squared. 00:41:01.000 --> 00:41:10.120 So I would think that if I ask you to prove this without 00:41:10.120 --> 00:41:13.920 knowing the projection theorem, I would hope that if 00:41:13.920 --> 00:41:16.640 you weren't afraid of it or something, and you just sat 00:41:16.640 --> 00:41:18.980 down and tried to do it, you would all do it in 00:41:18.980 --> 00:41:20.750 about half an hour. 00:41:20.750 --> 00:41:24.620 It would probably take most of you longer than that, because 00:41:24.620 --> 00:41:27.360 everybody gets screwed up in the notation when they first 00:41:27.360 --> 00:41:28.600 try to do this. 00:41:28.600 --> 00:41:30.900 But, in fact, this is not a complicated thing. 00:41:30.900 --> 00:41:33.420 This is not rocket science. 00:41:38.170 --> 00:41:38.970 Now. 00:41:38.970 --> 00:41:40.860 What is this norm squared doing here? 00:41:40.860 --> 00:41:45.100 The thing we have just proven is that with any two vectors v 00:41:45.100 --> 00:41:51.390 and u, the projection of v on u, namely that vector, there 00:41:51.390 --> 00:41:55.390 which has the property that v minus v u is perpendicular to 00:41:55.390 --> 00:42:01.510 u, we showed, is this inner product divided by the norm 00:42:01.510 --> 00:42:03.320 squared times u. 00:42:03.320 --> 00:42:06.070 Now, let me break up this norm squared. 00:42:06.070 --> 00:42:07.790 Which is just some positive number. 00:42:07.790 --> 00:42:10.360 It's a positive real number. 00:42:10.360 --> 00:42:12.970 Into the length times the length. 00:42:12.970 --> 00:42:16.670 Namely, the norm u is just some real number. 00:42:16.670 --> 00:42:21.950 And write it this way, as the inner product of v with u 00:42:21.950 --> 00:42:27.930 divided by the length of u times u divided by 00:42:27.930 --> 00:42:29.890 the length of u. 00:42:29.890 --> 00:42:33.540 Now, what is the vector u divided by the length of u? 00:42:33.540 --> 00:42:35.860 AUDIENCE: [UNINTELLIGIBLE] 00:42:35.860 --> 00:42:36.140 PROFESSOR: What? 00:42:36.140 --> 00:42:38.620 AUDIENCE: [UNINTELLIGIBLE] 00:42:38.620 --> 00:42:41.220 PROFESSOR: It's the same direction of u, but 00:42:41.220 --> 00:42:43.500 it has length 1. 00:42:43.500 --> 00:42:46.360 In other words, this is the normalized form of u. 00:42:49.020 --> 00:42:52.430 And I have the normalized form of u in here also. 00:42:52.430 --> 00:42:59.560 So what this is saying is that this projection is also equal 00:42:59.560 --> 00:43:06.640 to the projection of v on the normalized form of u, times 00:43:06.640 --> 00:43:09.560 the normalized form of u. 00:43:09.560 --> 00:43:13.530 Which says that it doesn't make any difference what the 00:43:13.530 --> 00:43:15.300 length of u is. 00:43:15.300 --> 00:43:21.670 This projection is a function only of the direction of u. 00:43:21.670 --> 00:43:25.070 I mean, this is obvious from the picture, isn't it? 00:43:28.900 --> 00:43:33.230 But again, since we can't draw pictures for complex valued 00:43:33.230 --> 00:43:36.200 things, it's nice to be able to see it analytically. 00:43:36.200 --> 00:43:41.380 If I shorten u, or lengthen u, this projection is still going 00:43:41.380 --> 00:43:43.080 to be exactly the same thing. 00:43:46.980 --> 00:43:49.730 And that's what the norm squared of u is doing here. 00:43:49.730 --> 00:43:54.060 The norm squared of u is simply sitting there so it 00:43:54.060 --> 00:43:57.430 does this normalization function for us. 00:43:57.430 --> 00:44:03.520 It makes this projection equal to the inner product of v with 00:44:03.520 --> 00:44:05.970 the normalized form of u, times the 00:44:05.970 --> 00:44:07.730 normalized vector for u. 00:44:11.780 --> 00:44:16.090 The Pythagorean theorem, which doesn't follow from this, it's 00:44:16.090 --> 00:44:20.170 something else, but it's simple -- in fact, we can do 00:44:20.170 --> 00:44:24.660 it right away -- it says if v and u are orthogonal, then the 00:44:24.660 --> 00:44:29.320 norm squared of u plus v is equal to u 00:44:29.320 --> 00:44:31.990 squared plus v squared. 00:44:31.990 --> 00:44:33.120 I mean, this is something you use 00:44:33.120 --> 00:44:35.090 geometrically all the time. 00:44:35.090 --> 00:44:36.270 And you're familiar with this. 00:44:36.270 --> 00:44:39.990 And the argument is, you just break this up into the norm 00:44:39.990 --> 00:44:43.450 squared of u plus the norm squared of v, plus the two 00:44:43.450 --> 00:44:48.440 cross-products, the inner product of u times the inner 00:44:48.440 --> 00:44:52.410 product of u with v, which is 0, and the inner product of v 00:44:52.410 --> 00:44:53.880 with u, which is 0. 00:44:53.880 --> 00:44:56.320 So the two cross-terms go away and you're 00:44:56.320 --> 00:44:59.700 left with just this. 00:44:59.700 --> 00:45:03.870 And for any v and u, the Schwarz inequality says that 00:45:03.870 --> 00:45:06.720 the inner product, the magnitude of the inner product 00:45:06.720 --> 00:45:12.140 of v and u, is less than or equal to the length of v times 00:45:12.140 --> 00:45:13.390 the length of u. 00:45:15.990 --> 00:45:20.290 The Schwarz inequality is probably -- well, I'm not sure 00:45:20.290 --> 00:45:22.670 that it's probably, but it's perhaps the most used 00:45:22.670 --> 00:45:25.680 inequality in mathematics. 00:45:25.680 --> 00:45:29.570 Any time you use vectors, you use this all the time. 00:45:29.570 --> 00:45:31.020 And it's extremely useful. 00:45:31.020 --> 00:45:34.540 I'm not going to prove it here because it's in the notes. 00:45:34.540 --> 00:45:37.940 It's a two-step proof from what we've done. 00:45:37.940 --> 00:45:43.020 And the trouble is watching two-step proofs in class. 00:45:43.020 --> 00:45:45.970 At a certain point you saturate on them. 00:45:45.970 --> 00:45:48.160 And I have more important things I want to do later 00:45:48.160 --> 00:45:50.860 today, so I don't want you to saturate on this. 00:45:50.860 --> 00:45:53.590 You can read this at your leisure and 00:45:53.590 --> 00:45:54.840 see why this is true. 00:45:57.610 --> 00:46:00.620 I did want to say something about it, though. 00:46:00.620 --> 00:46:05.880 If you divide the left side by the right side, you can write 00:46:05.880 --> 00:46:09.660 this as the magnitude of the inner product of the 00:46:09.660 --> 00:46:15.780 normalized form of v with the normalized form of u. 00:46:15.780 --> 00:46:20.610 If we're talking about real vector space, this in fact is 00:46:20.610 --> 00:46:26.390 the cosine of the angle between v and u. 00:46:26.390 --> 00:46:28.170 It's less than or equal to 1. 00:46:28.170 --> 00:46:31.410 So for real two-dimensional vectors, the fact that the 00:46:31.410 --> 00:46:35.170 cosine is less than or equal to 1 is really equivalent to 00:46:35.170 --> 00:46:36.920 the Schwarz inequality. 00:46:36.920 --> 00:46:40.210 And this is the appropriate generalization for any old 00:46:40.210 --> 00:46:41.460 vectors at all. 00:46:45.900 --> 00:46:51.500 And the notes would say more about that if that went a 00:46:51.500 --> 00:46:54.900 little too quickly. 00:46:54.900 --> 00:47:01.430 OK, the inner product space of interest to us is this thing 00:47:01.430 --> 00:47:02.720 we've called signal space. 00:47:02.720 --> 00:47:08.550 Namely, it's a space of functions which are measurable 00:47:08.550 --> 00:47:10.010 n square integrals. 00:47:10.010 --> 00:47:15.730 In other words, finite value when you take the square and 00:47:15.730 --> 00:47:18.410 integrate it. 00:47:18.410 --> 00:47:20.790 And we want to be able to talk about the set of either real 00:47:20.790 --> 00:47:24.940 or complex L2 functions. 00:47:24.940 --> 00:47:28.290 So, either one of them, we're going to define the inner 00:47:28.290 --> 00:47:30.770 product in the same way for each. 00:47:30.770 --> 00:47:33.270 It really is the only natural way to define an 00:47:33.270 --> 00:47:34.260 inner product here. 00:47:34.260 --> 00:47:38.640 And you'll see this more later as we start doing other things 00:47:38.640 --> 00:47:40.090 with these inner products. 00:47:40.090 --> 00:47:45.070 But just like what when you're dealing with n-tuples, there's 00:47:45.070 --> 00:47:47.830 only one sensible way to define an inner product. 00:47:47.830 --> 00:47:51.880 You can define inner products in other ways. 00:47:51.880 --> 00:47:54.480 But it's just a little bizarre to do so. 00:47:54.480 --> 00:47:57.510 And here it's a little bizarre also. 00:47:57.510 --> 00:48:01.500 There's a big technical problem here. 00:48:01.500 --> 00:48:04.090 And if you look at, it can anybody spot what it -- no, 00:48:04.090 --> 00:48:06.090 no, of course you can't spot what it is unless you've read 00:48:06.090 --> 00:48:09.990 the notes and you know what it is. 00:48:09.990 --> 00:48:17.070 One of the axioms of an inner product space is that the only 00:48:17.070 --> 00:48:22.310 vector in the space which has an inner product with itself, 00:48:22.310 --> 00:48:27.810 a squared norm equal to zero is the zero vector. 00:48:27.810 --> 00:48:30.940 Now, we have all these crazy vector we've been talking 00:48:30.940 --> 00:48:35.880 about, which are zero, except on a set of measures zero. 00:48:35.880 --> 00:48:39.990 In other words, vectors' functions which have zero 00:48:39.990 --> 00:48:45.430 energy but just pop up at various isolated points and 00:48:45.430 --> 00:48:47.400 have values there. 00:48:47.400 --> 00:48:50.320 Which we really can't get rid of if we view a function as 00:48:50.320 --> 00:48:54.320 something which is defined at every value of t. 00:48:54.320 --> 00:48:56.760 You have to accept those things as part of what you're 00:48:56.760 --> 00:48:57.800 dealing with. 00:48:57.800 --> 00:48:59.780 As soon as you started integrating things, those 00:48:59.780 --> 00:49:01.600 things all disappear. 00:49:01.600 --> 00:49:05.930 But the trouble is, those functions, which are zero 00:49:05.930 --> 00:49:08.900 almost everywhere, are not zero. 00:49:08.900 --> 00:49:10.830 They're only zero almost everywhere. 00:49:10.830 --> 00:49:13.240 They're zero for all engineering purposes. 00:49:13.240 --> 00:49:17.700 But they're not zero, they're only zero almost everywhere. 00:49:17.700 --> 00:49:24.330 Well, if you define the inner product in this way and you 00:49:24.330 --> 00:49:28.790 want to satisfy the axioms of an inner product space, you're 00:49:28.790 --> 00:49:30.250 out of luck. 00:49:30.250 --> 00:49:31.900 There's no way you can do it, because this 00:49:31.900 --> 00:49:35.530 axiom just gets violated. 00:49:35.530 --> 00:49:37.010 So what do you do? 00:49:37.010 --> 00:49:39.040 Well, you do what we've been doing all along. 00:49:39.040 --> 00:49:41.940 We've been sort of squinting a little bit and saying, well, 00:49:41.940 --> 00:49:47.060 really, these functions of measure 0 are really 0 for all 00:49:47.060 --> 00:49:49.400 practical purposes. 00:49:49.400 --> 00:49:52.820 Mathematically, what we have to say is, we want to talk 00:49:52.820 --> 00:49:56.730 about an equivalence class of functions. 00:49:56.730 --> 00:50:00.510 And two functions are in the same equivalence class if 00:50:00.510 --> 00:50:03.630 their difference has 0 energy. 00:50:03.630 --> 00:50:08.340 Which is equivalent to saying their difference is zero 00:50:08.340 --> 00:50:09.820 almost everywhere. 00:50:09.820 --> 00:50:12.580 It's one of these bizarre functions which just jumps up 00:50:12.580 --> 00:50:15.890 at isolated points and doesn't do anything else. 00:50:15.890 --> 00:50:19.610 Not impulses at isolated points, just non-zero at 00:50:19.610 --> 00:50:21.450 isolated points. 00:50:21.450 --> 00:50:23.850 Impulses are not really functions at all. 00:50:23.850 --> 00:50:27.010 So we're talking about things that are functions, but 00:50:27.010 --> 00:50:30.290 they're these bizarre functions which we talked 00:50:30.290 --> 00:50:31.840 about and we've said they're unimportant. 00:50:31.840 --> 00:50:35.190 So, but they are there. 00:50:35.190 --> 00:50:38.450 So the solution is to associate vectors with 00:50:38.450 --> 00:50:40.880 equivalence classes. 00:50:40.880 --> 00:50:44.500 And d of t and u of t are equivalent if the v of t minus 00:50:44.500 --> 00:50:47.750 u of t is zero almost everywhere. 00:50:47.750 --> 00:50:51.680 In other words, when we talk about a vector, u, what we're 00:50:51.680 --> 00:50:53.050 talking about is an 00:50:53.050 --> 00:50:55.240 equivalence class of functions. 00:50:55.240 --> 00:51:01.640 It's the equivalence class of functions for which two 00:51:01.640 --> 00:51:04.700 functions are in the same equivalence class if they 00:51:04.700 --> 00:51:08.020 differ only on a set of measure zero. 00:51:08.020 --> 00:51:10.100 In other words, these are the things that gave us trouble 00:51:10.100 --> 00:51:12.490 when we were talking about Fourier transforms. 00:51:12.490 --> 00:51:14.380 These are the things that gave us trouble when we were 00:51:14.380 --> 00:51:18.090 talking about Fourier series. 00:51:18.090 --> 00:51:21.230 When you take anything in the same equivalence class, 00:51:21.230 --> 00:51:24.570 time-limited functions, and you form a Fourier series, 00:51:24.570 --> 00:51:26.830 what happens? 00:51:26.830 --> 00:51:30.000 All of the things in the same equivalence class have the 00:51:30.000 --> 00:51:31.250 same Fourier coefficients. 00:51:33.710 --> 00:51:36.630 But when you go back from the Fourier series coefficients 00:51:36.630 --> 00:51:40.530 back to the function, then you might go back in a bunch of 00:51:40.530 --> 00:51:41.600 different ways. 00:51:41.600 --> 00:51:44.970 So, we started using this limit in the mean notation and 00:51:44.970 --> 00:51:47.250 all of that stuff. 00:51:47.250 --> 00:51:50.020 And what we're doing here now is, for these vectors, we're 00:51:50.020 --> 00:51:54.630 just saying, let's represent a vector as this whole 00:51:54.630 --> 00:51:55.880 equivalence class. 00:52:03.300 --> 00:52:06.090 While we're talking about vectors, we're almost always 00:52:06.090 --> 00:52:09.460 interested in orthogonal expansions. 00:52:09.460 --> 00:52:14.340 And when we're interested in orthogonal expansions, the 00:52:14.340 --> 00:52:18.400 coefficients in the orthogonal expansions are found as 00:52:18.400 --> 00:52:21.040 integrals with the function. 00:52:21.040 --> 00:52:23.520 And the integrals with different functions in the 00:52:23.520 --> 00:52:26.260 same equivalence class are identical. 00:52:26.260 --> 00:52:28.710 In other words, any two functions in the same 00:52:28.710 --> 00:52:33.500 equivalence class have the same coefficients in any 00:52:33.500 --> 00:52:36.430 orthogonal expansion. 00:52:36.430 --> 00:52:40.410 So if you talk only about the orthogonal expansion, and 00:52:40.410 --> 00:52:44.645 leave out these detailed notions of what the function 00:52:44.645 --> 00:52:51.310 is doing at individual times, then you don't have to worry 00:52:51.310 --> 00:52:52.860 about equivalence classes. 00:52:52.860 --> 00:52:56.190 In other words, when you -- 00:52:56.190 --> 00:52:59.020 I'm going to say this again more carefully later, but let 00:52:59.020 --> 00:53:01.910 me try to say it now a little bit crudely. 00:53:01.910 --> 00:53:04.500 One of the things we're interested in doing this 00:53:04.500 --> 00:53:08.740 taking this class of functions, mapping each 00:53:08.740 --> 00:53:12.660 function into a set of coefficients, where the set of 00:53:12.660 --> 00:53:14.880 coefficients are the coefficients in some 00:53:14.880 --> 00:53:18.400 particular orthogonal expansion that we're using. 00:53:18.400 --> 00:53:21.990 Namely, that's the whole way that we're using to get from 00:53:21.990 --> 00:53:24.610 waveforms to sequences. 00:53:24.610 --> 00:53:27.600 It's the whole -- 00:53:27.600 --> 00:53:30.620 it's the entire thing we're doing when we start out on a 00:53:30.620 --> 00:53:35.260 channel with a sequence of binary digits and then a 00:53:35.260 --> 00:53:36.790 sequence of symbols. 00:53:36.790 --> 00:53:40.110 And we modulate it into a waveform. 00:53:40.110 --> 00:53:46.310 Again, it's the mapping from sequence to waveform. 00:53:46.310 --> 00:53:49.320 Now, the important thing here about these equivalence 00:53:49.320 --> 00:53:54.790 classes is, you can't tell any of the members of the 00:53:54.790 --> 00:53:59.190 equivalence class apart within the sequence that we're 00:53:59.190 --> 00:54:00.900 dealing with. 00:54:00.900 --> 00:54:02.760 Everything we're interested in has to 00:54:02.760 --> 00:54:05.680 do with these sequences. 00:54:05.680 --> 00:54:08.180 I mean, if we could we'd just ignore the waveforms 00:54:08.180 --> 00:54:09.970 altogether. 00:54:09.970 --> 00:54:14.970 Because all the processing that we do is with sequences. 00:54:14.970 --> 00:54:19.340 So the only reason we have these equivalence classes is 00:54:19.340 --> 00:54:23.970 because we need them to really define what the functions are. 00:54:23.970 --> 00:54:27.280 So, we will come back to that later. 00:54:31.780 --> 00:54:33.240 Boy, I think I'm going to get done today. 00:54:33.240 --> 00:54:34.490 That's surprising. 00:54:37.130 --> 00:54:39.040 The next idea that we want to talk 00:54:39.040 --> 00:54:41.940 about is vector subspaces. 00:54:41.940 --> 00:54:45.130 Again, that's an idea you've probably heard of, 00:54:45.130 --> 00:54:46.390 for the most part. 00:54:46.390 --> 00:54:51.250 A subspace of a vector space is a subset of the vector 00:54:51.250 --> 00:54:55.370 space such that that subspace is a vector 00:54:55.370 --> 00:54:58.440 space in its own right. 00:54:58.440 --> 00:55:04.230 An equivalent definition is, for all vectors u and v, in 00:55:04.230 --> 00:55:08.060 the subspace alpha times u plus beta 00:55:08.060 --> 00:55:11.390 times v is in s also. 00:55:11.390 --> 00:55:15.060 In other words, a subspace is something which you can't get 00:55:15.060 --> 00:55:17.140 out of by linear combinations. 00:55:19.940 --> 00:55:25.760 If I take one of these diagrams back here that I keep 00:55:25.760 --> 00:55:29.230 looking at -- 00:55:29.230 --> 00:55:30.480 I'll use this one. 00:55:34.590 --> 00:55:38.820 If I want to form a subspace of this subspace, if I want to 00:55:38.820 --> 00:55:42.160 form a subspace of this two-dimensional vector space 00:55:42.160 --> 00:55:45.150 here, one of the subspaces includes 00:55:45.150 --> 00:55:48.500 u, but not v, perhaps. 00:55:48.500 --> 00:55:51.180 Now, if I want to make a one-dimensional subspace 00:55:51.180 --> 00:55:55.760 including u, what is that subspace? 00:55:55.760 --> 00:55:57.700 It's just all the scalars times u. 00:55:57.700 --> 00:56:01.120 In other words, it's this line that goes through the origin 00:56:01.120 --> 00:56:04.590 and through that vector, u. 00:56:04.590 --> 00:56:07.640 And a subspace has to include the whole line. 00:56:07.640 --> 00:56:08.490 That's what we're saying. 00:56:08.490 --> 00:56:10.950 It's all scalar multiples of u. 00:56:10.950 --> 00:56:14.760 If I want a subspace that includes u and v, where u and 00:56:14.760 --> 00:56:19.790 v are just arbitrary vectors I've chosen out of my hat, 00:56:19.790 --> 00:56:22.500 then I have this two-dimensional subspace, 00:56:22.500 --> 00:56:24.880 which is what I've drawn here. 00:56:24.880 --> 00:56:26.480 In other words, this idea is something 00:56:26.480 --> 00:56:28.900 we've been using already. 00:56:28.900 --> 00:56:32.010 It's just that we didn't need to be explicit about it. 00:56:32.010 --> 00:56:37.440 The subspace which includes u and v -- 00:56:37.440 --> 00:56:40.500 well, a subspace which includes u and v, is the 00:56:40.500 --> 00:56:44.610 subspace of all linear combinations of u and v. And 00:56:44.610 --> 00:56:45.580 nothing else. 00:56:45.580 --> 00:56:47.830 So, it's all vectors along here. 00:56:47.830 --> 00:56:49.570 It's all vectors along here. 00:56:49.570 --> 00:56:53.050 And you fill it all in with anything here added to 00:56:53.050 --> 00:56:54.070 anything here. 00:56:54.070 --> 00:56:56.210 So you get this two-dimensional space. 00:56:58.900 --> 00:57:03.050 Is 0 always in a subspace? 00:57:03.050 --> 00:57:04.140 Of course it is. 00:57:04.140 --> 00:57:07.120 I mean, you multiply any vector by 0 and 00:57:07.120 --> 00:57:09.640 you get the 0 vector. 00:57:09.640 --> 00:57:13.220 So you sort of get it as a linear -- 00:57:17.410 --> 00:57:18.660 what more can I say? 00:57:21.530 --> 00:57:25.750 If we have a vector space which is an inner product 00:57:25.750 --> 00:57:28.960 space; in other words, if we add this inner product 00:57:28.960 --> 00:57:33.310 definition to our vector space, and I take a subspace 00:57:33.310 --> 00:57:39.880 of it, that can be defined as an inner product space also, 00:57:39.880 --> 00:57:42.010 with the same definition of inner 00:57:42.010 --> 00:57:45.400 product that I had before. 00:57:45.400 --> 00:57:49.330 Because I can't get out of it by linear combinations, and 00:57:49.330 --> 00:57:53.450 the inner product is defined for every pair of vectors in 00:57:53.450 --> 00:57:54.770 that space. 00:57:54.770 --> 00:57:59.770 So we still have a nice, well-defined vector space, 00:57:59.770 --> 00:58:05.850 which is an inner product space, and which is a subspace 00:58:05.850 --> 00:58:09.030 of the space we started with. 00:58:09.030 --> 00:58:12.280 Everything I do from now on, I'm going to assume that v is 00:58:12.280 --> 00:58:14.650 an inner product space. 00:58:14.650 --> 00:58:18.570 And want to look at how we normalize vectors. 00:58:18.570 --> 00:58:20.200 We've already talked about that. 00:58:23.170 --> 00:58:25.400 If I have a vector in this vector space that's 00:58:25.400 --> 00:58:29.420 normalized, if its norm equals 1. 00:58:29.420 --> 00:58:31.950 We already decided how to normalize a vector. 00:58:31.950 --> 00:58:35.750 We took an arbitrary vector, u, divided by its norm. 00:58:35.750 --> 00:58:41.380 And as soon as we divide by its norm, that vector, v, 00:58:41.380 --> 00:58:48.540 divided by the norm of v, the norm of that is just 1. 00:58:48.540 --> 00:58:52.570 So the projection, what the projection theorem says, and 00:58:52.570 --> 00:58:55.020 all I need here is a one-dimensional projection 00:58:55.020 --> 00:59:00.770 theorem, it says that v, in the direction of this vector 00:59:00.770 --> 00:59:03.910 phi, is equal to the inner product of u 00:59:03.910 --> 00:59:06.000 with phi times phi. 00:59:06.000 --> 00:59:06.520 That's what we said. 00:59:06.520 --> 00:59:10.180 As soon as we normalized these vectors, that ugly denominator 00:59:10.180 --> 00:59:11.230 here disappears. 00:59:11.230 --> 00:59:16.020 Because the norm of phi is equal to -- 00:59:16.020 --> 00:59:18.760 because the norm is equal to 1. 00:59:18.760 --> 00:59:23.920 So, an orthonormal set of vectors is a set such that 00:59:23.920 --> 00:59:28.270 each pair of vectors is orthogonal to each other, and 00:59:28.270 --> 00:59:30.330 where each vector is normalized. 00:59:30.330 --> 00:59:34.230 In other words, it has norm squared equal to 1. 00:59:34.230 --> 00:59:40.690 So, the inner product of these vectors is just delta sub j k. 00:59:43.310 --> 00:59:50.390 If I have an orthogonal set, v sub j, say, then phi sub j is 00:59:50.390 --> 00:59:54.230 an orthonormal set just by taking each of these vectors 00:59:54.230 --> 00:59:55.890 and normalizing it. 00:59:55.890 --> 00:59:58.980 In other words, it's no big deal to take an orthogonal set 00:59:58.980 --> 01:00:01.440 of functions and turn them into an 01:00:01.440 --> 01:00:02.990 orthonormal set of functions. 01:00:02.990 --> 01:00:07.060 The Fourier series was natural to define that, and most 01:00:07.060 --> 01:00:12.070 people define it, in such a way that it's not orthonormal. 01:00:12.070 --> 01:00:15.650 Because we're defining it over some interval of time, t, that 01:00:15.650 --> 01:00:20.250 has a norm squared of t and, therefore, you have to divide 01:00:20.250 --> 01:00:23.920 by square root of t to normalize it. 01:00:23.920 --> 01:00:27.310 If you want to put everything in a common framework, it's 01:00:27.310 --> 01:00:30.200 nice to deal with orthonormal series. 01:00:30.200 --> 01:00:33.510 And therefore, that's what we're going to be stressing 01:00:33.510 --> 01:00:34.760 from now on. 01:00:38.360 --> 01:00:42.670 So I want to go on so the real projection theorem. 01:00:42.670 --> 01:00:45.290 Actually, there are three projection theorems. 01:00:45.290 --> 01:00:48.370 There's the one-dimensional projection theorem. 01:00:48.370 --> 01:00:50.900 There's the n-dimensional projection theorem, which is 01:00:50.900 --> 01:00:52.500 what this is. 01:00:52.500 --> 01:00:55.200 And then there's an infinite-dimensional 01:00:55.200 --> 01:00:58.660 projection theorem, which is not general for all inner 01:00:58.660 --> 01:01:04.860 product spaces, but is certainly general for L2. 01:01:04.860 --> 01:01:10.070 So I'm going to assume that phi 1 to phi sub n is an 01:01:10.070 --> 01:01:13.960 orthonormal basis for an n-dimensional subspace, s, 01:01:13.960 --> 01:01:17.490 which is a subspace of v. How do I know there is such an 01:01:17.490 --> 01:01:20.220 orthonormal basis? 01:01:20.220 --> 01:01:22.820 Well, I don't know that yet, and I'm going to come back to 01:01:22.820 --> 01:01:24.010 that later. 01:01:24.010 --> 01:01:26.550 But for the time being, I'm just going to assume that as 01:01:26.550 --> 01:01:28.510 part of the theorem. 01:01:28.510 --> 01:01:35.170 Assume I have some particular subspace which has the 01:01:35.170 --> 01:01:39.320 property that it has an orthonormal basis. 01:01:39.320 --> 01:01:43.180 So this is an orthonormal basis for this n-dimensional 01:01:43.180 --> 01:01:50.760 subspace, s and v. For each vector in v, s now is some 01:01:50.760 --> 01:01:52.290 small subspace. 01:01:52.290 --> 01:01:57.430 v is a big subspace out around s. 01:01:57.430 --> 01:02:00.340 What I want to do now is, I want to take some vector in 01:02:00.340 --> 01:02:01.920 the big subspace. 01:02:01.920 --> 01:02:04.170 I want to project it onto the subspace. 01:02:06.800 --> 01:02:10.410 By projecting it onto the subspace, what I mean is, I 01:02:10.410 --> 01:02:18.210 want to find some vector in the subspace such that the 01:02:18.210 --> 01:02:22.620 difference between v and that point in the subspace is 01:02:22.620 --> 01:02:25.980 orthogonal to the subspace itself. 01:02:25.980 --> 01:02:33.020 In other words, it's the same idea as we used before for 01:02:33.020 --> 01:02:36.950 this over-used picture. 01:02:36.950 --> 01:02:39.910 Which I keep looking at. 01:02:39.910 --> 01:02:43.330 Here, the subspace that I'm looking at is just the 01:02:43.330 --> 01:02:48.000 subspace of vectors colinear with u. 01:02:48.000 --> 01:02:52.620 And what I'm trying to do here is to find, from v I'm trying 01:02:52.620 --> 01:02:56.880 to drop a perpendicular to this subspace, which is just a 01:02:56.880 --> 01:02:58.370 straight line. 01:02:58.370 --> 01:03:02.490 In general, what I'm trying to do is, I have an 01:03:02.490 --> 01:03:03.950 n-dimensional subspace. 01:03:07.040 --> 01:03:09.990 We can sort of visualize a two-dimensional subspace if 01:03:09.990 --> 01:03:12.750 you think of this in three dimensions. 01:03:12.750 --> 01:03:16.950 And think of replacing u with some space, some 01:03:16.950 --> 01:03:20.950 two-dimensional space, which is going through 0. 01:03:20.950 --> 01:03:23.410 And now I have this vector, v, which is 01:03:23.410 --> 01:03:25.800 outside of that plane. 01:03:25.800 --> 01:03:29.710 And what I'm trying to do here is to drop a perpendicular 01:03:29.710 --> 01:03:32.610 from v onto that subspace. 01:03:32.610 --> 01:03:36.850 The projection is where that perpendicular lands. 01:03:36.850 --> 01:03:42.260 So, in other words, v, minus the projection, is this v perp 01:03:42.260 --> 01:03:44.410 we're talking about. 01:03:44.410 --> 01:03:47.520 And what I want to do is exactly the same thing that we 01:03:47.520 --> 01:03:48.510 did before with the 01:03:48.510 --> 01:03:51.590 one-dimensional projection theorem. 01:03:51.590 --> 01:03:54.160 And that's what we're going to do. 01:03:54.160 --> 01:03:56.620 And it works. 01:03:56.620 --> 01:03:57.770 And it isn't really any more 01:03:57.770 --> 01:04:02.070 complicated, except for notation. 01:04:02.070 --> 01:04:05.550 So the theorem says, assume you have this orthonormal 01:04:05.550 --> 01:04:09.180 basis for this subspace. 01:04:09.180 --> 01:04:14.610 And then you take any old v in the entire vector space. 01:04:14.610 --> 01:04:18.710 This can be an infinite dimensional vector space or 01:04:18.710 --> 01:04:19.470 anything else. 01:04:19.470 --> 01:04:23.470 And what we really want it to be is some element in L2, 01:04:23.470 --> 01:04:27.330 which is some infinite-dimensional element. 01:04:27.330 --> 01:04:34.950 It says there's a unique projection in the subspace s. 01:04:34.950 --> 01:04:41.220 And it's given by the inner product of v with each of 01:04:41.220 --> 01:04:43.810 those basis vectors. 01:04:43.810 --> 01:04:46.270 That inner product, times v sub j. 01:04:46.270 --> 01:04:49.160 For the case of a one-dimensional vector, you 01:04:49.160 --> 01:04:51.800 take the sum out and that was exactly the 01:04:51.800 --> 01:04:54.190 projection we had before. 01:04:54.190 --> 01:04:57.540 Now we just have the multi-dimensional projection. 01:04:57.540 --> 01:05:02.820 And it has a property that v is equal to this projection 01:05:02.820 --> 01:05:06.910 plus the orthogonal thing. 01:05:06.910 --> 01:05:11.600 And the orthogonal thing, the inner product of that with s 01:05:11.600 --> 01:05:15.790 equal to 0 for all s in the subspace. 01:05:15.790 --> 01:05:18.940 In other words, it's just what we got in this picture we were 01:05:18.940 --> 01:05:22.290 just trying to construct, of a two-dimensional plane. 01:05:22.290 --> 01:05:25.410 You drop a perpendicular two-dimensional plane. 01:05:25.410 --> 01:05:27.030 And when I drop a perpendicular to a 01:05:27.030 --> 01:05:31.040 two-dimensional plane, and I take any old vector in this 01:05:31.040 --> 01:05:34.150 two-dimensional plane, we still have the 01:05:34.150 --> 01:05:36.890 perpendicularity. 01:05:36.890 --> 01:05:44.530 In other words, you have the notion of this vector being 01:05:44.530 --> 01:05:48.240 perpendicular to a plane if it's perpendicular whichever 01:05:48.240 --> 01:05:50.600 way you look at it. 01:05:50.600 --> 01:05:53.470 Let me outline the proof of this. 01:05:53.470 --> 01:05:55.430 Actually, it's pretty much a complete proof. 01:05:55.430 --> 01:06:01.970 But with a couple small details left out. 01:06:01.970 --> 01:06:04.510 We're going to start out the same way I did before. 01:06:04.510 --> 01:06:08.170 Namely, I don't know how to choose this vector. 01:06:08.170 --> 01:06:12.390 But I know I want to choose it to be in this subspace. 01:06:12.390 --> 01:06:14.940 And any element in this subspace is a linear 01:06:14.940 --> 01:06:17.820 combination of these p sub i's. 01:06:20.560 --> 01:06:24.730 So this is just a generic element in s. 01:06:24.730 --> 01:06:28.040 And I want to find out what element I have to use. 01:06:28.040 --> 01:06:32.050 I want to find the conditions on these coefficients here 01:06:32.050 --> 01:06:36.310 such that v minus the projection -- in other words, 01:06:36.310 --> 01:06:40.160 this v perp, as we've been calling it -- is orthogonal to 01:06:40.160 --> 01:06:42.680 each phi sub i. 01:06:42.680 --> 01:06:45.980 Now, if it's orthogonal to each phi sub i, it's also 01:06:45.980 --> 01:06:48.400 orthogonal to each linear combination 01:06:48.400 --> 01:06:49.980 of the phi sub i's. 01:06:49.980 --> 01:06:53.190 So in fact, that solved our problem for us. 01:06:53.190 --> 01:06:58.310 So what I want to do is, I want to set 0 equal to v minus 01:06:58.310 --> 01:07:00.050 this projection. 01:07:00.050 --> 01:07:02.710 Where I don't yet know how to make the projection, because I 01:07:02.710 --> 01:07:04.890 don't know what the alpha sub i's are. 01:07:04.890 --> 01:07:09.265 But I'm trying to choose these so that this, minus the 01:07:09.265 --> 01:07:14.590 projection, the inner product of that with phi j is equal to 01:07:14.590 --> 01:07:18.410 0 for every j. 01:07:18.410 --> 01:07:25.750 This inner product here is equal to the inner product of 01:07:25.750 --> 01:07:28.280 v with phi sub j. 01:07:28.280 --> 01:07:32.420 I have this difference here, so the inner product is the 01:07:32.420 --> 01:07:37.460 inner product of v with phi sub j minus the inner product 01:07:37.460 --> 01:07:40.590 of this with phi sub j. 01:07:40.590 --> 01:07:44.130 Let me write that here. 01:07:47.380 --> 01:08:03.020 v minus sum alpha i phi sub i comma phi sub j is equal to v 01:08:03.020 --> 01:08:15.370 phi sub j minus summation of i alpha i t sub i. 01:08:15.370 --> 01:08:18.110 phi sub j. 01:08:18.110 --> 01:08:20.730 Which is equal to this. 01:08:20.730 --> 01:08:25.540 All of these terms are 0 except where j is equal to i. 01:08:30.270 --> 01:08:34.540 Where i is equal to j. 01:08:34.540 --> 01:08:38.590 So, alpha sub j has to be equal to the inner product of 01:08:38.590 --> 01:08:41.560 v with this basis vector here. 01:08:41.560 --> 01:08:47.240 And, therefore, this projection is equal, which we 01:08:47.240 --> 01:08:51.630 said was sum of alpha i phi sub i, that's really the sum 01:08:51.630 --> 01:08:58.340 of v phi sub j, this inner product, times p sub j, 01:08:58.340 --> 01:09:01.070 Now, if you really use your imagination and you really 01:09:01.070 --> 01:09:05.580 think hard about the formula we were using for the Fourier 01:09:05.580 --> 01:09:11.350 series coefficients, was really the same formula 01:09:11.350 --> 01:09:12.952 without the normalization in it. 01:09:12.952 --> 01:09:16.100 It's simplified by being already normalized for us. 01:09:16.100 --> 01:09:19.470 We don't have that 1 over t in here, which we had in the 01:09:19.470 --> 01:09:21.880 Fourier series because now we've gone 01:09:21.880 --> 01:09:24.570 to orthonormal functions. 01:09:28.070 --> 01:09:30.400 So, in fact that sort of proves the theorem. 01:09:42.200 --> 01:09:47.050 If we express v as some linear combination of these 01:09:47.050 --> 01:09:52.960 orthonormal vectors, then if I take the norm squared of v, 01:09:52.960 --> 01:09:55.860 this is something we've done many times already. 01:09:55.860 --> 01:09:59.570 I just express the norm squared, just by expanding 01:09:59.570 --> 01:10:08.530 this the sum of -- well, here I've done it this way, so 01:10:08.530 --> 01:10:09.930 let's do it this way again. 01:10:09.930 --> 01:10:13.000 When I take the inner product of v with all of these terms 01:10:13.000 --> 01:10:17.640 here, I get the sum of alpha sub j complex conjugated, 01:10:17.640 --> 01:10:21.660 times the inner product of v with phi sub j. 01:10:21.660 --> 01:10:25.940 But the inner product of v with phi sub j is just alpha 01:10:25.940 --> 01:10:27.910 sub j times 1. 01:10:27.910 --> 01:10:31.670 So it's a sum of alpha sub j squared. 01:10:31.670 --> 01:10:34.440 OK, this is this energy relationship we've 01:10:34.440 --> 01:10:35.630 been using all along. 01:10:35.630 --> 01:10:37.400 We've been using it for the Fourier series. 01:10:37.400 --> 01:10:41.330 We've been using it for everything we've been doing. 01:10:41.330 --> 01:10:45.910 It's just a special case of this relationship here, in 01:10:45.910 --> 01:10:49.120 this n-dimensional projection, except that there we were 01:10:49.120 --> 01:10:51.050 dealing with infinite dimensions and here we're 01:10:51.050 --> 01:10:53.160 dealing with finite dimensions. 01:10:53.160 --> 01:10:56.500 But it's the same formula, and you'll see how it generalizes 01:10:56.500 --> 01:10:59.340 in a little bit. 01:10:59.340 --> 01:11:02.400 We still have the Pythagorean theorem, which in this case 01:11:02.400 --> 01:11:07.380 says that the norm squared of vector v is equal to the norm 01:11:07.380 --> 01:11:11.200 squared of a projection, plus the norm squared of the 01:11:11.200 --> 01:11:12.690 perpendicular part. 01:11:12.690 --> 01:11:15.550 In other words, when I start to represent this vector 01:11:15.550 --> 01:11:23.640 outside of the space by a vector inside the space, by 01:11:23.640 --> 01:11:27.140 this projection, I wind up with two things. 01:11:27.140 --> 01:11:30.410 I wind up both with the part that's outside of the space 01:11:30.410 --> 01:11:34.130 entirely, and is orthogonal to the space, plus the part which 01:11:34.130 --> 01:11:35.510 is in the space. 01:11:35.510 --> 01:11:38.320 And each of those has a certain amount of energy. 01:11:38.320 --> 01:11:45.630 When I expand this by this relationship here -- 01:11:45.630 --> 01:11:49.160 I'm not doing that yet -- what I'm doing here is what's 01:11:49.160 --> 01:11:51.930 called a norm bound. 01:11:51.930 --> 01:11:57.230 Which says both of these terms are non-negative. 01:11:57.230 --> 01:12:00.940 This term is non-negative in particular, and therefore the 01:12:00.940 --> 01:12:05.780 difference between this and this is always positive, or 01:12:05.780 --> 01:12:07.450 non-negative. 01:12:07.450 --> 01:12:11.200 Which says that 0 has to be less than or equal to this, 01:12:11.200 --> 01:12:13.720 because it's non-negative. 01:12:13.720 --> 01:12:17.385 And this has to be less than or equal to the norm of v. In 01:12:17.385 --> 01:12:22.020 other words, the projection always has less energy then 01:12:22.020 --> 01:12:25.500 the vector itself. 01:12:25.500 --> 01:12:27.420 Which is not very surprising. 01:12:27.420 --> 01:12:30.000 So the norm bound is no big deal. 01:12:30.000 --> 01:12:36.500 When I substitute this for the actual value, what I get is 01:12:36.500 --> 01:12:43.310 the sum j equals 1 to n of the norm of the inner product of 01:12:43.310 --> 01:12:48.690 v, with each one of these basis vectors, magnitude 01:12:48.690 --> 01:12:53.280 squared, that's less than or equal to the energy in v. In 01:12:53.280 --> 01:12:57.370 other words, if we start to expand, as n gets bigger and 01:12:57.370 --> 01:13:01.670 bigger, and we look at these terms, we take these inner 01:13:01.670 --> 01:13:03.780 products, square them. 01:13:03.780 --> 01:13:07.400 No matter how many terms I take here, the sum is always 01:13:07.400 --> 01:13:12.620 less than or equal to the energy in v. That's called 01:13:12.620 --> 01:13:21.030 Bessel's inequality, and it's a nice, straightforward thing. 01:13:21.030 --> 01:13:27.436 And finally, the last of these things -- 01:13:32.500 --> 01:13:38.520 well, I'll use the other one if I need it. 01:13:38.520 --> 01:13:40.770 The last is this thing called the mean 01:13:40.770 --> 01:13:42.150 square error property. 01:13:44.670 --> 01:13:48.220 It says that if you take the difference between the vector 01:13:48.220 --> 01:13:52.130 and its projection onto this space, this is less than or 01:13:52.130 --> 01:13:55.290 equal to the difference between the vector and the 01:13:55.290 --> 01:13:57.095 other s in the space. 01:13:57.095 --> 01:14:00.670 Any other -- 01:14:00.670 --> 01:14:07.050 I can always represent v as being equal to this plus the 01:14:07.050 --> 01:14:09.460 orthogonal component. 01:14:09.460 --> 01:14:13.010 So I wind up with a sum here of two terms. 01:14:13.010 --> 01:14:18.910 One is the difference between -- well, it's the -- 01:14:22.140 --> 01:14:29.180 it's the length squared of the projection. 01:14:29.180 --> 01:14:30.430 Write it out. 01:14:33.170 --> 01:14:51.410 v minus v s -- let me write this term out. v minus s is 01:14:51.410 --> 01:15:04.220 equal to v, the projection, plus v perpendicular to the 01:15:04.220 --> 01:15:06.920 subspace s minus this vector s. 01:15:12.820 --> 01:15:19.155 This is perpendicular to this and this, so -- ah, 01:15:19.155 --> 01:15:20.490 to hell with it. 01:15:20.490 --> 01:15:24.310 Excuse my language. 01:15:24.310 --> 01:15:25.770 I mean, this is proven in the notes. 01:15:25.770 --> 01:15:30.600 I'm not going to go through it now because I want to finish 01:15:30.600 --> 01:15:32.480 these other things. 01:15:32.480 --> 01:15:36.410 I don't want to play around with it. 01:15:36.410 --> 01:15:39.750 We left something out of the 01:15:39.750 --> 01:15:42.690 n-dimensional projection theorem. 01:15:42.690 --> 01:15:46.800 How do you find an orthonormal basis to start with? 01:15:46.800 --> 01:15:50.630 And there's this neat thing called Gram-Schmidt, which I 01:15:50.630 --> 01:15:54.640 suspect most of you have seen before also. 01:15:54.640 --> 01:15:57.310 Which is pretty simple now in terms of 01:15:57.310 --> 01:15:59.350 the projection theorem. 01:15:59.350 --> 01:16:02.890 Gram-Schmidt is really a bootstrap operation starting 01:16:02.890 --> 01:16:05.450 with the one-dimensional projection theorem, working 01:16:05.450 --> 01:16:08.830 your way up to larger and larger dimensions. 01:16:08.830 --> 01:16:12.630 And each case winding up with an orthonormal basis for what 01:16:12.630 --> 01:16:13.490 you started with. 01:16:13.490 --> 01:16:16.420 Let's see how that happens. 01:16:16.420 --> 01:16:20.540 I start out with a basis for an inner product subspace. 01:16:20.540 --> 01:16:24.390 So, s1 up to s sub n as a basis for this 01:16:24.390 --> 01:16:26.540 inner product space. 01:16:26.540 --> 01:16:29.970 First thing I do is, I start out with s1. 01:16:29.970 --> 01:16:32.490 I find the normalized version of s1. 01:16:32.490 --> 01:16:34.500 I call that phi 1. 01:16:34.500 --> 01:16:44.740 So phi 1 is now an orthonormal basis for the subspace whose 01:16:44.740 --> 01:16:47.670 basis is just phi 1 itself. 01:16:50.280 --> 01:16:54.210 phi 1 is the basis for the subspace of all linear 01:16:54.210 --> 01:16:57.470 combinations of s1. 01:16:57.470 --> 01:17:01.490 So it's just a straight line in space. 01:17:01.490 --> 01:17:04.520 The next thing I do is, I take s2. 01:17:04.520 --> 01:17:09.310 I find the projection of s2 on this subspace s1. 01:17:09.310 --> 01:17:11.480 I can do that. 01:17:11.480 --> 01:17:15.720 So I find a part which is colinear with s1. 01:17:15.720 --> 01:17:18.240 I find the part which is orthogonal. 01:17:18.240 --> 01:17:21.460 I take the orthogonal part, and that's 01:17:21.460 --> 01:17:22.880 orthogonal, to phi 1. 01:17:22.880 --> 01:17:25.300 And I normalize it. 01:17:25.300 --> 01:17:29.540 So I then have two vectors, phi 1 and phi 2, which span 01:17:29.540 --> 01:17:32.810 the space of functions of linear 01:17:32.810 --> 01:17:34.650 combinations of s1 and s2. 01:17:37.600 --> 01:17:45.040 And I call that subspace S2, capital S2. 01:17:45.040 --> 01:17:46.700 And then I go on. 01:17:46.700 --> 01:17:50.930 So, given any orthonormal basis, phi 1 up to phi sub k 01:17:50.930 --> 01:17:55.640 of the subspace s k generated by s1 to s k, I'm going to 01:17:55.640 --> 01:18:00.730 project s k plus 1 onto this subspace s sub k, and then I'm 01:18:00.730 --> 01:18:03.170 going to normalize it. 01:18:03.170 --> 01:18:07.210 And by going through this procedure, I can in fact find 01:18:07.210 --> 01:18:10.780 an orthonormal basis to any set of vectors that I want to, 01:18:10.780 --> 01:18:12.810 to any subspace that I want to. 01:18:12.810 --> 01:18:15.730 Why is this important, is this something you want to do? 01:18:15.730 --> 01:18:20.310 Well, it's something you can program a computer to do 01:18:20.310 --> 01:18:22.770 almost trivially. 01:18:22.770 --> 01:18:24.980 But that's not why we want it here. 01:18:24.980 --> 01:18:28.780 The thing we want it here is to say that there's no reason 01:18:28.780 --> 01:18:31.870 to deal with bases other than orthonormal bases. 01:18:31.870 --> 01:18:35.280 We can generate orthonormal bases easily. 01:18:35.280 --> 01:18:38.760 The projection theorem now is valid for any n-dimensional 01:18:38.760 --> 01:18:42.230 space because for any n-dimensional space we can 01:18:42.230 --> 01:18:45.670 form this basis that we want. 01:18:45.670 --> 01:18:53.480 Let me just go on and finish this, so we can start dealing 01:18:53.480 --> 01:18:56.840 with channels next time. 01:18:56.840 --> 01:19:01.370 So far, the projection theorem is just for 01:19:01.370 --> 01:19:03.810 finite dimensional vectors. 01:19:03.810 --> 01:19:08.660 We want to now extend it to infinite dimensional vectors. 01:19:08.660 --> 01:19:13.370 To accountably infinite set of vectors. 01:19:13.370 --> 01:19:16.790 So I'm given any orthogonal set of functions, status sub 01:19:16.790 --> 01:19:20.580 i, we can first generate orthonormal functions as phi 01:19:20.580 --> 01:19:23.870 sub i, which are normalized. 01:19:23.870 --> 01:19:26.130 And that's old stuff. 01:19:26.130 --> 01:19:30.750 I can now think of doing the same thing that we did before. 01:19:30.750 --> 01:19:36.800 Namely, starting out, taking any old vector I want to, and 01:19:36.800 --> 01:19:39.880 projecting it first on to the subspace with 01:19:39.880 --> 01:19:41.860 only phi 1 in it. 01:19:41.860 --> 01:19:45.820 Then the subspace generated by phi 1 and phi 2, then the 01:19:45.820 --> 01:19:51.220 subspace generated by phi 1, phi 2 and phi 3, and so forth. 01:19:51.220 --> 01:19:55.360 When I do that successively, which is successive 01:19:55.360 --> 01:20:00.420 approximations in a Fourier expansion, or in any 01:20:00.420 --> 01:20:06.100 orthonormal expansion, what I'm going to wind up with is 01:20:06.100 --> 01:20:09.910 the following theorem that says, let phi sub m be a set 01:20:09.910 --> 01:20:12.090 of orthonormal functions. 01:20:12.090 --> 01:20:15.970 Let v be any L2 vector. 01:20:15.970 --> 01:20:20.140 Then there exists an L2 vector, u, such that v minus u 01:20:20.140 --> 01:20:22.880 is orthogonal to each phi sub n. 01:20:22.880 --> 01:20:26.720 In other words, this is the projection theorem, carried on 01:20:26.720 --> 01:20:29.220 as n goes to infinity. 01:20:29.220 --> 01:20:31.690 But I can't quite state it in the way I did before. 01:20:31.690 --> 01:20:33.180 I need a limit in here. 01:20:33.180 --> 01:20:40.280 Which says the limit as n goes to infinity of u, namely what 01:20:40.280 --> 01:20:45.610 is now going to be this projection, minus this term 01:20:45.610 --> 01:20:49.540 here, which is the term in the subspace of 01:20:49.540 --> 01:20:51.570 these orthonormal functions. 01:20:51.570 --> 01:20:55.180 This difference goes to zero. 01:20:55.180 --> 01:20:56.090 What does this say? 01:20:56.090 --> 01:21:00.360 It doesn't say that I can take any function, v, and expand it 01:21:00.360 --> 01:21:03.630 in an orthonormal expansion. 01:21:03.630 --> 01:21:06.840 I couldn't say that, because I have nothing to know whether 01:21:06.840 --> 01:21:10.750 this arbitrary orthonormal expansion I started with 01:21:10.750 --> 01:21:13.130 actually spans L2 or not. 01:21:13.130 --> 01:21:17.350 And without knowing that, I can't state a 01:21:17.350 --> 01:21:18.210 theorem like this. 01:21:18.210 --> 01:21:21.450 But what the theorem does say is, you take any orthonormal 01:21:21.450 --> 01:21:23.250 expansion you want to. 01:21:23.250 --> 01:21:26.900 Like the Fourier series, which only spans functions which are 01:21:26.900 --> 01:21:28.480 time limited. 01:21:28.480 --> 01:21:30.390 I take an arbitrary function. 01:21:30.390 --> 01:21:32.960 I expand them in this Fourier series. 01:21:32.960 --> 01:21:36.780 And, bingo, what I get is a function u, which is the part 01:21:36.780 --> 01:21:40.430 of v, which is within these time limits and what's left 01:21:40.430 --> 01:21:43.930 over, which is orthogonal, is the stuff outside of those 01:21:43.930 --> 01:21:46.730 time limits. 01:21:46.730 --> 01:21:51.420 So this is the theorem that says that in fact you can -- 01:21:51.420 --> 01:21:52.650 AUDIENCE: [UNINTELLIGIBLE] 01:21:52.650 --> 01:21:52.950 PROFESSOR: What? 01:21:52.950 --> 01:21:55.550 AUDIENCE: [UNINTELLIGIBLE] 01:21:55.550 --> 01:21:59.290 PROFESSOR: It's similar to Plancherel. 01:21:59.290 --> 01:22:04.300 Plancherel is done for the Fourier integral, and in 01:22:04.300 --> 01:22:08.840 Plancherel you need this limit in the mean on both sides. 01:22:08.840 --> 01:22:13.280 Here we're just dealing with a series. 01:22:13.280 --> 01:22:18.230 I mean, we still have a limit in the mean sort of thing, but 01:22:18.230 --> 01:22:20.720 we have a weaker theorem in the sense that we're not 01:22:20.720 --> 01:22:24.030 asserting that this orthonormal series actually 01:22:24.030 --> 01:22:26.750 spans all of L2. 01:22:26.750 --> 01:22:29.125 I mean, we found a couple of orthonormal expansions 01:22:29.125 --> 01:22:32.380 that do span L2. 01:22:32.380 --> 01:22:33.980 So it's lacking in that. 01:22:33.980 --> 01:22:35.380 OK, going to stop there.