WEBVTT 00:00:00.000 --> 00:00:00.500 align:middle line:90% [SQUEAKING] 00:00:00.500 --> 00:00:01.000 align:middle line:90% [RUSTLING] 00:00:01.000 --> 00:00:05.000 align:middle line:90% [CLICKING] 00:00:05.000 --> 00:00:18.500 align:middle line:90% 00:00:18.500 --> 00:00:25.420 align:middle line:84% PROFESSOR: OK, so last time, we discussed orthonormal bases. 00:00:25.420 --> 00:00:28.810 align:middle line:84% And then we considered the concrete question 00:00:28.810 --> 00:00:33.880 align:middle line:84% of the complex exponentials being an orthonormal basis 00:00:33.880 --> 00:00:39.100 align:middle line:84% for L2 of minus pi to pi, that Fourier series actually 00:00:39.100 --> 00:00:43.960 align:middle line:84% converge to an L2 function in the L2 norm. 00:00:43.960 --> 00:00:47.020 align:middle line:84% So now, we're going to go back to a general discussion 00:00:47.020 --> 00:00:47.770 align:middle line:90% of Hilbert space. 00:00:47.770 --> 00:00:49.728 align:middle line:84% This is kind of the way the rest of the course, 00:00:49.728 --> 00:00:52.960 align:middle line:84% will be some general theory followed 00:00:52.960 --> 00:00:58.150 align:middle line:84% by some concrete applications scattered in, 00:00:58.150 --> 00:01:00.010 align:middle line:84% concrete in the sense that it just 00:01:00.010 --> 00:01:03.740 align:middle line:84% won't be about general Hilbert spaces, but about maybe 00:01:03.740 --> 00:01:08.000 align:middle line:84% specific operators, specific problems we're trying to solve. 00:01:08.000 --> 00:01:20.900 align:middle line:90% 00:01:20.900 --> 00:01:25.080 align:middle line:84% Now, we're going to discuss length minimizers. 00:01:25.080 --> 00:01:26.320 align:middle line:90% And what do I mean by that? 00:01:26.320 --> 00:01:39.050 align:middle line:84% I mean by the following, is if I have 00:01:39.050 --> 00:01:46.470 align:middle line:84% a closed subspace of a Hilbert space, then I could define-- 00:01:46.470 --> 00:01:49.130 align:middle line:84% of course, so way back when, we could then 00:01:49.130 --> 00:01:54.570 align:middle line:84% define a norm on H modulo, this closed Hilbert 00:01:54.570 --> 00:01:58.830 align:middle line:84% space, and the norm on that is the infimum of the norm of v 00:01:58.830 --> 00:02:03.330 align:middle line:84% minus w, where w, little w, is in the linear subspace capital 00:02:03.330 --> 00:02:04.240 align:middle line:90% W. 00:02:04.240 --> 00:02:06.690 align:middle line:84% Now, a natural question is, is that infimum, 00:02:06.690 --> 00:02:11.130 align:middle line:84% is that minimum, that minimal distance actually achieved 00:02:11.130 --> 00:02:16.350 align:middle line:90% by some element in the subspace? 00:02:16.350 --> 00:02:20.300 align:middle line:84% Now, the answer is, in fact, yes. 00:02:20.300 --> 00:02:24.380 align:middle line:84% And it's, in fact, true for a larger 00:02:24.380 --> 00:02:27.650 align:middle line:84% class of subsets of Hilbert spaces than closed subspaces. 00:02:27.650 --> 00:02:30.660 align:middle line:90% So we have the following-- 00:02:30.660 --> 00:02:35.420 align:middle line:84% suppose C is a subset of a Hilbert space-- 00:02:35.420 --> 00:02:41.520 align:middle line:90% 00:02:41.520 --> 00:02:54.710 align:middle line:84% of a Hilbert space H such that three conditions hold. 00:02:54.710 --> 00:02:59.615 align:middle line:84% The silliest-- C is non-empty, C is closed. 00:02:59.615 --> 00:03:06.190 align:middle line:90% 00:03:06.190 --> 00:03:10.130 align:middle line:84% And the final is that C is what's called convex. 00:03:10.130 --> 00:03:16.250 align:middle line:90% 00:03:16.250 --> 00:03:23.660 align:middle line:84% This means the following, that if v1 and v2 are 00:03:23.660 --> 00:03:35.290 align:middle line:84% in the subset C and t is in 0, 1, then tv 1 00:03:35.290 --> 00:03:42.850 align:middle line:84% plus 1 minus tv2 is in C. So another way of stating 00:03:42.850 --> 00:03:48.220 align:middle line:84% this last condition is that for any two elements in C, 00:03:48.220 --> 00:03:52.840 align:middle line:84% the line segment from v1 to v2 is contained in C. 00:03:52.840 --> 00:03:56.650 align:middle line:84% So this is nothing but an element on the line 00:03:56.650 --> 00:04:01.060 align:middle line:90% segment joining v1 to v2. 00:04:01.060 --> 00:04:08.300 align:middle line:84% So here, we have what would look like a convex set C. 00:04:08.300 --> 00:04:13.650 align:middle line:84% And for every v1, v2, it's convex, 00:04:13.650 --> 00:04:16.018 align:middle line:84% meaning the line segment joining them is in there. 00:04:16.018 --> 00:04:16.935 align:middle line:90% So you wouldn't have-- 00:04:16.935 --> 00:04:20.110 align:middle line:90% 00:04:20.110 --> 00:04:21.880 align:middle line:90% let's say you wouldn't have-- 00:04:21.880 --> 00:04:26.080 align:middle line:90% so this is convex. 00:04:26.080 --> 00:04:35.520 align:middle line:84% And so for example, something like this picture is not convex 00:04:35.520 --> 00:04:40.650 align:middle line:84% because I could take two elements-- v1, v2-- 00:04:40.650 --> 00:04:53.725 align:middle line:84% here and the line segment does not stay in the set C. 00:04:53.725 --> 00:04:58.760 align:middle line:84% All right, so if we assume that we have a non-empty, closed, 00:04:58.760 --> 00:05:01.850 align:middle line:84% and convex subset of a Hilbert space H, 00:05:01.850 --> 00:05:16.900 align:middle line:84% then there exists a unique element v in C such that-- 00:05:16.900 --> 00:05:19.511 align:middle line:90% or I should say-- 00:05:19.511 --> 00:05:27.590 align:middle line:84% yeah, v in C such that v is equal to the minimum 00:05:27.590 --> 00:05:40.380 align:middle line:90% of the length of u in C. 00:05:40.380 --> 00:05:44.510 align:middle line:84% Now, we'll see, when we apply this theorem, 00:05:44.510 --> 00:05:46.740 align:middle line:90% how such C's pop up. 00:05:46.740 --> 00:05:49.570 align:middle line:84% One of them is, like I said a minute ago, that-- 00:05:49.570 --> 00:05:51.320 align:middle line:84% OK, I didn't exactly say it this way, 00:05:51.320 --> 00:05:56.735 align:middle line:90% but C being a vector plus a-- 00:05:56.735 --> 00:05:59.750 align:middle line:90% 00:05:59.750 --> 00:06:04.220 align:middle line:84% so C could be some fixed vector plus a subspace, 00:06:04.220 --> 00:06:06.590 align:middle line:84% meaning the set of all vectors of the form v 00:06:06.590 --> 00:06:12.052 align:middle line:84% plus w, where w is in some subspace. 00:06:12.052 --> 00:06:14.510 align:middle line:84% But that's not necessarily the only kind we'll come across. 00:06:14.510 --> 00:06:18.710 align:middle line:84% OK, so if we have a closed, convex, non-empty subset 00:06:18.710 --> 00:06:23.930 align:middle line:84% of a Hilbert space, then there exists a unique minimizer 00:06:23.930 --> 00:06:24.830 align:middle line:90% in this subspace. 00:06:24.830 --> 00:06:27.730 align:middle line:90% 00:06:27.730 --> 00:06:33.120 align:middle line:84% So of course, I encourage you to think about 00:06:33.120 --> 00:06:36.960 align:middle line:84% if I drop either of these conditions 00:06:36.960 --> 00:06:41.340 align:middle line:84% whether or not this theorem remains true. 00:06:41.340 --> 00:06:45.455 align:middle line:84% So it's clear you cannot drop C being closed. 00:06:45.455 --> 00:06:48.350 align:middle line:90% 00:06:48.350 --> 00:06:55.220 align:middle line:84% So for example, this just remark you cannot drop the condition C 00:06:55.220 --> 00:06:55.730 align:middle line:90% closed. 00:06:55.730 --> 00:07:07.530 align:middle line:84% For example, let's say you have an open ball in R2 now. 00:07:07.530 --> 00:07:13.810 align:middle line:84% And you take just any point outside the ball. 00:07:13.810 --> 00:07:21.900 align:middle line:84% Then the minimum-- now, let's see. 00:07:21.900 --> 00:07:28.320 align:middle line:90% 00:07:28.320 --> 00:07:29.660 align:middle line:90% I've got this backwards. 00:07:29.660 --> 00:07:34.880 align:middle line:84% Let's take C to be everything outside the ball. 00:07:34.880 --> 00:07:39.530 align:middle line:84% So it's neither closed nor convex. 00:07:39.530 --> 00:07:43.460 align:middle line:84% Then of course, this will not have a minimum length because-- 00:07:43.460 --> 00:07:45.440 align:middle line:84% or this subset will not have a vector 00:07:45.440 --> 00:07:48.620 align:middle line:84% that has minimum length because the minimum length will 00:07:48.620 --> 00:07:50.240 align:middle line:90% want to be on this-- 00:07:50.240 --> 00:07:53.210 align:middle line:90% 00:07:53.210 --> 00:07:55.430 align:middle line:84% let's say that's radius 1-- on this circle of radius 00:07:55.430 --> 00:08:06.170 align:middle line:84% 1, which is not included in the set C. 00:08:06.170 --> 00:08:09.020 align:middle line:84% So that one's taken away both conditions-- 00:08:09.020 --> 00:08:10.730 align:middle line:90% that C is closed and convex. 00:08:10.730 --> 00:08:15.890 align:middle line:84% Of course, you can do something where it's just convex, 00:08:15.890 --> 00:08:19.490 align:middle line:84% but not closed-- for example, take a square now in R2. 00:08:19.490 --> 00:08:26.120 align:middle line:84% Then let's say this is minus 1, 1. 00:08:26.120 --> 00:08:28.850 align:middle line:90% This is corners at 1, 1. 00:08:28.850 --> 00:08:30.140 align:middle line:90% And there's the origin. 00:08:30.140 --> 00:08:32.900 align:middle line:84% Then the vector of minimum length 00:08:32.900 --> 00:08:38.900 align:middle line:84% will want to be right here at the point 0, 1. 00:08:38.900 --> 00:08:42.140 align:middle line:84% But that's not in C. So you can also 00:08:42.140 --> 00:08:48.020 align:middle line:84% show that for C given by this rectangle missing a spot, 00:08:48.020 --> 00:08:50.100 align:middle line:90% it will not have a minimizer. 00:08:50.100 --> 00:08:51.830 align:middle line:84% And then you can play with this and find 00:08:51.830 --> 00:08:55.490 align:middle line:84% a closed subset that is not convex that doesn't 00:08:55.490 --> 00:08:57.680 align:middle line:90% have a unique minimizer. 00:08:57.680 --> 00:09:00.690 align:middle line:84% So these conditions are necessary for this theorem 00:09:00.690 --> 00:09:01.190 align:middle line:90% to hold. 00:09:01.190 --> 00:09:07.750 align:middle line:84% It's not just me giving you a lesser-- 00:09:07.750 --> 00:09:10.018 align:middle line:84% or a theorem that's with assumptions that are 00:09:10.018 --> 00:09:11.185 align:middle line:90% much stronger than you need. 00:09:11.185 --> 00:09:15.470 align:middle line:90% 00:09:15.470 --> 00:09:23.670 align:middle line:90% OK, so what's the proof? 00:09:23.670 --> 00:09:30.575 align:middle line:84% So this infimum-- so let me recall just real quick that-- 00:09:30.575 --> 00:09:33.250 align:middle line:90% 00:09:33.250 --> 00:09:38.440 align:middle line:84% so something, let's say a, is equal to the infiumum of a set 00:09:38.440 --> 00:09:42.970 align:middle line:84% S, where S is a subset of the real numbers if and only 00:09:42.970 --> 00:09:45.835 align:middle line:90% if two things-- 00:09:45.835 --> 00:09:56.050 align:middle line:84% a is a lower bound for S and there exists a sequence 00:09:56.050 --> 00:10:05.110 align:middle line:84% Sn in S such that Sn converged to a. 00:10:05.110 --> 00:10:10.170 align:middle line:84% So this should have been covered in 18.100. 00:10:10.170 --> 00:10:13.620 align:middle line:90% So let's call this number d. 00:10:13.620 --> 00:10:21.390 align:middle line:84% So let d be the infimum of u, which 00:10:21.390 --> 00:10:26.730 align:middle line:84% we know exists because norms are bounded below by 0. 00:10:26.730 --> 00:10:30.840 align:middle line:84% So this set of norms where u ranges over C 00:10:30.840 --> 00:10:33.390 align:middle line:84% is bounded below by 0, so the infimum exists. 00:10:33.390 --> 00:10:36.480 align:middle line:90% OK, so let d be that. 00:10:36.480 --> 00:10:42.300 align:middle line:84% Then there exists a sequence un of elements 00:10:42.300 --> 00:10:49.610 align:middle line:84% in C such that the norms of these un's converge to d. 00:10:49.610 --> 00:10:56.150 align:middle line:90% 00:10:56.150 --> 00:11:03.245 align:middle line:84% So I now make the claim that this sequence-- 00:11:03.245 --> 00:11:04.910 align:middle line:84% so what we're trying to do is come up 00:11:04.910 --> 00:11:11.570 align:middle line:84% with some element in C that achieves this d, 00:11:11.570 --> 00:11:17.750 align:middle line:84% so that the norm of v is equal to this d. 00:11:17.750 --> 00:11:23.050 align:middle line:84% So I now claim that this sequence is, in fact, Cauchy. 00:11:23.050 --> 00:11:35.190 align:middle line:90% 00:11:35.190 --> 00:11:40.280 align:middle line:84% And you'll see how we use the Hilbert space structure of-- 00:11:40.280 --> 00:11:42.140 align:middle line:84% or the inner product structure that 00:11:42.140 --> 00:11:43.700 align:middle line:90% comes with the Hilbert space. 00:11:43.700 --> 00:11:47.930 align:middle line:84% So we'll just do this the old fashioned way. 00:11:47.930 --> 00:11:49.440 align:middle line:90% Let epsilon be positive. 00:11:49.440 --> 00:11:52.220 align:middle line:84% We now need to find a capital N so 00:11:52.220 --> 00:11:54.950 align:middle line:84% that the difference between un and um 00:11:54.950 --> 00:11:57.200 align:middle line:84% less than epsilon in norm for all n 00:11:57.200 --> 00:11:59.840 align:middle line:84% and m bigger than or equal to capital N. 00:11:59.840 --> 00:12:07.490 align:middle line:84% So since the norm of un is converging to d, 00:12:07.490 --> 00:12:10.940 align:middle line:84% there exists a natural number N such 00:12:10.940 --> 00:12:19.490 align:middle line:84% that for all n bigger than or equal to N, I have that 2 un 00:12:19.490 --> 00:12:29.770 align:middle line:84% squared is less than 2 d squared plus epsilon squared over 2. 00:12:29.770 --> 00:12:34.760 align:middle line:90% 00:12:34.760 --> 00:12:37.430 align:middle line:84% So this thing here is converging to 2 times 00:12:37.430 --> 00:12:39.450 align:middle line:90% d squared as n goes to infinity. 00:12:39.450 --> 00:12:43.700 align:middle line:84% So if I perturb 2 d squared by a little bit, 00:12:43.700 --> 00:12:46.310 align:middle line:84% then since this is converging to 2 d squared, 00:12:46.310 --> 00:12:51.230 align:middle line:84% that will be less than this for insufficiently large. 00:12:51.230 --> 00:12:53.645 align:middle line:84% Now, I claim this capital N works. 00:12:53.645 --> 00:12:56.810 align:middle line:90% 00:12:56.810 --> 00:13:00.710 align:middle line:84% And for all nm bigger than or equal to capital 00:13:00.710 --> 00:13:09.280 align:middle line:90% N, if I take the norm-- 00:13:09.280 --> 00:13:12.520 align:middle line:84% now, I'm going to use this parallelogram law 00:13:12.520 --> 00:13:18.220 align:middle line:84% that I have for norms, which crucially relies on the Hilbert 00:13:18.220 --> 00:13:19.975 align:middle line:90% space property. 00:13:19.975 --> 00:13:23.400 align:middle line:90% 00:13:23.400 --> 00:13:28.080 align:middle line:84% So this I can write as 2 norm u squared 00:13:28.080 --> 00:13:41.410 align:middle line:84% plus 2 norm um squared minus 4 times un plus um over 2 00:13:41.410 --> 00:13:41.910 align:middle line:90% squared. 00:13:41.910 --> 00:13:45.390 align:middle line:90% 00:13:45.390 --> 00:13:47.880 align:middle line:90% Now, what's the point? 00:13:47.880 --> 00:13:51.960 align:middle line:84% Now, un plus um divided by 2, that's, 00:13:51.960 --> 00:13:56.850 align:middle line:84% if you like, t equals 1/2 in condition C. Now, 00:13:56.850 --> 00:14:04.030 align:middle line:84% since un and um are in C, un plus um over 2 is in C as well, 00:14:04.030 --> 00:14:04.530 align:middle line:90% right? 00:14:04.530 --> 00:14:10.530 align:middle line:90% So this thing is in C. 00:14:10.530 --> 00:14:15.210 align:middle line:84% And therefore, its norm squared is bigger than 00:14:15.210 --> 00:14:16.840 align:middle line:90% or equal to d squared. 00:14:16.840 --> 00:14:18.690 align:middle line:84% So when it gets hit with a negative, 00:14:18.690 --> 00:14:20.110 align:middle line:90% that switches the inequality. 00:14:20.110 --> 00:14:23.310 align:middle line:84% This norm squared is less than or equal to-- 00:14:23.310 --> 00:14:27.300 align:middle line:84% or minus 4 this norm squared is less than or equal to minus 4 00:14:27.300 --> 00:14:28.860 align:middle line:90% d squared. 00:14:28.860 --> 00:14:34.440 align:middle line:84% So this is less than or equal to 2 un squared 00:14:34.440 --> 00:14:41.130 align:middle line:84% plus 2 um squared minus 4 d squared-- again, 00:14:41.130 --> 00:14:46.710 align:middle line:84% because d is a lower bound for the norms of all elements in C 00:14:46.710 --> 00:14:51.600 align:middle line:84% and this element is in C. And this minus sign flips, 00:14:51.600 --> 00:14:52.440 align:middle line:90% all right? 00:14:52.440 --> 00:14:55.620 align:middle line:84% And now, based on how we've chosen capital N, 00:14:55.620 --> 00:15:00.400 align:middle line:84% we've chosen it so that we have this inequality here. 00:15:00.400 --> 00:15:05.730 align:middle line:84% I get this is less than 2 d squared plus epsilon squared 00:15:05.730 --> 00:15:09.540 align:middle line:84% over 2 from the first one plus 2 d 00:15:09.540 --> 00:15:13.770 align:middle line:84% squared plus epsilon squared over 2 from the second one, 00:15:13.770 --> 00:15:17.790 align:middle line:84% and then minus d squared from this last one. 00:15:17.790 --> 00:15:20.830 align:middle line:90% And that equals epsilon squared. 00:15:20.830 --> 00:15:23.760 align:middle line:84% So for all nm bigger than or equal to capital N, 00:15:23.760 --> 00:15:28.900 align:middle line:84% the norm squared of un minus um is less than epsilon squared. 00:15:28.900 --> 00:15:31.142 align:middle line:84% And therefore, that proves the claim 00:15:31.142 --> 00:15:32.350 align:middle line:90% that this sequence is Cauchy. 00:15:32.350 --> 00:15:37.120 align:middle line:90% 00:15:37.120 --> 00:15:39.190 align:middle line:84% Now, since this sequence is Cauchy 00:15:39.190 --> 00:15:44.680 align:middle line:84% and we're in a Hilbert space, a complete inner product space, 00:15:44.680 --> 00:15:57.880 align:middle line:84% there's a limit such that un converges to v. Now, 00:15:57.880 --> 00:15:58.750 align:middle line:90% C is closed. 00:15:58.750 --> 00:16:07.570 align:middle line:84% So v is also in H. Since C is closed, 00:16:07.570 --> 00:16:11.510 align:middle line:84% it contains all limits, all subsequential limits. 00:16:11.510 --> 00:16:15.580 align:middle line:84% So C is closed is equivalent to for every sequence converging 00:16:15.580 --> 00:16:18.760 align:middle line:84% to something, that something has to be in the set. 00:16:18.760 --> 00:16:24.880 align:middle line:90% v is in C. 00:16:24.880 --> 00:16:28.540 align:middle line:84% Finally, since the un's converge to v, 00:16:28.540 --> 00:16:33.880 align:middle line:84% we have v norm is equal to the limit as n goes to infinity 00:16:33.880 --> 00:16:35.770 align:middle line:90% of the norms of the un's. 00:16:35.770 --> 00:16:38.620 align:middle line:84% And remember, where did we come up with these un's? 00:16:38.620 --> 00:16:40.450 align:middle line:84% They are supposed to be-- their norms are 00:16:40.450 --> 00:16:43.480 align:middle line:90% supposed to be converging to d. 00:16:43.480 --> 00:16:51.823 align:middle line:84% So v is an element in C whose norm gives me d. 00:16:51.823 --> 00:16:53.740 align:middle line:84% Now, there's one last statement in the theorem 00:16:53.740 --> 00:16:54.940 align:middle line:90% that v is unique. 00:16:54.940 --> 00:16:56.300 align:middle line:90% There can't be two of them. 00:16:56.300 --> 00:17:00.020 align:middle line:84% And it follows from a similar argument that we made here. 00:17:00.020 --> 00:17:05.020 align:middle line:84% So what we've shown so far is that there exists 00:17:05.020 --> 00:17:11.359 align:middle line:90% a v in C whose norm gives me d. 00:17:11.359 --> 00:17:16.010 align:middle line:84% Now, we claim that there can't be more than one. 00:17:16.010 --> 00:17:24.119 align:middle line:84% Suppose v and v-bar are in C and both of their norms 00:17:24.119 --> 00:17:29.730 align:middle line:84% give me d, this infimum of norms over C. Now, 00:17:29.730 --> 00:17:32.820 align:middle line:84% I use the parallelogram law again. 00:17:32.820 --> 00:17:36.960 align:middle line:84% I get that norm of v minus v-bar squared, 00:17:36.960 --> 00:17:44.640 align:middle line:84% this is equal 2 norm squared of v plus norm of v-bar squared 00:17:44.640 --> 00:17:46.590 align:middle line:90% with a 2-- 00:17:46.590 --> 00:17:55.725 align:middle line:84% plus 2-- minus 4 norm v plus v-bar squared. 00:17:55.725 --> 00:17:58.970 align:middle line:90% 00:17:58.970 --> 00:18:01.640 align:middle line:90% Now, this equals d squared. 00:18:01.640 --> 00:18:03.030 align:middle line:90% This equals d squared. 00:18:03.030 --> 00:18:10.410 align:middle line:84% So this combines and gives me 4 d squared minus the same thing. 00:18:10.410 --> 00:18:16.340 align:middle line:84% And again, since C is convex in both v and v-bar are in C-- 00:18:16.340 --> 00:18:18.740 align:middle line:84% v-bar does not mean complex conjugate. 00:18:18.740 --> 00:18:21.200 align:middle line:84% It just means something other than v. 00:18:21.200 --> 00:18:24.830 align:middle line:84% So since both of these elements are in C, their midpoint, which 00:18:24.830 --> 00:18:28.520 align:middle line:84% is, again t equals 1/2 if you like in condition 00:18:28.520 --> 00:18:31.580 align:middle line:84% C for this theorem, this is also in C. 00:18:31.580 --> 00:18:35.810 align:middle line:84% And since d is the smallest of all the norms, 00:18:35.810 --> 00:18:38.330 align:middle line:84% the norm of the norm squared has to be bigger 00:18:38.330 --> 00:18:40.380 align:middle line:90% than or equal to d squared. 00:18:40.380 --> 00:18:43.350 align:middle line:84% And when I hit a minus sign, that flips the inequality. 00:18:43.350 --> 00:18:47.510 align:middle line:84% So that's less than or equal to 4 d squared minus 4 00:18:47.510 --> 00:18:49.910 align:middle line:90% d squared equals 0. 00:18:49.910 --> 00:18:55.460 align:middle line:84% And therefore, this norm squared has to be 0, i.e. 00:18:55.460 --> 00:18:56.180 align:middle line:90% v equals v-bar. 00:18:56.180 --> 00:18:59.020 align:middle line:90% 00:18:59.020 --> 00:19:04.480 align:middle line:84% Now, this simple theorem has important consequences which 00:19:04.480 --> 00:19:06.640 align:middle line:90% we're going to obtain from it. 00:19:06.640 --> 00:19:13.880 align:middle line:90% 00:19:13.880 --> 00:19:20.690 align:middle line:84% So the first application of this theorem is going to be to-- 00:19:20.690 --> 00:19:24.830 align:middle line:84% a way to always decompose a Hilbert space 00:19:24.830 --> 00:19:29.450 align:middle line:84% if we're given a closed linear subspace, which you're 00:19:29.450 --> 00:19:36.070 align:middle line:84% kind of used to if, you're working in Rn or Cn, 00:19:36.070 --> 00:19:38.560 align:middle line:84% but we haven't touched on yet for Hilbert spaces. 00:19:38.560 --> 00:19:41.950 align:middle line:84% And there's a reason, because we didn't have the technology yet. 00:19:41.950 --> 00:19:49.820 align:middle line:84% But now, let's discuss orthocomplements. 00:19:49.820 --> 00:19:55.630 align:middle line:90% 00:19:55.630 --> 00:19:56.680 align:middle line:90% So let's see. 00:19:56.680 --> 00:19:58.300 align:middle line:84% OK, I actually spelled it correctly. 00:19:58.300 --> 00:20:02.920 align:middle line:90% 00:20:02.920 --> 00:20:05.290 align:middle line:84% When I was at the University of Chicago, 00:20:05.290 --> 00:20:09.970 align:middle line:90% I sent my thesis to my advisor. 00:20:09.970 --> 00:20:19.790 align:middle line:84% And he had some questions about math points of the thesis 00:20:19.790 --> 00:20:21.740 align:middle line:84% that I needed to make a little bit clearer. 00:20:21.740 --> 00:20:24.950 align:middle line:84% These weren't big deals math-wise. 00:20:24.950 --> 00:20:28.640 align:middle line:84% But of course, every time I got an email from my advisor 00:20:28.640 --> 00:20:34.135 align:middle line:84% with a question about my thesis, it struck the fear of God 00:20:34.135 --> 00:20:34.635 align:middle line:90% in me. 00:20:34.635 --> 00:20:37.140 align:middle line:90% 00:20:37.140 --> 00:20:38.460 align:middle line:90% But everything was fine. 00:20:38.460 --> 00:20:39.485 align:middle line:90% Everything was fixable. 00:20:39.485 --> 00:20:40.860 align:middle line:84% And then the last comment he made 00:20:40.860 --> 00:20:44.520 align:middle line:84% was there was like 50 times throughout the paper 00:20:44.520 --> 00:20:46.560 align:middle line:90% that I needed to-- 00:20:46.560 --> 00:20:48.810 align:middle line:84% or the thesis that I needed to change 00:20:48.810 --> 00:20:50.940 align:middle line:84% complement because I kept spelling 00:20:50.940 --> 00:20:57.630 align:middle line:84% it compliment with an I, so complement 00:20:57.630 --> 00:21:00.270 align:middle line:90% meaning what's not in there. 00:21:00.270 --> 00:21:02.520 align:middle line:84% Compliment is something nice you say. 00:21:02.520 --> 00:21:06.240 align:middle line:84% And so mistakes get made sometimes. 00:21:06.240 --> 00:21:09.330 align:middle line:90% 00:21:09.330 --> 00:21:15.550 align:middle line:84% So for the complements we're interested in, 00:21:15.550 --> 00:21:16.480 align:middle line:90% we have the following. 00:21:16.480 --> 00:21:27.280 align:middle line:84% If H is a Hilbert space and W is a linear-- 00:21:27.280 --> 00:21:33.910 align:middle line:90% 00:21:33.910 --> 00:21:44.720 align:middle line:84% is a subspace, then the following set, 00:21:44.720 --> 00:21:51.230 align:middle line:84% W-perp which is equal to the set of all u in H which are 00:21:51.230 --> 00:21:53.600 align:middle line:90% orthogonal to everything in W-- 00:21:53.600 --> 00:22:00.800 align:middle line:84% so u inner product w equals 0 for all w in W-- 00:22:00.800 --> 00:22:15.455 align:middle line:84% this is a closed linear subspace of H. So that's the first part. 00:22:15.455 --> 00:22:19.840 align:middle line:90% 00:22:19.840 --> 00:22:28.980 align:middle line:84% If the subspace we started with, W, is closed then, in fact, 00:22:28.980 --> 00:22:34.490 align:middle line:84% we can write H as the direct product of-- 00:22:34.490 --> 00:22:36.950 align:middle line:84% or the direct sum, I'm sorry, of W 00:22:36.950 --> 00:22:39.357 align:middle line:90% and the orthogonal complement. 00:22:39.357 --> 00:22:40.190 align:middle line:90% What does this mean? 00:22:40.190 --> 00:22:43.280 align:middle line:84% Again, I'll recall from linear algebra what this means, i.e.-- 00:22:43.280 --> 00:22:50.618 align:middle line:90% 00:22:50.618 --> 00:22:54.190 align:middle line:84% let me make sure I don't leave anything out. 00:22:54.190 --> 00:22:54.690 align:middle line:90% i.e. 00:22:54.690 --> 00:23:07.022 align:middle line:84% for all u in H, there exists a unique w in W, 00:23:07.022 --> 00:23:15.270 align:middle line:84% w-perp in W-perp, little w, such that u 00:23:15.270 --> 00:23:18.000 align:middle line:90% is equal to w plus w-perp. 00:23:18.000 --> 00:23:40.060 align:middle line:90% 00:23:40.060 --> 00:23:44.860 align:middle line:90% So it's simple to show that-- 00:23:44.860 --> 00:24:02.940 align:middle line:84% so to see that W-perp is a subspace of H. What 00:24:02.940 --> 00:24:06.210 align:middle line:84% I have to check is that linear combinations of elements 00:24:06.210 --> 00:24:08.290 align:middle line:90% of W-perp remain in W-perp. 00:24:08.290 --> 00:24:09.450 align:middle line:90% But this is kind of clear. 00:24:09.450 --> 00:24:16.890 align:middle line:84% If u inner product w1 is equal to 0 for all u in capital W-- 00:24:16.890 --> 00:24:18.270 align:middle line:90% no, no, no, backwards. 00:24:18.270 --> 00:24:21.630 align:middle line:84% If u1 inner product w is equal to 0 00:24:21.630 --> 00:24:26.160 align:middle line:84% for all w in capital W and u2 inner product w 00:24:26.160 --> 00:24:29.310 align:middle line:84% is equal to 0 for all w in capital W, 00:24:29.310 --> 00:24:33.720 align:middle line:84% then the linear combination of u1 and u2 00:24:33.720 --> 00:24:38.070 align:middle line:84% will be orthogonal to w for all little w in capital W. 00:24:38.070 --> 00:24:41.670 align:middle line:84% So that's pretty easy to see why it's a subspace. 00:24:41.670 --> 00:24:53.040 align:middle line:84% And the only thing these two subspaces have in common 00:24:53.040 --> 00:24:58.650 align:middle line:84% is the zero vector because if I have something in W and W-perp, 00:24:58.650 --> 00:25:03.010 align:middle line:84% then it must be orthogonal to itself. 00:25:03.010 --> 00:25:05.818 align:middle line:84% And one of the conditions that we have for an inner product 00:25:05.818 --> 00:25:07.110 align:middle line:90% is that it's positive definite. 00:25:07.110 --> 00:25:09.070 align:middle line:84% If I have something orthogonal to itself, 00:25:09.070 --> 00:25:11.110 align:middle line:90% it has to be the zero vector. 00:25:11.110 --> 00:25:15.240 align:middle line:84% So that's why we get these fairly quickly. 00:25:15.240 --> 00:25:21.470 align:middle line:84% Why is W-perp-- why is this closed? 00:25:21.470 --> 00:25:24.150 align:middle line:84% This follows from the continuity of the inner product. 00:25:24.150 --> 00:25:41.780 align:middle line:84% So to show the W-perp is closed, let 00:25:41.780 --> 00:25:55.240 align:middle line:84% un be a sequence in W-perp and u in H 00:25:55.240 --> 00:25:58.140 align:middle line:90% such that un converges to u. 00:25:58.140 --> 00:26:03.050 align:middle line:84% So W-perp is closed if we show that u is, in fact, in W-perp. 00:26:03.050 --> 00:26:06.350 align:middle line:84% That's the condition of a subset of a metric space being 00:26:06.350 --> 00:26:07.040 align:middle line:90% closed, right? 00:26:07.040 --> 00:26:07.700 align:middle line:90% It's closed. 00:26:07.700 --> 00:26:09.950 align:middle line:84% I mean, there's several different ways to phrase this. 00:26:09.950 --> 00:26:14.000 align:middle line:84% But the most useful one is often that a subset is closed if 00:26:14.000 --> 00:26:16.940 align:middle line:84% and only if it's closed undertaking 00:26:16.940 --> 00:26:17.870 align:middle line:90% subsequential limits. 00:26:17.870 --> 00:26:22.160 align:middle line:84% Every limit of a sequence is contained in the set. 00:26:22.160 --> 00:26:25.430 align:middle line:90% 00:26:25.430 --> 00:26:28.160 align:middle line:84% So we need to verify that if I have a sequence of elements 00:26:28.160 --> 00:26:30.500 align:middle line:84% in W-perp converging to something, then 00:26:30.500 --> 00:26:35.140 align:middle line:90% that something has to be in C. 00:26:35.140 --> 00:26:40.390 align:middle line:84% Now, let little w be in capital W. Then 00:26:40.390 --> 00:26:43.660 align:middle line:84% the inner product of this limit, u with w, 00:26:43.660 --> 00:26:47.680 align:middle line:84% by continuity of the inner product, 00:26:47.680 --> 00:26:49.930 align:middle line:84% since the un's are converging to u, 00:26:49.930 --> 00:26:51.700 align:middle line:84% this is equal to the inner product of un 00:26:51.700 --> 00:26:54.400 align:middle line:90% with w, taking the limit. 00:26:54.400 --> 00:26:58.060 align:middle line:84% And all of these are 0 for all n. 00:26:58.060 --> 00:27:01.500 align:middle line:90% So this equals 0 . 00:27:01.500 --> 00:27:04.050 align:middle line:84% And therefore, this inner product 00:27:04.050 --> 00:27:08.850 align:middle line:84% is 0 for all w in capital W. But this is the condition 00:27:08.850 --> 00:27:12.350 align:middle line:90% that u is in W-perp. 00:27:12.350 --> 00:27:13.280 align:middle line:90% Thus, u is closed. 00:27:13.280 --> 00:27:23.440 align:middle line:90% 00:27:23.440 --> 00:27:28.025 align:middle line:90% Now, let's do the second part. 00:27:28.025 --> 00:27:29.650 align:middle line:84% So maybe I should have numbered these-- 00:27:29.650 --> 00:27:32.530 align:middle line:90% 1, 2. 00:27:32.530 --> 00:27:35.020 align:middle line:90% So this is a proof of 1. 00:27:35.020 --> 00:27:37.960 align:middle line:84% So now, we're onto proof of the statement 2. 00:27:37.960 --> 00:27:42.310 align:middle line:84% If W is closed, then H is equal to the direct product 00:27:42.310 --> 00:27:43.150 align:middle line:90% of W and W-perp. 00:27:43.150 --> 00:27:46.300 align:middle line:90% 00:27:46.300 --> 00:27:49.440 align:middle line:90% So now, suppose W is closed. 00:27:49.440 --> 00:27:55.930 align:middle line:90% 00:27:55.930 --> 00:28:00.880 align:middle line:84% If W is just the entire Hilbert space, 00:28:00.880 --> 00:28:09.730 align:middle line:84% then clearly the only thing orthogonal to everything 00:28:09.730 --> 00:28:11.020 align:middle line:90% is the zero vector. 00:28:11.020 --> 00:28:13.670 align:middle line:90% 00:28:13.670 --> 00:28:24.690 align:middle line:84% And we have the decomposition as trivially. 00:28:24.690 --> 00:28:29.880 align:middle line:90% 00:28:29.880 --> 00:28:34.410 align:middle line:84% So let's assume w is not the entire-- 00:28:34.410 --> 00:28:35.520 align:middle line:90% is not the entire space. 00:28:35.520 --> 00:28:50.150 align:middle line:90% 00:28:50.150 --> 00:28:52.880 align:middle line:84% So now, we actually have something to check. 00:28:52.880 --> 00:29:00.383 align:middle line:84% Let u be in H take away W. I can't 00:29:00.383 --> 00:29:02.550 align:middle line:84% remember if the backslash goes that way or that way. 00:29:02.550 --> 00:29:12.720 align:middle line:84% I don't mean H modulo W. I mean H take away the set W. 00:29:12.720 --> 00:29:18.520 align:middle line:84% And let's define set C seems to be-- 00:29:18.520 --> 00:29:21.550 align:middle line:90% 00:29:21.550 --> 00:29:23.950 align:middle line:84% OK, so maybe it looks like I'm just 00:29:23.950 --> 00:29:29.140 align:middle line:84% lying to you about not having to deal with H mod W. 00:29:29.140 --> 00:29:36.460 align:middle line:84% But in any case, the actual set u plus capital W, meaning set u 00:29:36.460 --> 00:29:43.880 align:middle line:84% plus little w, little w in capital W. All right? 00:29:43.880 --> 00:29:44.755 align:middle line:90% So just this set. 00:29:44.755 --> 00:29:53.350 align:middle line:90% 00:29:53.350 --> 00:29:55.780 align:middle line:84% Now, I claim that C is-- so first off, 00:29:55.780 --> 00:29:57.010 align:middle line:90% C is clearly non-empty. 00:29:57.010 --> 00:29:58.360 align:middle line:90% It contains u. 00:29:58.360 --> 00:30:10.010 align:middle line:90% 00:30:10.010 --> 00:30:12.320 align:middle line:90% So it's non-empty. 00:30:12.320 --> 00:30:16.193 align:middle line:90% I claim it's also closed. 00:30:16.193 --> 00:30:18.485 align:middle line:84% Well, first, let's do the easier bit, that it's convex. 00:30:18.485 --> 00:30:24.794 align:middle line:90% 00:30:24.794 --> 00:30:38.440 align:middle line:84% It's convex since if u plus w1 is in C, u plus w2 is in C. 00:30:38.440 --> 00:30:41.080 align:middle line:84% So these are two elements now in C. 00:30:41.080 --> 00:30:44.800 align:middle line:84% They're of the form u plus an element of capital W-- 00:30:44.800 --> 00:30:47.510 align:middle line:90% 00:30:47.510 --> 00:30:55.370 align:middle line:84% w1, w2 are in capital W and t is between 0 and 1-- 00:30:55.370 --> 00:31:12.090 align:middle line:84% then t times u plus w1 plus 1 minus t times u plus w2, 00:31:12.090 --> 00:31:15.990 align:middle line:84% this equals t times u plus 1 minus t times u 00:31:15.990 --> 00:31:23.430 align:middle line:84% just gives me back u plus t w1 plus 1 minus t w2. 00:31:23.430 --> 00:31:29.640 align:middle line:84% And now, you note that w1 and w2, they're part of a-- 00:31:29.640 --> 00:31:31.440 align:middle line:84% they're elements in a subspace capital 00:31:31.440 --> 00:31:34.110 align:middle line:84% W. And therefore, this linear combination of them 00:31:34.110 --> 00:31:40.990 align:middle line:84% is also in W. So therefore, u plus this element 00:31:40.990 --> 00:31:46.090 align:middle line:84% is in C, which is all elements of the form u 00:31:46.090 --> 00:31:49.780 align:middle line:90% plus w for w in capital W. 00:31:49.780 --> 00:31:50.980 align:middle line:90% So C is convex. 00:31:50.980 --> 00:31:54.130 align:middle line:84% Now, let's just show C is closed. 00:31:54.130 --> 00:32:03.350 align:middle line:90% 00:32:03.350 --> 00:32:08.140 align:middle line:90% Now, why is C closed? 00:32:08.140 --> 00:32:15.700 align:middle line:90% So suppose u plus wn-- 00:32:15.700 --> 00:32:20.230 align:middle line:84% so this is the sequence of elements in C. So each of these 00:32:20.230 --> 00:32:21.700 align:middle line:90% is in C-- 00:32:21.700 --> 00:32:24.760 align:middle line:90% converges to an element-- 00:32:24.760 --> 00:32:32.700 align:middle line:84% let's call it v in H. We want to show that v is in C. 00:32:32.700 --> 00:32:36.340 align:middle line:90% We want to show v is in C. 00:32:36.340 --> 00:32:43.900 align:middle line:84% Now, u plus wn converging to v implies 00:32:43.900 --> 00:32:48.745 align:middle line:90% that wn converges to v minus u. 00:32:48.745 --> 00:32:54.580 align:middle line:90% 00:32:54.580 --> 00:33:00.015 align:middle line:84% And since wn is coming from a closed subspace-- remember, 00:33:00.015 --> 00:33:01.390 align:middle line:84% this is actually what we're using 00:33:01.390 --> 00:33:03.160 align:middle line:90% the fact that W is closed. 00:33:03.160 --> 00:33:19.570 align:middle line:84% So this implies since W is closed v minus u is in W. 00:33:19.570 --> 00:33:23.560 align:middle line:90% And since v is in-- 00:33:23.560 --> 00:33:28.690 align:middle line:84% since v minus u is in W, this implies 00:33:28.690 --> 00:33:36.040 align:middle line:84% that v is equal to u plus some element w 00:33:36.040 --> 00:33:39.820 align:middle line:90% with w in capital W, i.e. 00:33:39.820 --> 00:33:43.540 align:middle line:84% v is in C. Again, capital C is the set 00:33:43.540 --> 00:33:45.970 align:middle line:84% of all elements of the form little u 00:33:45.970 --> 00:33:50.740 align:middle line:84% plus something from capital W. So we've 00:33:50.740 --> 00:33:53.560 align:middle line:84% shown that if we have a sequence converging to an element in H, 00:33:53.560 --> 00:33:55.750 align:middle line:84% then that limit must be in the set. 00:33:55.750 --> 00:33:59.050 align:middle line:90% So C is closed. 00:33:59.050 --> 00:34:02.840 align:middle line:84% All right, so let me draw a picture now of what's going on. 00:34:02.840 --> 00:34:06.325 align:middle line:84% Let's imagine that W, the subspace W, is-- 00:34:06.325 --> 00:34:13.210 align:middle line:90% 00:34:13.210 --> 00:34:21.070 align:middle line:84% so imagine H is R2 and W is just the x-axis. 00:34:21.070 --> 00:34:29.110 align:middle line:90% And u is this vector here. 00:34:29.110 --> 00:34:32.650 align:middle line:84% u plus w is now the horizontal line 00:34:32.650 --> 00:34:33.984 align:middle line:90% that goes through this point. 00:34:33.984 --> 00:34:37.330 align:middle line:90% 00:34:37.330 --> 00:34:39.390 align:middle line:90% This is u plus w. 00:34:39.390 --> 00:34:49.150 align:middle line:84% This is-- let's just let's just call it C. 00:34:49.150 --> 00:34:52.120 align:middle line:84% Now, we have the set C. We have W. 00:34:52.120 --> 00:34:58.420 align:middle line:84% We would like to break up u into an element which is parallel 00:34:58.420 --> 00:35:03.640 align:middle line:84% to W and something that's perpendicular to w, right? 00:35:03.640 --> 00:35:09.280 align:middle line:90% 00:35:09.280 --> 00:35:14.440 align:middle line:84% Now, based on this picture, what is-- 00:35:14.440 --> 00:35:18.950 align:middle line:84% let's call this element v. This is not the same v from before, 00:35:18.950 --> 00:35:31.740 align:middle line:90% so new v. 00:35:31.740 --> 00:35:35.880 align:middle line:84% We would like for this v to be perpendicular to C. 00:35:35.880 --> 00:35:38.400 align:middle line:84% And based on this picture, what would it satisfy? 00:35:38.400 --> 00:35:42.360 align:middle line:84% It would be the element of C of minimal length. 00:35:42.360 --> 00:35:45.920 align:middle line:90% 00:35:45.920 --> 00:35:49.580 align:middle line:84% So that's how we'll define v. Or if you like, 00:35:49.580 --> 00:35:51.890 align:middle line:84% that's the element that's the W-perp. 00:35:51.890 --> 00:35:53.660 align:middle line:84% It'll end up being the perpendicular part. 00:35:53.660 --> 00:35:56.780 align:middle line:90% 00:35:56.780 --> 00:36:13.850 align:middle line:84% So since C is closed and convex, there exists a unique element v 00:36:13.850 --> 00:36:21.540 align:middle line:84% in C such that the norm of v equals 00:36:21.540 --> 00:36:26.270 align:middle line:84% the infimum of all elements-- the norms of the elements 00:36:26.270 --> 00:36:29.000 align:middle line:84% and C. but I'll write it a slightly different way, w 00:36:29.000 --> 00:36:33.800 align:middle line:84% in capital W of norm of u plus w because this 00:36:33.800 --> 00:36:36.360 align:middle line:84% is all of the elements in C come in this way. 00:36:36.360 --> 00:36:39.322 align:middle line:84% So this is the way this norm of v is-- 00:36:39.322 --> 00:36:42.040 align:middle line:90% 00:36:42.040 --> 00:36:43.840 align:middle line:84% or the way this infimum can be written. 00:36:43.840 --> 00:36:47.840 align:middle line:90% 00:36:47.840 --> 00:36:52.580 align:middle line:84% So I've identified a candidate for the part 00:36:52.580 --> 00:36:54.270 align:middle line:90% that will be orthogonal. 00:36:54.270 --> 00:36:59.390 align:middle line:84% And then simply, u minus v will be the part that's in C, 00:36:59.390 --> 00:37:00.140 align:middle line:90% hopefully. 00:37:00.140 --> 00:37:02.820 align:middle line:90% 00:37:02.820 --> 00:37:04.680 align:middle line:90% And let's check this. 00:37:04.680 --> 00:37:05.310 align:middle line:90% So claim. 00:37:05.310 --> 00:37:22.380 align:middle line:90% 00:37:22.380 --> 00:37:30.860 align:middle line:84% So first off, let's note a simple thing, note that v in C 00:37:30.860 --> 00:37:37.310 align:middle line:84% implies that u minus v is in W. v 00:37:37.310 --> 00:37:39.290 align:middle line:90% is of the form u plus little w. 00:37:39.290 --> 00:37:43.730 align:middle line:84% So u minus that has to be an element of W. 00:37:43.730 --> 00:37:51.620 align:middle line:84% And we have that u is equal to u minus v plus v. 00:37:51.620 --> 00:37:55.550 align:middle line:84% So this would be the element of w. 00:37:55.550 --> 00:37:58.550 align:middle line:84% And we hope to show that this element is in W-perp. 00:37:58.550 --> 00:38:05.010 align:middle line:90% 00:38:05.010 --> 00:38:20.480 align:middle line:84% OK, so now, we claim little w is in capital W-perp. 00:38:20.480 --> 00:38:22.520 align:middle line:84% Now, how we do this is what's called, 00:38:22.520 --> 00:38:28.040 align:middle line:84% I guess, a variational argument or Euler-Lagrange equations 00:38:28.040 --> 00:38:28.740 align:middle line:90% argument. 00:38:28.740 --> 00:38:31.940 align:middle line:84% But in any case, something is the infimum 00:38:31.940 --> 00:38:38.330 align:middle line:84% of this if and only if v satisfies certain equations-- 00:38:38.330 --> 00:38:41.840 align:middle line:84% or not if not only if, but v being the infimum of this 00:38:41.840 --> 00:38:44.870 align:middle line:84% implies that v satisfies certain equations. 00:38:44.870 --> 00:38:48.050 align:middle line:84% If you've taken classical mechanics, 00:38:48.050 --> 00:38:51.620 align:middle line:84% those equations end up being the Euler-Lagrange equations. 00:38:51.620 --> 00:38:56.570 align:middle line:84% But anyways, I claim v is in W-perp. 00:38:56.570 --> 00:39:02.600 align:middle line:84% So let w be in capital W. We want to show 00:39:02.600 --> 00:39:06.500 align:middle line:90% v inner product w equals 0. 00:39:06.500 --> 00:39:10.295 align:middle line:90% Let f of t be the norm of u-- 00:39:10.295 --> 00:39:13.340 align:middle line:90% 00:39:13.340 --> 00:39:18.830 align:middle line:84% norm of, sorry, v plus tw squared, which 00:39:18.830 --> 00:39:21.110 align:middle line:90% this is just a polynomial in t. 00:39:21.110 --> 00:39:25.880 align:middle line:84% This is equal to norm v squared plus t 00:39:25.880 --> 00:39:37.550 align:middle line:84% squared norm w squared plus 2 real part of 2t real part vw. 00:39:37.550 --> 00:39:39.000 align:middle line:90% So it's just a polynomial. 00:39:39.000 --> 00:39:42.180 align:middle line:90% Now, what do we know? 00:39:42.180 --> 00:39:57.450 align:middle line:84% So then f of t has a minimum at t equals 0 because for each t, 00:39:57.450 --> 00:40:02.970 align:middle line:84% this is an element of capital C. And the norm of everything 00:40:02.970 --> 00:40:05.910 align:middle line:84% in capital C is minimized exactly 00:40:05.910 --> 00:40:09.600 align:middle line:90% at v, which is t equals 0. 00:40:09.600 --> 00:40:12.060 align:middle line:84% So this has a minimum at t equals 00:40:12.060 --> 00:40:18.630 align:middle line:84% 0, which implies f-prime evaluated at t equals 0 is 0. 00:40:18.630 --> 00:40:22.920 align:middle line:84% And therefore, if I take the derivative of f-prime 00:40:22.920 --> 00:40:25.680 align:middle line:84% of t and set t equal to 0, then I just 00:40:25.680 --> 00:40:30.000 align:middle line:84% pick up twice the real part of vw equaling 0. 00:40:30.000 --> 00:40:35.340 align:middle line:84% And therefore, I get the real part of vw equals 0. 00:40:35.340 --> 00:40:39.170 align:middle line:90% 00:40:39.170 --> 00:40:43.610 align:middle line:84% So I got that the real part of the inner product is 0. 00:40:43.610 --> 00:40:49.780 align:middle line:84% Now, it's not-- so what we can do 00:40:49.780 --> 00:41:05.560 align:middle line:84% is then repeat the previous argument with iw, i times w 00:41:05.560 --> 00:41:16.910 align:middle line:84% in place of w to get that the real part of v inner product 00:41:16.910 --> 00:41:23.210 align:middle line:84% iw, which is equal to, in fact, the imaginary part of vw, 00:41:23.210 --> 00:41:24.950 align:middle line:90% equals 0. 00:41:24.950 --> 00:41:29.180 align:middle line:84% And therefore, the inner product of v with w 00:41:29.180 --> 00:41:33.470 align:middle line:84% is 0, since its real part and imaginary part equals 0. 00:41:33.470 --> 00:41:43.030 align:middle line:84% And therefore, vw equals 0 and v is 00:41:43.030 --> 00:41:45.848 align:middle line:90% in the orthogonal complement. 00:41:45.848 --> 00:41:49.440 align:middle line:90% 00:41:49.440 --> 00:41:53.580 align:middle line:84% And so v is in the orthogonal complement 00:41:53.580 --> 00:41:55.680 align:middle line:84% and u can be written as something 00:41:55.680 --> 00:42:00.070 align:middle line:90% in W plus something in W-perp. 00:42:00.070 --> 00:42:06.430 align:middle line:90% 00:42:06.430 --> 00:42:09.520 align:middle line:90% Now, why is this? 00:42:09.520 --> 00:42:13.270 align:middle line:84% So let's take just two seconds to say why 00:42:13.270 --> 00:42:15.730 align:middle line:90% this decomposition is unique. 00:42:15.730 --> 00:42:20.290 align:middle line:84% It's unique because the only thing the two subspaces have 00:42:20.290 --> 00:42:22.000 align:middle line:90% in common is the zero vector. 00:42:22.000 --> 00:42:25.094 align:middle line:90% 00:42:25.094 --> 00:42:35.180 align:middle line:84% OK, so if I have u is equal to two different decompositions-- 00:42:35.180 --> 00:42:42.500 align:middle line:84% w1 plus w1-perp equals w2 plus w2-perp, 00:42:42.500 --> 00:42:44.780 align:middle line:84% where each of these is in capital W, 00:42:44.780 --> 00:42:47.120 align:middle line:84% each of these is in the orthogonal complement 00:42:47.120 --> 00:42:49.610 align:middle line:90% of capital W-- 00:42:49.610 --> 00:42:54.860 align:middle line:84% I didn't say-- actually, that's the terminology I'm using, 00:42:54.860 --> 00:43:00.260 align:middle line:84% but w-perp I'll call the orthogonal complement-- 00:43:00.260 --> 00:43:05.240 align:middle line:84% then this implies that w2 minus w1 is 00:43:05.240 --> 00:43:08.810 align:middle line:90% equal to w1-perp minus w2-perp. 00:43:08.810 --> 00:43:14.060 align:middle line:84% And this is in W. This is in W-perp. 00:43:14.060 --> 00:43:19.100 align:middle line:84% And therefore, since the only thing in W and W-perp is 0, 00:43:19.100 --> 00:43:22.820 align:middle line:84% that implies that the left and right side have to be 0. 00:43:22.820 --> 00:43:26.290 align:middle line:90% And therefore, w equals w1. 00:43:26.290 --> 00:43:29.260 align:middle line:90% w1-perp equals w2-perp. 00:43:29.260 --> 00:43:34.120 align:middle line:84% And that gives us the uniqueness of the decomposition. 00:43:34.120 --> 00:43:49.610 align:middle line:90% 00:43:49.610 --> 00:43:51.110 align:middle line:90% So it's in the assignment-- 00:43:51.110 --> 00:43:54.098 align:middle line:90% 00:43:54.098 --> 00:43:55.640 align:middle line:84% I should say the optional assignment. 00:43:55.640 --> 00:44:10.450 align:middle line:90% But this subspace, then-- 00:44:10.450 --> 00:44:15.190 align:middle line:84% so I have a subspace W. I can take its orthogonal complement. 00:44:15.190 --> 00:44:18.130 align:middle line:84% If it's closed, then H is equal to w plus w-perp. 00:44:18.130 --> 00:44:21.148 align:middle line:84% But if I just have an arbitrary subspace 00:44:21.148 --> 00:44:22.690 align:middle line:84% and I take its orthogonal complement, 00:44:22.690 --> 00:44:24.773 align:middle line:84% I can then take the orthogonal complement of that. 00:44:24.773 --> 00:44:25.850 align:middle line:90% What do I get? 00:44:25.850 --> 00:44:29.530 align:middle line:84% Well, the orthogonal complement of a set is always closed. 00:44:29.530 --> 00:44:33.250 align:middle line:84% So I may not get back the actual subspace again. 00:44:33.250 --> 00:44:35.050 align:middle line:90% But I will get back its closure. 00:44:35.050 --> 00:44:37.330 align:middle line:84% So the closure of W, which you can 00:44:37.330 --> 00:44:40.660 align:middle line:84% check is, again, a subspace, is equal 00:44:40.660 --> 00:44:43.540 align:middle line:84% to the orthogonal complement of the orthogonal complement. 00:44:43.540 --> 00:44:45.920 align:middle line:84% So in particular, if W is closed, 00:44:45.920 --> 00:44:49.150 align:middle line:84% then the orthogonal complement of the orthogonal complement 00:44:49.150 --> 00:44:51.680 align:middle line:90% is the set again. 00:44:51.680 --> 00:44:55.290 align:middle line:84% So this is in the optional subspace-- 00:44:55.290 --> 00:44:58.050 align:middle line:90% 00:44:58.050 --> 00:45:04.000 align:middle line:84% in the optional assignment from the last week. 00:45:04.000 --> 00:45:08.700 align:middle line:90% 00:45:08.700 --> 00:45:15.090 align:middle line:84% All right, now, given a closed linear subspace, 00:45:15.090 --> 00:45:18.450 align:middle line:90% I can define an operator-- 00:45:18.450 --> 00:45:21.660 align:middle line:84% maybe it's-- so let's just call it a map for now-- 00:45:21.660 --> 00:45:30.040 align:middle line:84% that takes a vector u and spits out-- 00:45:30.040 --> 00:45:35.830 align:middle line:84% let's say the part that's in capital W. It's W-part. 00:45:35.830 --> 00:45:37.420 align:middle line:84% Now, what kind of operator is that? 00:45:37.420 --> 00:45:39.850 align:middle line:84% Or I could have said, it takes an element u 00:45:39.850 --> 00:45:46.030 align:middle line:84% and spits out the part that's in the orthogonal complement 00:45:46.030 --> 00:45:49.630 align:middle line:84% of capital W. What kind of map is that part? 00:45:49.630 --> 00:45:50.600 align:middle line:90% Or is that? 00:45:50.600 --> 00:45:53.755 align:middle line:84% So there's a very special name for that. 00:45:53.755 --> 00:45:59.500 align:middle line:90% 00:45:59.500 --> 00:46:05.625 align:middle line:84% So of course, in R2, if I have an element u-- 00:46:05.625 --> 00:46:08.240 align:middle line:90% 00:46:08.240 --> 00:46:15.090 align:middle line:90% so let's say W is the x-axis. 00:46:15.090 --> 00:46:17.290 align:middle line:90% W-perp would then be the y-axis. 00:46:17.290 --> 00:46:25.970 align:middle line:84% And then this would be the part that's 00:46:25.970 --> 00:46:32.040 align:middle line:84% in W. This vector would be the part that's in W-perp. 00:46:32.040 --> 00:46:36.690 align:middle line:84% Now, what exactly is this usually called, at least going 00:46:36.690 --> 00:46:38.535 align:middle line:90% back to your calculus days? 00:46:38.535 --> 00:46:40.650 align:middle line:84% You usually refer to it as the projection 00:46:40.650 --> 00:46:44.220 align:middle line:84% of u onto the, let's say, x-axis. 00:46:44.220 --> 00:46:46.760 align:middle line:90% 00:46:46.760 --> 00:46:49.020 align:middle line:90% But that name has-- 00:46:49.020 --> 00:46:51.650 align:middle line:84% or that word has a very specific meaning. 00:46:51.650 --> 00:46:54.770 align:middle line:90% 00:46:54.770 --> 00:46:58.820 align:middle line:84% And then we'll show that what I was just discussing-- 00:46:58.820 --> 00:47:02.210 align:middle line:84% taking u to it's W part or it's W-perp part-- 00:47:02.210 --> 00:47:04.670 align:middle line:90% is, in fact, a projection. 00:47:04.670 --> 00:47:25.820 align:middle line:84% So a founded linear operator P going from H to H 00:47:25.820 --> 00:47:45.460 align:middle line:84% is a projection if p squared equals P. So this is 00:47:45.460 --> 00:47:47.090 align:middle line:90% a new bit of terminology. 00:47:47.090 --> 00:47:51.310 align:middle line:84% So for example, it doesn't have to exactly look like this 00:47:51.310 --> 00:47:55.900 align:middle line:84% or come from this way I was describing of obtaining 00:47:55.900 --> 00:47:59.080 align:middle line:90% a map from H to W or W-perp. 00:47:59.080 --> 00:48:01.730 align:middle line:84% That's not the definition of a projection. 00:48:01.730 --> 00:48:03.490 align:middle line:84% The definition of a projection is this. 00:48:03.490 --> 00:48:05.740 align:middle line:84% So for example, taking everything to 0 00:48:05.740 --> 00:48:10.430 align:middle line:84% is certainly a projection, which I 00:48:10.430 --> 00:48:12.110 align:middle line:84% guess you could think of as projecting 00:48:12.110 --> 00:48:15.740 align:middle line:84% onto the subspace consisting only of 0. 00:48:15.740 --> 00:48:23.460 align:middle line:84% But what I want to say is that, in fact, 00:48:23.460 --> 00:48:26.850 align:middle line:84% the map that I just outlined is, in fact, 00:48:26.850 --> 00:48:29.220 align:middle line:90% a projection as defined here. 00:48:29.220 --> 00:48:52.420 align:middle line:84% So let H be a Hilbert space, as usual; W, a closed subspace. 00:48:52.420 --> 00:48:57.032 align:middle line:90% 00:48:57.032 --> 00:48:58.490 align:middle line:84% So then by the previous theorem, we 00:48:58.490 --> 00:49:02.870 align:middle line:84% have H is equal to the direct sum of W 00:49:02.870 --> 00:49:04.700 align:middle line:90% and its orthogonal complement. 00:49:04.700 --> 00:49:09.980 align:middle line:90% 00:49:09.980 --> 00:49:16.475 align:middle line:84% The map pi sub W going from H to H given by-- 00:49:16.475 --> 00:49:25.260 align:middle line:90% 00:49:25.260 --> 00:49:33.140 align:middle line:84% defined by the following-- if v is equal to w plus w-perp-- 00:49:33.140 --> 00:49:36.530 align:middle line:84% so I take an element in the Hilbert space, 00:49:36.530 --> 00:49:38.900 align:middle line:84% decompose it as a part that's in W 00:49:38.900 --> 00:49:40.640 align:middle line:90% and as a part that's in W-perp. 00:49:40.640 --> 00:49:51.435 align:middle line:84% Then the definition of this map evaluated on v is just w. 00:49:51.435 --> 00:49:54.050 align:middle line:90% 00:49:54.050 --> 00:49:58.360 align:middle line:90% This map is a projection. 00:49:58.360 --> 00:50:06.890 align:middle line:90% 00:50:06.890 --> 00:50:10.430 align:middle line:84% So we need to show that it is a bounded linear operator 00:50:10.430 --> 00:50:14.450 align:middle line:84% and its square gives you back the original map. 00:50:14.450 --> 00:50:39.540 align:middle line:90% 00:50:39.540 --> 00:50:43.905 align:middle line:84% So first, let's show this map is linear. 00:50:43.905 --> 00:50:47.890 align:middle line:90% 00:50:47.890 --> 00:50:51.305 align:middle line:84% So the first claim is that pi is linear. 00:50:51.305 --> 00:51:07.590 align:middle line:90% 00:51:07.590 --> 00:51:15.450 align:middle line:84% So if I have v1 is equal to w1 plus w1-perp, 00:51:15.450 --> 00:51:22.260 align:middle line:84% v2 is equal to w2 plus w2-perp, and I have two scalars-- 00:51:22.260 --> 00:51:26.385 align:middle line:84% lambda 1, lambda 2, complex numbers-- 00:51:26.385 --> 00:51:28.990 align:middle line:90% 00:51:28.990 --> 00:51:35.650 align:middle line:84% then lambda 1 times v1 plus lambda 2 v2, this is equal to-- 00:51:35.650 --> 00:51:41.480 align:middle line:84% multiplying out and combining, this 00:51:41.480 --> 00:51:50.000 align:middle line:84% is equal to lambda 2 w2 plus lambda 1 w1-perp plus 00:51:50.000 --> 00:51:54.800 align:middle line:90% lambda 2 times w2-perp. 00:51:54.800 --> 00:52:02.840 align:middle line:84% Now, since w1 and w2 are in W, this linear combination of them 00:52:02.840 --> 00:52:10.650 align:middle line:84% is also in W. And this part also, 00:52:10.650 --> 00:52:14.460 align:middle line:84% since that's in the orthogonal complement of W 00:52:14.460 --> 00:52:16.620 align:middle line:84% and so is that, their linear combination is also 00:52:16.620 --> 00:52:18.970 align:middle line:90% in the orthogonal complement. 00:52:18.970 --> 00:52:27.030 align:middle line:84% So the decomposition of this linear combination of v1 and v2 00:52:27.030 --> 00:52:32.100 align:middle line:84% is a linear combination of the decompositions. 00:52:32.100 --> 00:52:41.680 align:middle line:84% And therefore, by how we define this map 00:52:41.680 --> 00:52:44.320 align:middle line:84% as the part that's in W, this is equal to lambda 00:52:44.320 --> 00:52:48.280 align:middle line:90% 1 w1 plus lambda 2 w2. 00:52:48.280 --> 00:52:52.055 align:middle line:84% And w1, that's just pi sub W. That's just the projection-- 00:52:52.055 --> 00:52:53.680 align:middle line:84% I'm calling it a projection, although I 00:52:53.680 --> 00:52:56.680 align:middle line:90% haven't proved that yet-- 00:52:56.680 --> 00:53:00.070 align:middle line:84% this thing applied to v1 and w2 is this thing 00:53:00.070 --> 00:53:03.160 align:middle line:90% applied to v2 by definition. 00:53:03.160 --> 00:53:05.825 align:middle line:90% This is pi. 00:53:05.825 --> 00:53:08.680 align:middle line:90% 00:53:08.680 --> 00:53:14.550 align:middle line:90% w2 is pi by definition. 00:53:14.550 --> 00:53:25.678 align:middle line:84% So this is equal to lambda 1 v1 plus lambda 2 v2. 00:53:25.678 --> 00:53:27.095 align:middle line:84% And therefore, this map is linear. 00:53:27.095 --> 00:53:47.050 align:middle line:90% 00:53:47.050 --> 00:53:48.670 align:middle line:90% OK, so it's linear. 00:53:48.670 --> 00:53:49.585 align:middle line:90% Why is it bounded? 00:53:49.585 --> 00:53:52.830 align:middle line:90% 00:53:52.830 --> 00:54:06.660 align:middle line:84% So now, pi is bounded to v is equal to w plus w-perp. 00:54:06.660 --> 00:54:13.890 align:middle line:84% w is, again, equal to pi of v. Then because these two things 00:54:13.890 --> 00:54:17.850 align:middle line:84% are orthogonal, I get that norm of v squared 00:54:17.850 --> 00:54:22.960 align:middle line:84% is equal to norm of w plus w-perp squared. 00:54:22.960 --> 00:54:24.270 align:middle line:90% And now, what do I pick up? 00:54:24.270 --> 00:54:34.981 align:middle line:84% I pick up the norm of w squared plus norm of w-perp squared 00:54:34.981 --> 00:54:38.225 align:middle line:84% plus 2 times the real part of the inner product of w 00:54:38.225 --> 00:54:38.970 align:middle line:90% with w-perp. 00:54:38.970 --> 00:54:50.090 align:middle line:84% But that's 0, so I just get the sum of the norms. 00:54:50.090 --> 00:54:52.090 align:middle line:84% And that's bigger than or equal-- 00:54:52.090 --> 00:54:55.090 align:middle line:84% or since it's a sum of two non-negative things, 00:54:55.090 --> 00:54:58.030 align:middle line:84% that's bigger than or equal to one 00:54:58.030 --> 00:55:00.010 align:middle line:90% of them, which is w squared. 00:55:00.010 --> 00:55:10.030 align:middle line:84% Or rephrasing, since w is equal to pi applied to v, 00:55:10.030 --> 00:55:13.030 align:middle line:84% I've said that this is less than or equal to the norm of v 00:55:13.030 --> 00:55:19.130 align:middle line:84% so that pi is a bounded linear operator. 00:55:19.130 --> 00:55:28.430 align:middle line:90% 00:55:28.430 --> 00:55:30.260 align:middle line:84% In fact, what we've shown is that its norm 00:55:30.260 --> 00:55:33.300 align:middle line:90% is less than or equal to 1. 00:55:33.300 --> 00:55:36.900 align:middle line:84% OK, and finally, so the last piece that we need to check 00:55:36.900 --> 00:55:38.730 align:middle line:90% is that pi squared equals pi. 00:55:38.730 --> 00:55:40.410 align:middle line:90% But that's pretty easy to check. 00:55:40.410 --> 00:55:46.524 align:middle line:90% 00:55:46.524 --> 00:56:00.080 align:middle line:84% We simply note that if v is equal to w plus w-perp, then 00:56:00.080 --> 00:56:03.890 align:middle line:84% I need to check that I get pi w of v again. 00:56:03.890 --> 00:56:10.070 align:middle line:84% So this is equal to the part of this that's in w. 00:56:10.070 --> 00:56:14.480 align:middle line:84% Now, this is equal to w, little w. 00:56:14.480 --> 00:56:19.310 align:middle line:84% And again, this picks out the part of the element in here 00:56:19.310 --> 00:56:22.490 align:middle line:84% that's in capital W. But this is in capital W. 00:56:22.490 --> 00:56:25.820 align:middle line:90% So this is just equal to w. 00:56:25.820 --> 00:56:32.330 align:middle line:84% And this is, by definition again, equal to pi of v 00:56:32.330 --> 00:56:52.750 align:middle line:84% So we've shown that pi squared of v equals pi of v. 00:56:52.750 --> 00:56:55.850 align:middle line:84% OK, so one last application we'll 00:56:55.850 --> 00:56:59.810 align:middle line:84% do of minimizers, which is probably 00:56:59.810 --> 00:57:04.070 align:middle line:84% the most important application, which 00:57:04.070 --> 00:57:06.680 align:middle line:84% one could prove for separable Hilbert spaces 00:57:06.680 --> 00:57:12.290 align:middle line:84% based on what we know and have done so far. 00:57:12.290 --> 00:57:22.110 align:middle line:84% But this proof works also for non-separable Hilbert spaces. 00:57:22.110 --> 00:57:25.070 align:middle line:84% And OK, so what is this theorem I'm referring to? 00:57:25.070 --> 00:57:27.860 align:middle line:84% It's probably one of the most important theorems 00:57:27.860 --> 00:57:29.180 align:middle line:90% in all of this business. 00:57:29.180 --> 00:57:31.025 align:middle line:84% It's the Riesz representation theorem. 00:57:31.025 --> 00:57:41.450 align:middle line:90% 00:57:41.450 --> 00:57:43.710 align:middle line:84% OK, so the only category theory I know 00:57:43.710 --> 00:57:45.030 align:middle line:90% is the Baire category theorem. 00:57:45.030 --> 00:57:46.890 align:middle line:84% The only representation theory I remember 00:57:46.890 --> 00:57:52.020 align:middle line:84% is the Riesz representation theorem, which tells us we 00:57:52.020 --> 00:57:55.860 align:middle line:84% can identify a Hilbert space, the dual of a Hilbert space 00:57:55.860 --> 00:57:58.960 align:middle line:90% with the Hilbert space itself. 00:57:58.960 --> 00:58:20.820 align:middle line:84% So if H is a Hilbert space, then for all f in the dual, 00:58:20.820 --> 00:58:32.713 align:middle line:84% there exists a unique element v in H such that f of u-- 00:58:32.713 --> 00:58:34.130 align:middle line:84% so this is an element in the dual, 00:58:34.130 --> 00:58:37.190 align:middle line:84% meaning it takes u to a complex number-- 00:58:37.190 --> 00:58:39.320 align:middle line:84% and is linear in u, you can write it 00:58:39.320 --> 00:58:48.780 align:middle line:84% as u inner product with this element v All right, 00:58:48.780 --> 00:58:55.260 align:middle line:84% so every element of the dual can be 00:58:55.260 --> 00:59:01.210 align:middle line:84% realized as the inner product with a vector. 00:59:01.210 --> 00:59:04.793 align:middle line:84% Now, we saw this in a certain form already in, 00:59:04.793 --> 00:59:06.960 align:middle line:84% I think, it was maybe the first or second assignment 00:59:06.960 --> 00:59:09.240 align:middle line:90% when you computed the-- 00:59:09.240 --> 00:59:12.540 align:middle line:84% or proved that the dual space of little lp 00:59:12.540 --> 00:59:16.620 align:middle line:84% is a little lq, where 1 over p plus 1 over q equals 1. 00:59:16.620 --> 00:59:20.040 align:middle line:90% When p equals 2, q is 2. 00:59:20.040 --> 00:59:25.830 align:middle line:84% So you saw that the dual space could be identified with itself 00:59:25.830 --> 00:59:30.720 align:middle line:84% when we're looking at little l2, which is the only Hilbert space 00:59:30.720 --> 00:59:34.290 align:middle line:90% out of all the little lp's. 00:59:34.290 --> 00:59:39.810 align:middle line:84% And remember, how we proved that little lq was dual to little lp 00:59:39.810 --> 00:59:45.030 align:middle line:84% was via pairing between the two, which specifically 00:59:45.030 --> 00:59:49.155 align:middle line:84% was the sum of the sequences multiplied entry by entry. 00:59:49.155 --> 00:59:52.960 align:middle line:90% 00:59:52.960 --> 00:59:55.880 align:middle line:84% And now, so this theorem says that wasn't a fluke. 00:59:55.880 --> 00:59:57.880 align:middle line:84% This is, in fact, true for every Hilbert space, 00:59:57.880 --> 01:00:01.540 align:middle line:84% that the dual can be identified with the space itself 01:00:01.540 --> 01:00:06.280 align:middle line:84% in this canonical way, where every element of the dual 01:00:06.280 --> 01:00:08.950 align:middle line:84% can be realized as taking the inner product with a vector. 01:00:08.950 --> 01:00:16.490 align:middle line:90% 01:00:16.490 --> 01:00:17.380 align:middle line:90% So for the proof-- 01:00:17.380 --> 01:00:24.010 align:middle line:90% 01:00:24.010 --> 01:00:32.020 align:middle line:84% so first off, we note that v is unique. 01:00:32.020 --> 01:00:40.140 align:middle line:84% If such a v exists, it's unique, since if f of u is equal to u, 01:00:40.140 --> 01:00:48.820 align:middle line:84% inner product v is equal to u, inner product v-tilde for all u 01:00:48.820 --> 01:00:55.300 align:middle line:84% implies that u inner product v minus v-tilde 01:00:55.300 --> 01:01:02.740 align:middle line:84% equals 0 for all u in H, which, by setting 01:01:02.740 --> 01:01:10.330 align:middle line:84% u equal to v minus v-tilde, tells me v equals v-tilde. 01:01:10.330 --> 01:01:13.120 align:middle line:84% All right, so all we need to do is come up with-- 01:01:13.120 --> 01:01:16.810 align:middle line:84% given an element of the dual, come up with a vector 01:01:16.810 --> 01:01:23.325 align:middle line:84% that whenever I stick a u into the dual vector, 01:01:23.325 --> 01:01:24.700 align:middle line:84% it's equal to this inner product. 01:01:24.700 --> 01:01:27.380 align:middle line:90% 01:01:27.380 --> 01:01:30.835 align:middle line:84% OK, so the easiest case to deal with is, of course, f equals 0. 01:01:30.835 --> 01:01:33.700 align:middle line:90% 01:01:33.700 --> 01:01:37.480 align:middle line:84% In other words, it maps u to 0 no matter what u is. 01:01:37.480 --> 01:01:41.440 align:middle line:84% We just choose this vector that-- 01:01:41.440 --> 01:01:43.300 align:middle line:84% we just choose this vector v to be 0. 01:01:43.300 --> 01:01:48.130 align:middle line:84% Then we'll always have f of u equal to u inner product 01:01:48.130 --> 01:01:49.045 align:middle line:90% with the 0 vector. 01:01:49.045 --> 01:01:54.280 align:middle line:90% 01:01:54.280 --> 01:02:05.835 align:middle line:84% So suppose now f does not equal 0. 01:02:05.835 --> 01:02:08.530 align:middle line:90% 01:02:08.530 --> 01:02:18.400 align:middle line:84% Then there exists, let's say, a u1 in H such that f of u1 01:02:18.400 --> 01:02:21.160 align:middle line:90% does not equal 0. 01:02:21.160 --> 01:02:30.200 align:middle line:84% Then if I take u0 to be u1 over f of u1, 01:02:30.200 --> 01:02:34.730 align:middle line:84% this implies that f of u0 equals 1. 01:02:34.730 --> 01:02:40.960 align:middle line:90% 01:02:40.960 --> 01:02:52.270 align:middle line:84% OK, so now, let C b e the set of all elements 01:02:52.270 --> 01:03:01.108 align:middle line:84% u in H which give me 1 when I stick it into F. Now, 01:03:01.108 --> 01:03:02.650 align:middle line:84% this set is non-empty, because I just 01:03:02.650 --> 01:03:06.910 align:middle line:84% gave you an element that when I stick it into f gives me 1. 01:03:06.910 --> 01:03:09.500 align:middle line:90% So this is non-empty. 01:03:09.500 --> 01:03:11.050 align:middle line:90% And what is this? 01:03:11.050 --> 01:03:14.830 align:middle line:84% Actually, this is the inverse image 01:03:14.830 --> 01:03:21.820 align:middle line:84% of the singleton in the set of complex numbers. 01:03:21.820 --> 01:03:23.980 align:middle line:84% Now, a singleton is a closed set. 01:03:23.980 --> 01:03:26.290 align:middle line:84% f is a continuous function, right? 01:03:26.290 --> 01:03:28.720 align:middle line:84% An element of the dual is a bounded linear map-- 01:03:28.720 --> 01:03:33.930 align:middle line:84% or bounded linear map from the Hilbert space 01:03:33.930 --> 01:03:34.958 align:middle line:90% to the complex numbers. 01:03:34.958 --> 01:03:36.250 align:middle line:90% And therefore, it's continuous. 01:03:36.250 --> 01:03:39.960 align:middle line:84% So the inverse image by a continuous function 01:03:39.960 --> 01:03:41.490 align:middle line:90% of a closed set is closed. 01:03:41.490 --> 01:03:42.615 align:middle line:90% So this is also closed. 01:03:42.615 --> 01:03:46.710 align:middle line:90% 01:03:46.710 --> 01:04:01.507 align:middle line:84% So C is that, which is a non-empty closed subset of H. 01:04:01.507 --> 01:04:02.840 align:middle line:90% You can see where this is going. 01:04:02.840 --> 01:04:07.640 align:middle line:84% Now, let's check that C is convex. 01:04:07.640 --> 01:04:11.320 align:middle line:90% That writing is a bit skewed. 01:04:11.320 --> 01:04:19.350 align:middle line:84% Now, if u1, u2 are in C, t is in 0, 1, 01:04:19.350 --> 01:04:28.770 align:middle line:84% then f of t times u1 plus 1 minus t u2, this is equal to-- 01:04:28.770 --> 01:04:30.040 align:middle line:90% now, f is linear. 01:04:30.040 --> 01:04:38.610 align:middle line:84% So this is equal to t times f of u1 plus 1 minus t f of u2. 01:04:38.610 --> 01:04:41.220 align:middle line:90% Now, this is equal to 1. 01:04:41.220 --> 01:04:44.420 align:middle line:90% That's equal to 1. 01:04:44.420 --> 01:04:45.110 align:middle line:90% So I get 1. 01:04:45.110 --> 01:04:46.370 align:middle line:90% So C is also convex. 01:04:46.370 --> 01:04:58.380 align:middle line:90% 01:04:58.380 --> 01:05:01.370 align:middle line:84% Now, you can see how the proof is going. 01:05:01.370 --> 01:05:06.540 align:middle line:90% But we might guess that-- 01:05:06.540 --> 01:05:09.910 align:middle line:84% if you're just looking at the proof, why am I doing this? 01:05:09.910 --> 01:05:13.915 align:middle line:84% Well, in fact, almost the minimum vector, 01:05:13.915 --> 01:05:15.540 align:middle line:84% or the vector with the smallest length, 01:05:15.540 --> 01:05:20.680 align:middle line:84% will be the vector that satisfies that inequality. 01:05:20.680 --> 01:05:27.990 align:middle line:84% So since C is a closed, convex, non-empty set, 01:05:27.990 --> 01:05:39.180 align:middle line:84% there exists a v0 in C such that v0 is equal to the inf 01:05:39.180 --> 01:05:46.960 align:middle line:84% of, let's say, u in C such that-- 01:05:46.960 --> 01:05:48.780 align:middle line:84% so we're looking at the norm of u. 01:05:48.780 --> 01:05:54.120 align:middle line:90% 01:05:54.120 --> 01:06:05.810 align:middle line:84% So let v equal norm of v0 over norm squared of v0. 01:06:05.810 --> 01:06:10.070 align:middle line:84% So first off, note that v0 cannot be 0, right? 01:06:10.070 --> 01:06:13.280 align:middle line:84% If v0 is 0, then f of v0 is equal to 0. 01:06:13.280 --> 01:06:17.300 align:middle line:84% But v0 is in C. f of v0 has to be 1. 01:06:17.300 --> 01:06:18.170 align:middle line:90% So v0 is non-zero. 01:06:18.170 --> 01:06:23.140 align:middle line:90% 01:06:23.140 --> 01:06:24.865 align:middle line:90% And I claim this v does a job. 01:06:24.865 --> 01:06:33.570 align:middle line:90% 01:06:33.570 --> 01:06:37.410 align:middle line:84% For all-- OK, I was using u's to denote elements in C. 01:06:37.410 --> 01:06:39.420 align:middle line:90% But claim-- 01:06:39.420 --> 01:06:42.930 align:middle line:90% I'll just write does a job-- 01:06:42.930 --> 01:06:46.690 align:middle line:90% 01:06:46.690 --> 01:06:50.170 align:middle line:84% i.e. for all u in capital H, f of u 01:06:50.170 --> 01:06:55.857 align:middle line:84% is equal to u inner product with v. 01:06:55.857 --> 01:07:06.910 align:middle line:84% All right, to see that, let N be the null space 01:07:06.910 --> 01:07:11.530 align:middle line:84% of f, the set of all vectors that get sent to the number 0. 01:07:11.530 --> 01:07:19.360 align:middle line:84% So this is a set of all w in H such that f of w equals 0. 01:07:19.360 --> 01:07:24.520 align:middle line:84% This is a closed linear subspace of N-- 01:07:24.520 --> 01:07:25.295 align:middle line:90% I mean of h. 01:07:25.295 --> 01:07:28.380 align:middle line:90% 01:07:28.380 --> 01:07:31.920 align:middle line:84% Then it's not too difficult to convince yourself 01:07:31.920 --> 01:07:38.490 align:middle line:84% that I can write C as, in fact, v0 plus w, where 01:07:38.490 --> 01:07:41.625 align:middle line:90% w is in this subspace. 01:07:41.625 --> 01:07:45.030 align:middle line:90% 01:07:45.030 --> 01:07:45.780 align:middle line:90% And what is v0? 01:07:45.780 --> 01:07:50.980 align:middle line:84% Again, v0 is the infimum of the norms of all of these guys. 01:07:50.980 --> 01:08:00.230 align:middle line:84% So it's equal to the infimum over, 01:08:00.230 --> 01:08:08.770 align:middle line:84% if I'm writing C in that way, w in N norm v0 plus w. 01:08:08.770 --> 01:08:12.330 align:middle line:90% 01:08:12.330 --> 01:08:15.300 align:middle line:90% All right, why am I doing this? 01:08:15.300 --> 01:08:17.609 align:middle line:84% Because of the argument we gave a minute ago, 01:08:17.609 --> 01:08:23.399 align:middle line:84% I can conclude that v0 is orthogonal to everything in n. 01:08:23.399 --> 01:08:27.359 align:middle line:84% So by the previous argument, I mean, look back at-- 01:08:27.359 --> 01:08:28.859 align:middle line:90% is it up there or-- 01:08:28.859 --> 01:08:31.380 align:middle line:84% no, it was on this board and I erased it. 01:08:31.380 --> 01:08:35.250 align:middle line:84% But where we define this function 01:08:35.250 --> 01:08:37.970 align:middle line:84% f of t equal to-- in this case, it 01:08:37.970 --> 01:08:40.890 align:middle line:90% would be v0 plus t w squared. 01:08:40.890 --> 01:08:43.109 align:middle line:84% Since that has a minimum at t equals 0, 01:08:43.109 --> 01:08:46.410 align:middle line:84% we conclude that v0 is orthogonal to W. 01:08:46.410 --> 01:08:55.420 align:middle line:84% So by previous argument for when we 01:08:55.420 --> 01:08:58.779 align:middle line:84% showed that H can be written as W plus its orthogonal-- 01:08:58.779 --> 01:09:01.490 align:middle line:84% the direct sum of a closed linear subspace 01:09:01.490 --> 01:09:02.740 align:middle line:90% and its orthogonal complement. 01:09:02.740 --> 01:09:07.720 align:middle line:84% So by the previous argument from that proof, 01:09:07.720 --> 01:09:12.250 align:middle line:84% we can show that v0 is an element 01:09:12.250 --> 01:09:15.370 align:middle line:84% of the orthogonal complement of N, 01:09:15.370 --> 01:09:17.875 align:middle line:84% the set of all vectors that get sent to 0. 01:09:17.875 --> 01:09:24.887 align:middle line:90% 01:09:24.887 --> 01:09:25.845 align:middle line:90% Now, we're almost done. 01:09:25.845 --> 01:09:30.060 align:middle line:90% 01:09:30.060 --> 01:09:36.240 align:middle line:84% So let u be in H. I want to show that f of u 01:09:36.240 --> 01:09:40.625 align:middle line:90% is equal to v inner product u. 01:09:40.625 --> 01:09:47.340 align:middle line:84% I just needed that little bit of fact there. 01:09:47.340 --> 01:09:57.280 align:middle line:84% So remember, then if I look at f of u minus f of u v0, 01:09:57.280 --> 01:09:59.590 align:middle line:84% this is equal to-- so f is linear, remember? 01:09:59.590 --> 01:10:00.880 align:middle line:90% So scalars pop out. 01:10:00.880 --> 01:10:03.740 align:middle line:90% This is just a complex number. 01:10:03.740 --> 01:10:11.020 align:middle line:84% So this is equal to f of u minus f of u f of v0. 01:10:11.020 --> 01:10:12.430 align:middle line:90% This equals 0, right? 01:10:12.430 --> 01:10:19.700 align:middle line:90% 01:10:19.700 --> 01:10:29.530 align:middle line:84% And therefore, u, which is equal to u minus f of u v0 01:10:29.530 --> 01:10:32.590 align:middle line:90% plus f of u v0-- 01:10:32.590 --> 01:10:38.170 align:middle line:84% now, this thing here, because f applied to it gives me 0, 01:10:38.170 --> 01:10:45.010 align:middle line:84% this is in N. And this is in, again 01:10:45.010 --> 01:10:48.008 align:middle line:84% remember, N-perp, although I'm not 01:10:48.008 --> 01:10:49.300 align:middle line:90% going to use that specifically. 01:10:49.300 --> 01:10:51.700 align:middle line:84% I'm going to use the fact that v0 inner product 01:10:51.700 --> 01:10:56.790 align:middle line:90% with everything from N is 0. 01:10:56.790 --> 01:10:59.810 align:middle line:90% 01:10:59.810 --> 01:11:06.290 align:middle line:84% And therefore, if I take the inner product of u with v-- 01:11:06.290 --> 01:11:11.120 align:middle line:84% remember, v was v0 over v0 squared-- 01:11:11.120 --> 01:11:21.580 align:middle line:84% this is equal to u 1 over v0 squared inner product u v0. 01:11:21.580 --> 01:11:26.180 align:middle line:84% This is equal to 1 over norm v0 squared. 01:11:26.180 --> 01:11:32.200 align:middle line:84% Now, again, this element is in N. 01:11:32.200 --> 01:11:35.380 align:middle line:84% So when I take the inner product of this line 01:11:35.380 --> 01:11:41.410 align:middle line:84% with v0, when v0 hits this, I get 0 because it's in N. 01:11:41.410 --> 01:11:49.900 align:middle line:84% So if you like, call this-- so call this thing w, 01:11:49.900 --> 01:12:00.330 align:middle line:84% which is in N. So this is equal to w inner product v0 plus f 01:12:00.330 --> 01:12:05.820 align:middle line:90% of u v0 inner product v0. 01:12:05.820 --> 01:12:09.270 align:middle line:84% And again, this thing is in N. v0 01:12:09.270 --> 01:12:14.826 align:middle line:84% is in the orthogonal complement of N. 01:12:14.826 --> 01:12:25.480 align:middle line:84% So this inner product is 0 times f of u times 01:12:25.480 --> 01:12:30.010 align:middle line:84% v0 inner product with itself is the norm squared. 01:12:30.010 --> 01:12:31.120 align:middle line:90% And I get f of u. 01:12:31.120 --> 01:12:34.720 align:middle line:90% 01:12:34.720 --> 01:12:41.020 align:middle line:84% So I have found a vector, namely this certain minimizer 01:12:41.020 --> 01:12:43.810 align:middle line:84% over its length squared so that f of u 01:12:43.810 --> 01:12:47.500 align:middle line:84% is equal to u inner product with this vector for all u in H. 01:12:47.500 --> 01:12:49.910 align:middle line:84% And that completes the proof of the Riesz representation 01:12:49.910 --> 01:12:50.410 align:middle line:90% theorem. 01:12:50.410 --> 01:12:54.440 align:middle line:90% 01:12:54.440 --> 01:12:58.700 align:middle line:84% Now, next time, we'll talk about adjoints, 01:12:58.700 --> 01:13:03.050 align:middle line:84% which you had on an assignment at one point 01:13:03.050 --> 01:13:05.510 align:middle line:90% when we were talking about-- 01:13:05.510 --> 01:13:08.165 align:middle line:84% I think I defined the adjoint for a general Banach space. 01:13:08.165 --> 01:13:11.870 align:middle line:90% 01:13:11.870 --> 01:13:17.530 align:middle line:84% So if you have a map, let's say, from a Banach space to itself, 01:13:17.530 --> 01:13:21.140 align:middle line:84% then we define the adjoint, in a certain way, 01:13:21.140 --> 01:13:24.740 align:middle line:84% to be now a linear map that goes from the dual space 01:13:24.740 --> 01:13:27.140 align:middle line:90% to the dual space. 01:13:27.140 --> 01:13:28.940 align:middle line:84% Now, in the case of a Hilbert space, 01:13:28.940 --> 01:13:31.770 align:middle line:84% the dual space is equal to the space itself. 01:13:31.770 --> 01:13:33.800 align:middle line:84% So the adjoint for a Hilbert space 01:13:33.800 --> 01:13:37.550 align:middle line:84% will be a map, again, from the space to itself. 01:13:37.550 --> 01:13:42.900 align:middle line:84% And it will satisfy a certain identity. 01:13:42.900 --> 01:13:48.910 align:middle line:84% And we will see the connection between adjoints 01:13:48.910 --> 01:14:01.880 align:middle line:84% and what this has to do with when a map is-- 01:14:01.880 --> 01:14:03.800 align:middle line:84% or a bounded linear operator is onto. 01:14:03.800 --> 01:14:08.240 align:middle line:84% So adjoints pop up now when we're trying to solve equations 01:14:08.240 --> 01:14:10.010 align:middle line:90% on Hilbert spaces. 01:14:10.010 --> 01:14:11.960 align:middle line:84% The properties of the adjoint can tell us 01:14:11.960 --> 01:14:15.260 align:middle line:84% when we can always solve our equation. 01:14:15.260 --> 01:14:19.430 align:middle line:84% And not only that, they're going to end up-- 01:14:19.430 --> 01:14:25.480 align:middle line:84% they're the analog of the transpose that hopefully you 01:14:25.480 --> 01:14:25.980 align:middle line:90% saw. 01:14:25.980 --> 01:14:30.340 align:middle line:84% Maybe it was also referred to the adjoint in linear algebra 01:14:30.340 --> 01:14:31.680 align:middle line:90% on finite dimensions. 01:14:31.680 --> 01:14:34.780 align:middle line:90% 01:14:34.780 --> 01:14:38.570 align:middle line:90% And so this is way ahead. 01:14:38.570 --> 01:14:42.050 align:middle line:84% But hopefully, what you proved in linear algebra 01:14:42.050 --> 01:14:44.370 align:middle line:84% and finite dimensions was the spectral theorem, 01:14:44.370 --> 01:14:57.162 align:middle line:84% which that if you have a matrix that is equal to its adjoint-- 01:14:57.162 --> 01:14:58.870 align:middle line:84% more generally, that it's normal, that it 01:14:58.870 --> 01:15:00.130 align:middle line:90% commutes with this adjoint-- 01:15:00.130 --> 01:15:04.840 align:middle line:84% then-- and for finite dimensions, 01:15:04.840 --> 01:15:07.900 align:middle line:84% the adjoint of a matrix was switched the entries, 01:15:07.900 --> 01:15:10.300 align:middle line:84% switch i and j and take the complex conjugate. 01:15:10.300 --> 01:15:12.910 align:middle line:84% If that's equal to the matrix again, 01:15:12.910 --> 01:15:18.110 align:middle line:84% then what you can conclude, the statement 01:15:18.110 --> 01:15:19.760 align:middle line:84% of the spectral theorem is that you 01:15:19.760 --> 01:15:26.210 align:middle line:84% can find an orthonormal basis of Cn, say, or Rn, 01:15:26.210 --> 01:15:30.830 align:middle line:90% that diagonalizes the matrix. 01:15:30.830 --> 01:15:37.950 align:middle line:84% And we'll see something like that also holds in the Hilbert 01:15:37.950 --> 01:15:42.350 align:middle line:84% space setting, but it's not so simple 01:15:42.350 --> 01:15:46.100 align:middle line:84% as the eigenvalues-- as diagonalizing just 01:15:46.100 --> 01:15:48.560 align:middle line:84% does not mean finitely many eigenvalues. 01:15:48.560 --> 01:15:53.180 align:middle line:90% And we'll go over more of-- 01:15:53.180 --> 01:15:55.790 align:middle line:84% I'll go more into that when we get to it. 01:15:55.790 --> 01:15:58.300 align:middle line:84% All right, we'll stop there for now. 01:15:58.300 --> 01:16:04.000 align:middle line:90%