WEBVTT
00:00:00.130 --> 00:00:02.490
The following content is
provided under a Creative
00:00:02.490 --> 00:00:03.650
Commons license.
00:00:03.650 --> 00:00:06.920
Your support will help MIT
OpenCourseWare continue offer
00:00:06.920 --> 00:00:10.030
high quality educational
resources for free.
00:00:10.030 --> 00:00:12.780
To make a donation or to view
additional materials from
00:00:12.780 --> 00:00:16.560
hundreds of MIT courses, visit
MIT OpenCourseWare at
00:00:16.560 --> 00:00:19.270
ocw.mit.edu.
00:00:19.270 --> 00:00:26.460
PROFESSOR: I just started
talking a little bit last time
00:00:26.460 --> 00:00:33.230
about viewing L2, namely this
set of functions that are
00:00:33.230 --> 00:00:37.190
square integrable as
a vector space.
00:00:37.190 --> 00:00:39.230
And I want to reveal a
little bit about why
00:00:39.230 --> 00:00:41.370
we want to do that.
00:00:41.370 --> 00:00:45.320
Because after spending so long
worrying about all these
00:00:45.320 --> 00:00:49.620
questions of measurability and
all that, you must wonder, why
00:00:49.620 --> 00:00:53.460
do we want to look at it
a different way now.
00:00:53.460 --> 00:00:59.590
Well, a part of it is we want
to be able to look at
00:00:59.590 --> 00:01:02.820
orthogonal expansions
geometrically.
00:01:02.820 --> 00:01:04.300
In other words, we would
like to be able to
00:01:04.300 --> 00:01:06.390
draw pictures of them.
00:01:06.390 --> 00:01:09.190
You can draw pictures of a
function, but you're drawing a
00:01:09.190 --> 00:01:10.860
picture of one function.
00:01:10.860 --> 00:01:14.420
And you can't draw anything
about the relationship between
00:01:14.420 --> 00:01:17.180
different functions, except
by drawing the
00:01:17.180 --> 00:01:18.360
two different functions.
00:01:18.360 --> 00:01:20.990
You get all the detail
there, and you can't
00:01:20.990 --> 00:01:22.590
abstract it at all.
00:01:22.590 --> 00:01:25.980
So somehow we want to start
to abstract some of this
00:01:25.980 --> 00:01:27.740
information.
00:01:27.740 --> 00:01:30.650
And we want to be able to draw
pictures which look more like
00:01:30.650 --> 00:01:33.490
vector pictures than
like functions.
00:01:33.490 --> 00:01:37.250
And you'll see why that becomes
important in a while.
00:01:37.250 --> 00:01:40.090
The other thing is, when you
draw a function as a function
00:01:40.090 --> 00:01:44.520
of time, the only thing you see
it how it behaves in time.
00:01:44.520 --> 00:01:47.490
When you take the Fourier
transform and you draw it in
00:01:47.490 --> 00:01:49.960
frequency, the only thing
you see is how
00:01:49.960 --> 00:01:51.780
it behaves in frequency.
00:01:51.780 --> 00:01:55.110
And, again, you don't see what
the relationship is between
00:01:55.110 --> 00:01:56.310
different functions.
00:01:56.310 --> 00:01:58.120
So you lose all of that.
00:01:58.120 --> 00:02:01.780
When you take this signal space
viewpoint, what you're
00:02:01.780 --> 00:02:08.890
trying to do there is to not
stress time or frequency so
00:02:08.890 --> 00:02:11.270
much, but more to look
at the relationship
00:02:11.270 --> 00:02:13.030
between different functions.
00:02:13.030 --> 00:02:14.780
Why do we want to do that?
00:02:14.780 --> 00:02:17.390
Well, because as soon as we
start looking at noise and
00:02:17.390 --> 00:02:20.620
things like that, we want to
be able to tell something
00:02:20.620 --> 00:02:24.340
about how distinguishable
different functions are from
00:02:24.340 --> 00:02:25.180
each other.
00:02:25.180 --> 00:02:29.320
So the critical question there
you want to ask, when you ask
00:02:29.320 --> 00:02:32.380
how different functions are,
is how do you look at two
00:02:32.380 --> 00:02:34.870
functions both at
the same time.
00:02:34.870 --> 00:02:39.000
So, again, that's --
00:02:39.000 --> 00:02:41.450
all of this is coming back
to the same thing.
00:02:41.450 --> 00:02:45.050
So we want to be able to draw
pictures of functions which
00:02:45.050 --> 00:02:48.270
show how functions are related,
rather than show all
00:02:48.270 --> 00:02:50.910
the individual detail
of function.
00:02:50.910 --> 00:02:55.640
Finally, what we'll see at the
end of today that it gives us
00:02:55.640 --> 00:02:59.230
a lot of capability for
understanding much, much
00:02:59.230 --> 00:03:03.040
better what's going on when
we look at conversions of
00:03:03.040 --> 00:03:06.170
orthogonal series.
00:03:06.170 --> 00:03:09.040
This is something we haven't had
any way to look at before
00:03:09.040 --> 00:03:14.500
we talked about the Fourier
series, the discrete time
00:03:14.500 --> 00:03:17.510
Fourier transform, and
things like this.
00:03:17.510 --> 00:03:22.280
But we haven't been able to
really say something general.
00:03:22.280 --> 00:03:24.330
Which, again, is what
we want to do now.
00:03:28.020 --> 00:03:31.630
I'm going to go very quickly
through these axioms of a
00:03:31.630 --> 00:03:34.410
vector space.
00:03:34.410 --> 00:03:37.490
Most of you have seen
them before.
00:03:37.490 --> 00:03:41.430
Those of you who haven't seen
axiomatic treatments of
00:03:41.430 --> 00:03:44.540
various things in mathematics
are going to be
00:03:44.540 --> 00:03:46.020
puzzled by it anyway.
00:03:46.020 --> 00:03:49.510
And you're going to have to sit
at home or somewhere on
00:03:49.510 --> 00:03:52.690
your own and puzzle this out.
00:03:52.690 --> 00:03:55.720
But I just wanted to put them
up so I could start to say
00:03:55.720 --> 00:04:00.440
what it is that we're trying
to do with these axioms.
00:04:00.440 --> 00:04:05.700
What we're trying to do is to
say everything we know about
00:04:05.700 --> 00:04:11.430
n-tuples, which we've been
using all of our lives.
00:04:11.430 --> 00:04:15.580
All of these tricks that we use,
most of them we can use
00:04:15.580 --> 00:04:18.110
to deal with functions also.
00:04:18.110 --> 00:04:20.550
The pictures, we can use.
00:04:20.550 --> 00:04:23.900
All of the ideas about
dimension, all of the ideas
00:04:23.900 --> 00:04:25.880
about expansions.
00:04:25.880 --> 00:04:29.660
All of this stuff becomes
useful again.
00:04:29.660 --> 00:04:33.910
But, there are a lot of things
that are true about functions
00:04:33.910 --> 00:04:36.740
that aren't true
about vectors.
00:04:36.740 --> 00:04:42.150
There are lots of things which
are true about n-dimensional
00:04:42.150 --> 00:04:44.750
vectors that aren't true
about functions.
00:04:44.750 --> 00:04:48.750
And you want to be able to go
back to these axioms when you
00:04:48.750 --> 00:04:53.530
have to, and say, well, is this
property we're looking at
00:04:53.530 --> 00:04:56.160
something which is a consequence
of the axioms.
00:04:56.160 --> 00:05:00.850
Namely, is this an inherent
property of vector spaces, or
00:05:00.850 --> 00:05:05.830
is it in fact something else
which is just because of the
00:05:05.830 --> 00:05:08.770
particular kind of thing
we're looking at.
00:05:08.770 --> 00:05:10.180
So, vector spaces.
00:05:10.180 --> 00:05:13.690
Important thing is, there's
an addition operation.
00:05:13.690 --> 00:05:16.100
You can add any two vectors.
00:05:16.100 --> 00:05:18.000
You can't multiply
them, by the way.
00:05:18.000 --> 00:05:20.230
You can only add them.
00:05:20.230 --> 00:05:22.850
You can multiply by scalars,
which we'll talk about in the
00:05:22.850 --> 00:05:24.210
next slide.
00:05:24.210 --> 00:05:28.360
But you can only add the
vectors themselves.
00:05:28.360 --> 00:05:31.090
And the addition is commutative,
just like
00:05:31.090 --> 00:05:34.730
ordinary addition of real
numbers or complex numbers is.
00:05:34.730 --> 00:05:41.630
It's associative, which says
v, u plus w in parentheses.
00:05:41.630 --> 00:05:44.670
Now, you see why we're doing
this is, we've said by
00:05:44.670 --> 00:05:50.990
definition that for any two
vectors, u and w, there's
00:05:50.990 --> 00:05:54.300
another vector, which
is called u plus w.
00:05:54.300 --> 00:05:57.700
In other words, this axiomatic
system says, whenever you add
00:05:57.700 --> 00:06:00.110
two vectors you have
to get a vector.
00:06:00.110 --> 00:06:01.750
There's no way around that.
00:06:01.750 --> 00:06:04.720
So u plus w has to
be a vector.
00:06:04.720 --> 00:06:09.430
And that says that v plus u
plus w has to be a vector.
00:06:09.430 --> 00:06:13.520
Associativity says that you
get the same vector if you
00:06:13.520 --> 00:06:16.690
look at it the other way,
if first adding v and u
00:06:16.690 --> 00:06:18.190
and then adding w.
00:06:18.190 --> 00:06:21.940
So, anything what you call
a vector space has
00:06:21.940 --> 00:06:22.935
to have this property.
00:06:22.935 --> 00:06:24.965
Of course, the real numbers
have this property.
00:06:24.965 --> 00:06:27.340
The complex numbers have
this property.
00:06:27.340 --> 00:06:29.240
All the n-tuples you
deal with all the
00:06:29.240 --> 00:06:30.930
time have this property.
00:06:30.930 --> 00:06:32.780
Functions have this property.
00:06:32.780 --> 00:06:36.780
Sequences, infinite length
sequences have this property,
00:06:36.780 --> 00:06:38.370
so there's no big
deal about it.
00:06:38.370 --> 00:06:40.500
But that is one of the axioms.
00:06:40.500 --> 00:06:44.330
There's a unique vector 0, such
that when you add 0 to a
00:06:44.330 --> 00:06:48.300
vector, you get the
same vector again.
00:06:48.300 --> 00:06:52.380
Now, this is not the 0 in
the real number system.
00:06:52.380 --> 00:06:56.100
It's not the 0 in a complex
number system.
00:06:56.100 --> 00:07:00.040
In terms of vectors, if you're
thinking of n-tuples, this is
00:07:00.040 --> 00:07:02.590
the n-tuple which is all 0's.
00:07:02.590 --> 00:07:05.840
In terms of other vectors,
spaces, it
00:07:05.840 --> 00:07:06.820
might be other things.
00:07:06.820 --> 00:07:11.480
Like, in terms of functions, 0
means the function which is 0
00:07:11.480 --> 00:07:13.850
everywhere.
00:07:13.850 --> 00:07:17.400
And finally, there's the unique
vector minus v, which
00:07:17.400 --> 00:07:20.620
has the property that v plus
minus v -- in other words, v
00:07:20.620 --> 00:07:23.250
minus v -- is equal to 0.
00:07:23.250 --> 00:07:25.440
And in a sense that defines
00:07:25.440 --> 00:07:27.430
subtraction as well as addition.
00:07:27.430 --> 00:07:30.710
So we have addition and
subtraction, but we don't have
00:07:30.710 --> 00:07:31.960
multiplication.
00:07:38.480 --> 00:07:40.130
And there's scalar
multiplication.
00:07:40.130 --> 00:07:44.070
Which means you can multiply
a vector by a scalar.
00:07:44.070 --> 00:07:50.330
And the vector spaces we're
looking at here only have two
00:07:50.330 --> 00:07:51.400
kinds of scalars.
00:07:51.400 --> 00:07:54.730
One is real scalars and the
other is complex scalars.
00:07:54.730 --> 00:07:57.790
The two are quite different,
as you know.
00:07:57.790 --> 00:08:01.580
And so when we're talking about
a vector space we have
00:08:01.580 --> 00:08:06.340
to say what the scalar field is
that we're talking about.
00:08:06.340 --> 00:08:10.100
So, for every vector.
00:08:10.100 --> 00:08:13.800
And also for every scalar,
there's another vector which
00:08:13.800 --> 00:08:15.620
is the scalar times
the vector.
00:08:15.620 --> 00:08:18.300
Which means, you have scalar
multiplication.
00:08:18.300 --> 00:08:24.510
You can multiply vectors by
scalars in terms of n-tuples.
00:08:24.510 --> 00:08:27.750
When you're multiplying a scalar
by an n-tuple, you just
00:08:27.750 --> 00:08:30.100
multiply every component
by that scalar.
00:08:32.840 --> 00:08:37.430
When you take the scalar 1, and
in a field there's always
00:08:37.430 --> 00:08:40.070
an element 1, there's
always an element 0.
00:08:40.070 --> 00:08:43.180
When you take the element 1,
and you multiply it by a
00:08:43.180 --> 00:08:45.770
vector, you got the
same vector.
00:08:45.770 --> 00:08:48.230
Which, of course, is what you
would expect if you were
00:08:48.230 --> 00:08:49.920
looking at functions.
00:08:49.920 --> 00:08:52.290
You multiply a function.
00:08:52.290 --> 00:08:56.100
For every t you multiply it by
1, and you get the same thing
00:08:56.100 --> 00:08:57.150
back again.
00:08:57.150 --> 00:09:00.910
For an n-tuple, you multiply
every component by 1, you get
00:09:00.910 --> 00:09:02.480
the same thing back again.
00:09:02.480 --> 00:09:04.860
So that's --
00:09:04.860 --> 00:09:06.730
well, it's just another axiom.
00:09:06.730 --> 00:09:09.500
Then we have these distributive
laws.
00:09:09.500 --> 00:09:11.130
And I won't read them
out to you, they're
00:09:11.130 --> 00:09:12.330
just what they say.
00:09:12.330 --> 00:09:14.710
I want to talk about them in
a second a little bit.
00:09:18.500 --> 00:09:22.630
And as we said, the simplistic
example of a vector space, and
00:09:22.630 --> 00:09:27.160
the one you've been using for
years, partly because it saves
00:09:27.160 --> 00:09:28.840
you a lot of notation.
00:09:28.840 --> 00:09:32.240
Incidentally, one of the reasons
I didn't list for
00:09:32.240 --> 00:09:37.800
talking about L2 as a vector
space is, it saves you a lot
00:09:37.800 --> 00:09:38.320
of writing.
00:09:38.320 --> 00:09:41.720
Also, just like when you're
talking about n-tuples it
00:09:41.720 --> 00:09:42.670
saves you writing.
00:09:42.670 --> 00:09:46.640
It's a nice, convenient
notational device.
00:09:46.640 --> 00:09:50.700
I don't think any part of
mathematics becomes popular
00:09:50.700 --> 00:09:55.920
with everyone, unless it saves
notation as well as well as
00:09:55.920 --> 00:09:59.200
simplifying things.
00:09:59.200 --> 00:10:02.870
So we have these n-tuples, which
is what we mean by r sub
00:10:02.870 --> 00:10:04.210
n or c sub n.
00:10:04.210 --> 00:10:07.770
In other words, when I talk
about r sub n, I don't mean
00:10:07.770 --> 00:10:10.320
just a vector space
of dimension n.
00:10:10.320 --> 00:10:13.430
I haven't even defined
dimension yet.
00:10:13.430 --> 00:10:18.960
And when you talk in this
generality, you have to think
00:10:18.960 --> 00:10:21.000
a little bit about
what it means.
00:10:21.000 --> 00:10:25.670
So we're just talking about
n-tuples now. r sub n is
00:10:25.670 --> 00:10:29.090
n-tuples of real numbers.
c sub n is
00:10:29.090 --> 00:10:32.080
n-tuples of complex numbers.
00:10:32.080 --> 00:10:34.980
Addition and scalar
multiplication are component
00:10:34.980 --> 00:10:36.915
by component, which
we just said.
00:10:36.915 --> 00:10:42.450
In other words, w equals u plus
v means this, for all i.
00:10:42.450 --> 00:10:45.350
Scalar multiplication
means this.
00:10:45.350 --> 00:10:51.780
The unit vector, e sub i, is a
1 in the i'th position, a 0
00:10:51.780 --> 00:10:53.880
everywhere else.
00:10:53.880 --> 00:10:57.820
And what that means is that
any vector v which is an
00:10:57.820 --> 00:11:02.800
n-tuple, n rn or n cn, can
also be expressed as the
00:11:02.800 --> 00:11:06.040
equals summation of these
coefficients,
00:11:06.040 --> 00:11:08.430
times these unit factors.
00:11:08.430 --> 00:11:12.230
That looks like we're giving up
the simple notation we have
00:11:12.230 --> 00:11:14.650
and going to more complex
notation.
00:11:14.650 --> 00:11:18.390
This is a very useful
idea here.
00:11:18.390 --> 00:11:21.050
That you can take even something
as simple as
00:11:21.050 --> 00:11:25.820
n-tuples and express every
vector as a sum of these
00:11:25.820 --> 00:11:27.240
simple vectors.
00:11:27.240 --> 00:11:30.020
And when you start drawing
pictures, you start to see why
00:11:30.020 --> 00:11:31.090
this makes sense.
00:11:31.090 --> 00:11:33.930
I'm going to do this
on the next slide.
00:11:33.930 --> 00:11:35.950
So let's do it.
00:11:44.810 --> 00:11:47.990
And on the slide here, what I've
done is to draw a diagram
00:11:47.990 --> 00:11:50.940
which is one you've seen
many times, I'm sure.
00:11:50.940 --> 00:11:53.190
Except for the particular
things on it.
00:11:53.190 --> 00:11:56.700
Of two-dimensional space, where
you take a 2-tuple and
00:11:56.700 --> 00:12:00.930
you think of the first element
in the 2-tuple, v1, a being
00:12:00.930 --> 00:12:02.500
the horizontal axis.
00:12:02.500 --> 00:12:07.040
The second element, v2, as
being the vertical axis.
00:12:07.040 --> 00:12:12.250
And then you can draw any
2-tuple by going over v1 and
00:12:12.250 --> 00:12:16.680
then up v2, which is what you've
done all your lives.
00:12:16.680 --> 00:12:19.800
The reason I'm drawing this is
to show you that you can take
00:12:19.800 --> 00:12:21.380
any two vectors.
00:12:21.380 --> 00:12:26.670
First vector, v. Second
vector, u.
00:12:26.670 --> 00:12:29.070
One thing that --
00:12:29.070 --> 00:12:32.380
I'm sure you all traditionally
do is you view a
00:12:32.380 --> 00:12:34.470
vector in two ways.
00:12:34.470 --> 00:12:38.610
One, you view it as a point
in two-dimensional space.
00:12:38.610 --> 00:12:45.890
And the other is you view it as
a line from 0 up to v. And
00:12:45.890 --> 00:12:49.030
when you put a little arrow on
it, it looks like a vector,
00:12:49.030 --> 00:12:50.250
and we call it a vector.
00:12:50.250 --> 00:12:52.880
So it must be a vector, OK?
00:12:52.880 --> 00:12:56.470
So either the point or the
line represents a vector.
00:12:56.470 --> 00:12:58.780
Here's another vector, u.
00:12:58.780 --> 00:13:01.650
I can take the difference
between two vectors, u minus
00:13:01.650 --> 00:13:07.560
v, just by drawing a line
from v up to u.
00:13:07.560 --> 00:13:10.530
And thinking of that
as a vector also.
00:13:10.530 --> 00:13:16.790
So this vector really means a
vector starting here, going
00:13:16.790 --> 00:13:20.610
parallel to this, up to this
point, which is what w is.
00:13:20.610 --> 00:13:23.610
But it's very convenient to draw
it this way, which lets
00:13:23.610 --> 00:13:24.750
you know what's going on.
00:13:24.750 --> 00:13:30.950
Namely, you represent u as
the sum of v plus w.
00:13:30.950 --> 00:13:35.420
Or you represent w as a
difference between u and v.
00:13:35.420 --> 00:13:37.030
And all of this is trivial.
00:13:37.030 --> 00:13:41.090
I just want to say it explicitly
so you start to see
00:13:41.090 --> 00:13:46.200
what the connection is between
these axioms, which if I don't
00:13:46.200 --> 00:13:48.850
talk about them a little bit,
I'm sure you're just going to
00:13:48.850 --> 00:13:53.340
blow them off and decide, eh, I
know all of that, I can look
00:13:53.340 --> 00:13:55.750
at things in two dimensions
and I don't have to think
00:13:55.750 --> 00:13:57.010
about them at all.
00:13:57.010 --> 00:14:03.340
And then, in a few days, when
we're doing the same thing for
00:14:03.340 --> 00:14:06.460
functions, you're suddenly going
to become very confused.
00:14:06.460 --> 00:14:09.230
So you ought to think about
it in these simple terms.
00:14:09.230 --> 00:14:14.730
Now, when I take a scalar
multiple of the vector, in
00:14:14.730 --> 00:14:18.740
terms of these diagrams, what
we're doing is taking
00:14:18.740 --> 00:14:21.060
something on this
same line here.
00:14:21.060 --> 00:14:24.410
I can take scalar multiples
which go all the way up here,
00:14:24.410 --> 00:14:25.900
all the way down there.
00:14:25.900 --> 00:14:29.130
I can take scalar multiples
of u which go up here.
00:14:29.130 --> 00:14:31.610
And down here.
00:14:31.610 --> 00:14:37.560
The interesting thing here is
when I scale v down by alpha
00:14:37.560 --> 00:14:42.970
and I scale u down by alpha, I
can also scale w down by alpha
00:14:42.970 --> 00:14:47.000
and I connect it just like that,
from alpha v to alpha u.
00:14:47.000 --> 00:14:48.250
Which axiom is that?
00:14:54.360 --> 00:14:57.960
I mean, this is the
canonic example of
00:14:57.960 --> 00:15:01.360
one of those axioms.
00:15:01.360 --> 00:15:01.990
AUDIENCE: [UNINTELLIGIBLE]
00:15:01.990 --> 00:15:03.370
PROFESSOR: What?
00:15:03.370 --> 00:15:04.010
AUDIENCE: Distributed.
00:15:04.010 --> 00:15:05.110
PROFESSOR: Distributed, good.
00:15:05.110 --> 00:15:10.030
This is saying that alpha times
the quantity u minus v
00:15:10.030 --> 00:15:15.770
is equal to alpha u minus alpha
v. That's so trivial
00:15:15.770 --> 00:15:18.220
that it's hard to see it, but
that's what it's saying.
00:15:18.220 --> 00:15:22.540
So that the distributive law
really says the triangles
00:15:22.540 --> 00:15:23.790
maintain their shape.
00:15:26.200 --> 00:15:27.860
Maybe it's easier to just
say distributive
00:15:27.860 --> 00:15:29.110
law, I don't know.
00:15:35.960 --> 00:15:40.960
So, for the space of L2 complex
functions, we're going
00:15:40.960 --> 00:15:46.790
to define u plus v as w,
where w of t equals u
00:15:46.790 --> 00:15:48.100
of t plus v of t.
00:15:48.100 --> 00:15:51.470
Now, I've done a bunch of
things here all at once.
00:15:51.470 --> 00:15:56.350
One of them is to say what we
used to call a function, u of
00:15:56.350 --> 00:16:02.250
t, we're now referring to with
a single letter, boldface u.
00:16:02.250 --> 00:16:04.320
What's the advantage of that?
00:16:04.320 --> 00:16:08.460
Well, one advantage of it is
when you talk about a function
00:16:08.460 --> 00:16:11.090
as u of t, you're really
talking about
00:16:11.090 --> 00:16:12.630
two different things.
00:16:12.630 --> 00:16:16.530
One, you're talking about the
value of the function at a
00:16:16.530 --> 00:16:19.800
particular argument, t.
00:16:19.800 --> 00:16:20.950
And the other is,
you're talking
00:16:20.950 --> 00:16:22.860
about the whole function.
00:16:22.860 --> 00:16:24.530
I mean, sometimes you
want to say a
00:16:24.530 --> 00:16:26.130
function has some property.
00:16:26.130 --> 00:16:27.770
A function is L2.
00:16:27.770 --> 00:16:30.970
Well, u of t at a particular
t is just a number.
00:16:30.970 --> 00:16:33.220
It's not a function.
00:16:33.220 --> 00:16:36.780
So this gives us a nice way
of distinguishing between
00:16:36.780 --> 00:16:40.060
functions and between the
value of functions for
00:16:40.060 --> 00:16:42.310
particular arguments.
00:16:42.310 --> 00:16:46.060
So that's one more notational
advantage you get by talking
00:16:46.060 --> 00:16:49.310
about vectors here.
00:16:49.310 --> 00:16:54.250
We're going to define the sum
of two functions just as the
00:16:54.250 --> 00:16:55.340
point y sum.
00:16:55.340 --> 00:16:57.530
In other words, these
two functions are
00:16:57.530 --> 00:16:59.450
defined at each t.
00:16:59.450 --> 00:17:04.390
w defined at a t is the
sum of this and that.
00:17:04.390 --> 00:17:09.680
The scalar multiplication is
just defined by, at every t, u
00:17:09.680 --> 00:17:12.990
of t is equal to alpha
times v of t.
00:17:12.990 --> 00:17:14.810
Just what you'd expect.
00:17:14.810 --> 00:17:16.510
There's nothing strange here.
00:17:16.510 --> 00:17:20.600
I just want to be explicit in
saying how everything follows
00:17:20.600 --> 00:17:21.890
from these axioms here.
00:17:21.890 --> 00:17:24.260
And I won't say all
of it because
00:17:24.260 --> 00:17:27.720
there's too much of it.
00:17:27.720 --> 00:17:32.700
With this addition and scalar
multiplication, L2, the set of
00:17:32.700 --> 00:17:36.390
finite energy measurable
functions is in fact the
00:17:36.390 --> 00:17:38.390
complex vector space.
00:17:38.390 --> 00:17:40.950
And it's trivial to go back
and check through all the
00:17:40.950 --> 00:17:43.520
axioms with what we said
here, and I'm not
00:17:43.520 --> 00:17:45.610
going to do it now.
00:17:45.610 --> 00:17:48.510
There's only one of those
axioms which is a bit
00:17:48.510 --> 00:17:49.670
questionable.
00:17:49.670 --> 00:17:54.490
And that is, when you add up two
functions which are square
00:17:54.490 --> 00:17:57.880
integrable, is the sum going to
be square integrable also.
00:18:00.600 --> 00:18:04.060
Well, you nod and say yes, but
it's worthwhile proving it
00:18:04.060 --> 00:18:06.240
once in your lives.
00:18:06.240 --> 00:18:09.790
So the question is, is this
function here less than
00:18:09.790 --> 00:18:16.040
infinity if the integral of u of
t squared and the integral
00:18:16.040 --> 00:18:18.700
of v of t squared are
both integrable.
00:18:18.700 --> 00:18:22.340
Well, it's a useful inequality,
which looks like a
00:18:22.340 --> 00:18:26.260
very weak inequality but it,
in fact, is not weak.
00:18:26.260 --> 00:18:28.660
It says that u of
t plus v of t.
00:18:28.660 --> 00:18:30.975
This is just at a particular
value of t.
00:18:30.975 --> 00:18:34.320
So this is just an inequality
for real
00:18:34.320 --> 00:18:36.360
numbers and complex numbers.
00:18:36.360 --> 00:18:40.860
It says that this u of t plus
v of t, quantity squared,
00:18:40.860 --> 00:18:44.370
magnitude squared, is less than
or equal to 2 times u of
00:18:44.370 --> 00:18:47.780
t squared plus 2 times
v of t squared.
00:18:47.780 --> 00:18:50.990
You wonder, at first, what's
the 2 doing in there.
00:18:50.990 --> 00:18:55.310
But then think of making
v of t equal to u of t.
00:18:55.310 --> 00:18:59.740
You have 2 times u of t
squared, which is 4
00:18:59.740 --> 00:19:01.400
times u of t squared.
00:19:01.400 --> 00:19:03.460
Well, that's what you need
here to make this true.
00:19:03.460 --> 00:19:06.260
So, in that example
this inequality is
00:19:06.260 --> 00:19:09.350
satisfied with equality.
00:19:09.350 --> 00:19:12.110
To verify the inequality
in general, you just
00:19:12.110 --> 00:19:13.240
multiply this out.
00:19:13.240 --> 00:19:17.790
It's u of t squared plus v of t
squared plus two cross-terms
00:19:17.790 --> 00:19:19.970
and it all works.
00:19:23.250 --> 00:19:27.300
This vector space here is not
quite the vector space we want
00:19:27.300 --> 00:19:28.900
to talk about.
00:19:28.900 --> 00:19:31.800
But let's put off that question
for a while and we'll
00:19:31.800 --> 00:19:34.000
come to it later.
00:19:34.000 --> 00:19:37.290
We will come up with a vector
space which is just slightly
00:19:37.290 --> 00:19:38.540
different than that.
00:19:41.650 --> 00:19:46.620
The main thing you can do with
vector spaces is talk about
00:19:46.620 --> 00:19:48.760
their dimension.
00:19:48.760 --> 00:19:50.660
Well, there are a lot of other
things you can do, but this is
00:19:50.660 --> 00:19:53.460
one of the main things
we can do.
00:19:53.460 --> 00:19:57.080
And it's an important thing
which we have to talk about
00:19:57.080 --> 00:20:01.000
before going into inner product
spaces, which is what
00:20:01.000 --> 00:20:04.400
we're really interested in.
00:20:04.400 --> 00:20:06.620
So we need a bunch of
definitions here.
00:20:06.620 --> 00:20:10.120
All of this, I'm sure is
familiar to most of you.
00:20:10.120 --> 00:20:13.370
For those of you that it's not
familiar to, you just have to
00:20:13.370 --> 00:20:15.480
spend a little longer with it.
00:20:15.480 --> 00:20:19.650
There's not a whole
lot involved here.
00:20:19.650 --> 00:20:25.000
If you have a set of vectors,
which are in a vector space,
00:20:25.000 --> 00:20:28.940
you say that they span the
vector space if in fact every
00:20:28.940 --> 00:20:32.590
vector in this vector
space is a linear
00:20:32.590 --> 00:20:36.870
combination of those vectors.
00:20:36.870 --> 00:20:42.890
In other words, any vector, u,
can be made up as some sum of
00:20:42.890 --> 00:20:45.420
alpha i times v sub i.
00:20:45.420 --> 00:20:50.980
Now, notice we've gone a long
way here beyond those axioms.
00:20:50.980 --> 00:20:55.160
Because we're talking about
scalar multiplications.
00:20:55.160 --> 00:20:58.340
And then we're talking about
a sum of a lot of scalar
00:20:58.340 --> 00:20:59.860
multiplications.
00:20:59.860 --> 00:21:02.000
Each scalar multiplication
is a vector.
00:21:02.000 --> 00:21:05.330
The sum of a bunch of vectors,
by the associative law is, in
00:21:05.330 --> 00:21:06.670
fact, another vector.
00:21:06.670 --> 00:21:11.710
So every one of these
sums is a vector.
00:21:11.710 --> 00:21:17.110
And by definition, a set of
vectors spans a vector space
00:21:17.110 --> 00:21:21.180
if every vector can be
represented as some linear
00:21:21.180 --> 00:21:22.356
combination of them.
00:21:22.356 --> 00:21:22.530
In
00:21:22.530 --> 00:21:26.720
other words, there isn't
something outside of here
00:21:26.720 --> 00:21:29.000
sitting there waiting to
be discovered later.
00:21:29.000 --> 00:21:32.570
You really understand everything
that's there.
00:21:32.570 --> 00:21:36.230
And we say that v is finite
dimensional if it is spanned
00:21:36.230 --> 00:21:38.730
by a finite set of vectors.
00:21:38.730 --> 00:21:40.060
So that's another definition.
00:21:40.060 --> 00:21:42.390
That's what you mean by
finite dimensional.
00:21:42.390 --> 00:21:46.030
You have to be able to find a
finite set of vectors such
00:21:46.030 --> 00:21:49.020
that linear combinations of
those vectors gives you
00:21:49.020 --> 00:21:50.720
everything.
00:21:50.720 --> 00:21:52.780
It doesn't mean you have a
finite set of vectors.
00:21:52.780 --> 00:21:55.530
You have an infinite set of
vectors because you have an
00:21:55.530 --> 00:21:57.480
infinite set of scalars.
00:21:57.480 --> 00:21:59.320
In fact, you'd have an
uncountably infinite set of
00:21:59.320 --> 00:22:01.820
vectors because of
these scalars.
00:22:04.330 --> 00:22:05.300
Second definition.
00:22:05.300 --> 00:22:09.940
The vector v1 to vn are linearly
independent -- and
00:22:09.940 --> 00:22:15.150
this is a mouthful -- if u
equals the sum of alpha sub i
00:22:15.150 --> 00:22:20.280
v sub i equals 0, only
for alpha sub i equal
00:22:20.280 --> 00:22:22.780
to 0 for each i.
00:22:22.780 --> 00:22:26.900
In other words, you can't take
any linear combination of the
00:22:26.900 --> 00:22:32.130
v sub i's and get 0 unless that
linear combination is
00:22:32.130 --> 00:22:34.230
using all 0's.
00:22:34.230 --> 00:22:37.330
You can't, in any non-trivial
way, add up a bunch of
00:22:37.330 --> 00:22:39.870
vectors and get 0.
00:22:39.870 --> 00:22:43.660
To put it another way, none of
these basis vectors is a
00:22:43.660 --> 00:22:45.290
linear combination
of the others.
00:22:45.290 --> 00:22:50.130
That's usually a more convenient
way to put it.
00:22:50.130 --> 00:22:52.180
Although it takes
more writing.
00:22:52.180 --> 00:22:55.820
Now, we say that a set of
vectors are a basis for v if
00:22:55.820 --> 00:22:59.530
they're both linearly
independent and if they span
00:22:59.530 --> 00:23:02.850
v. When we first talked about
spanning, we didn't say
00:23:02.850 --> 00:23:06.740
anything about these vectors
being linearly independent, so
00:23:06.740 --> 00:23:10.110
we might have had many more
of them than we needed.
00:23:10.110 --> 00:23:14.360
Now, when we're talking about a
basis, we restrict ourselves
00:23:14.360 --> 00:23:18.330
to just the set we need
to span the space.
00:23:18.330 --> 00:23:20.920
And then the theorem, which I'm
not going to prove, but
00:23:20.920 --> 00:23:28.590
it's standard Theorem One of
any linear algebra book --
00:23:28.590 --> 00:23:31.110
well, it's probably Theorem One,
Two, Three and Four all
00:23:31.110 --> 00:23:36.990
put together -- but anyway, if
it says if v1 and v sub n span
00:23:36.990 --> 00:23:42.100
v, then a subset of these
vectors is the basis of b.
00:23:42.100 --> 00:23:45.180
In other words, if you have a
set of vectors which span a
00:23:45.180 --> 00:23:50.460
space, you can find the basis by
systematically eliminating
00:23:50.460 --> 00:23:53.470
vectors which are linear
combinations of the others,
00:23:53.470 --> 00:23:57.040
until you get to a point where
you can't do that any further.
00:23:57.040 --> 00:24:00.110
So this theorem has an algorithm
tied into it.
00:24:00.110 --> 00:24:03.280
Given any set of vectors which
span a space, you can find the
00:24:03.280 --> 00:24:06.840
basis from it by perhaps
throwing out
00:24:06.840 --> 00:24:09.510
some of those vectors.
00:24:09.510 --> 00:24:14.920
The next part of it is, if v is
a finite dimensional space,
00:24:14.920 --> 00:24:17.880
then every basis has
the same size.
00:24:17.880 --> 00:24:20.030
This, in fact, is a thing which
takes a little bit of
00:24:20.030 --> 00:24:21.940
work proving it.
00:24:21.940 --> 00:24:25.460
And, also, any linearly
independent set, v1 to v sub
00:24:25.460 --> 00:24:28.420
n, is part of the basis.
00:24:28.420 --> 00:24:31.150
In other words, here's another
algorithm you can use.
00:24:31.150 --> 00:24:35.230
You have this big, finite
dimensional space.
00:24:35.230 --> 00:24:38.960
You have a few vectors which
are linearly independent.
00:24:38.960 --> 00:24:43.230
You can build a basis starting
with these, and you just
00:24:43.230 --> 00:24:43.870
experiment.
00:24:43.870 --> 00:24:48.250
You experiment to find new
vectors, which are not linear
00:24:48.250 --> 00:24:50.200
combinations of that set.
00:24:50.200 --> 00:24:52.780
As soon as you find one, you
add it to the basis.
00:24:52.780 --> 00:24:54.200
You keep on going.
00:24:54.200 --> 00:24:56.450
And the theorem says, by
time you get to n of
00:24:56.450 --> 00:24:58.380
them, you're done.
00:24:58.380 --> 00:25:05.540
So that, in a sense, spanning
sets are too big.
00:25:05.540 --> 00:25:08.110
Linearly independent
sets are too small.
00:25:08.110 --> 00:25:11.720
And what you want to do is add
the linearly independent sets,
00:25:11.720 --> 00:25:15.150
shrink the spanning sets, and
come up with a bases.
00:25:15.150 --> 00:25:19.240
And all bases have the same
number of vectors.
00:25:19.240 --> 00:25:22.420
There many different bases you
come up with, but they all
00:25:22.420 --> 00:25:24.080
have the same number
of vectors.
00:25:26.980 --> 00:25:30.060
Not going to talk at all about
infinite dimensional spaces
00:25:30.060 --> 00:25:32.750
until the last slide today.
00:25:32.750 --> 00:25:36.330
Because the only way I know
to understand infinite
00:25:36.330 --> 00:25:42.170
dimensional vector spaces is by
thinking of them, in some
00:25:42.170 --> 00:25:46.350
sort of limiting way, as finite
dimensional spaces.
00:25:46.350 --> 00:25:50.630
And I think that's the only
way you can do it.
00:25:50.630 --> 00:25:57.260
A vector space in itself has
no sense of distance
00:25:57.260 --> 00:25:59.320
or angles in it.
00:25:59.320 --> 00:26:02.880
When I drew that picture before
for you -- let me put
00:26:02.880 --> 00:26:04.130
it up again --
00:26:11.820 --> 00:26:19.020
it almost looked like
there was some sense
00:26:19.020 --> 00:26:20.250
of distance in here.
00:26:20.250 --> 00:26:23.800
Because when we took scalar
multiples, we scaled these
00:26:23.800 --> 00:26:26.400
things down.
00:26:26.400 --> 00:26:29.540
When I took these triangles, we
scaled down the triangle,
00:26:29.540 --> 00:26:31.590
and the triangle
was maintained.
00:26:31.590 --> 00:26:35.880
But, in fact, I could do this
just as well if I took v and
00:26:35.880 --> 00:26:40.000
moved it down here almost down
on this axis, and if I took u
00:26:40.000 --> 00:26:42.770
and moved that almost
down on the axis.
00:26:42.770 --> 00:26:48.040
I have the same picture, the
same kind of distributive law.
00:26:48.040 --> 00:26:49.040
And everything else.
00:26:49.040 --> 00:26:50.700
You can't --
00:26:50.700 --> 00:26:55.030
I mean, one of the troubles with
n-dimensional space, to
00:26:55.030 --> 00:26:58.670
study what a vector space is
about, is it's very hard to
00:26:58.670 --> 00:27:04.650
think of n-tuples without
thinking of distance.
00:27:04.650 --> 00:27:07.760
And without thinking of things
being orthogonal to each
00:27:07.760 --> 00:27:09.600
other, and of all
of these things.
00:27:09.600 --> 00:27:14.520
None of that is talked about at
all in any of these axioms.
00:27:14.520 --> 00:27:17.940
And, in fact, you need some new
axioms to be able to talk
00:27:17.940 --> 00:27:23.370
about ideas of distance, or
angle, or any of these things.
00:27:23.370 --> 00:27:25.000
And that's what we
want to add here.
00:27:32.800 --> 00:27:34.860
So we need some new axioms.
00:27:34.860 --> 00:27:38.870
And what we need is a new
operation on the vector space,
00:27:38.870 --> 00:27:41.730
before the only -- the only
operations we have are
00:27:41.730 --> 00:27:43.960
addition and scalar
multiplication.
00:27:43.960 --> 00:27:46.880
So that vector spaces
are really
00:27:46.880 --> 00:27:48.760
incredibly simple animals.
00:27:48.760 --> 00:27:51.050
There's very little you
can do with them.
00:27:51.050 --> 00:27:55.120
And this added thing is called
an inner product.
00:27:55.120 --> 00:27:58.110
An inner product is
a scalar valued
00:27:58.110 --> 00:28:00.330
function of two vectors.
00:28:00.330 --> 00:28:05.580
And it's represented by
these little brackets.
00:28:05.580 --> 00:28:11.600
And the axioms that these inner
products have to satisfy
00:28:11.600 --> 00:28:15.330
is, if you're dealing with
a complex vector space.
00:28:15.330 --> 00:28:18.500
In other words, where the
scalars are complex numbers,
00:28:18.500 --> 00:28:24.530
then this inner product, when
you switch it around, you have
00:28:24.530 --> 00:28:25.340
Hermitian symmetry.
00:28:25.340 --> 00:28:31.460
Which means that this inner
product is equal to u v
00:28:31.460 --> 00:28:33.910
complex conjugate.
00:28:33.910 --> 00:28:35.500
We've already seen that
sort of thing in
00:28:35.500 --> 00:28:36.840
taking Fourier series.
00:28:36.840 --> 00:28:42.420
And the fact that when you're
dealing with complex numbers,
00:28:42.420 --> 00:28:46.340
symmetry doesn't usually hold,
and you usually need some kind
00:28:46.340 --> 00:28:51.720
of Hermitian symmetry in most
of the things you do.
00:28:51.720 --> 00:28:56.450
The next axiom is something
called bilinearity.
00:28:56.450 --> 00:28:59.920
It says that if you take a
vector which is alpha times a
00:28:59.920 --> 00:29:04.450
vector v, plus beta times a
vector u, take the inner
00:29:04.450 --> 00:29:09.210
product of that with w, it
splits up as alpha times v w
00:29:09.210 --> 00:29:11.600
plus beta times u w.
00:29:11.600 --> 00:29:13.830
How about if I do it
the other way?
00:29:13.830 --> 00:29:19.300
See if you understand what I'm
talking about at all here.
00:29:19.300 --> 00:29:30.850
Suppose we take w alpha u plus
beta v. What's that equal to?
00:29:36.380 --> 00:29:51.180
Well, it's equal to alpha
something w u plus beta w v.
00:29:51.180 --> 00:29:54.450
Except that's wrong, It's right
for real vector spaces,
00:29:54.450 --> 00:29:56.780
it's wrong for complex
vector spaces.
00:29:56.780 --> 00:30:00.030
What am I missing here?
00:30:00.030 --> 00:30:04.770
I need complex conjugates here
and complex conjugates here.
00:30:04.770 --> 00:30:07.530
I wanted to talk about that,
because when you're dealing
00:30:07.530 --> 00:30:10.760
with inner products, I don't
know whether you're like me,
00:30:10.760 --> 00:30:13.990
but every time I start dealing
with inner products and I get
00:30:13.990 --> 00:30:16.880
in a hurry writing things down,
I forgot to put those
00:30:16.880 --> 00:30:20.110
damn complex conjugates
in them.
00:30:20.110 --> 00:30:22.150
And, just be careful.
00:30:22.150 --> 00:30:23.430
Because you need them.
00:30:23.430 --> 00:30:27.110
At least go back after you're
all done and put the complex
00:30:27.110 --> 00:30:27.940
conjugates in.
00:30:27.940 --> 00:30:31.160
If you're dealing with real
vectors, of course you don't
00:30:31.160 --> 00:30:33.510
need to worry about
any of that.
00:30:33.510 --> 00:30:36.720
And all the pictures you draw
are always of real vectors.
00:30:40.340 --> 00:30:42.840
Think of trying to draw
this picture.
00:30:42.840 --> 00:30:45.520
This is the simplest picture
you can draw.
00:30:45.520 --> 00:30:46.890
Of two-dimensional vectors.
00:30:46.890 --> 00:30:52.450
This is really a picture of a
vector space of dimension two,
00:30:52.450 --> 00:30:54.550
for real vectors.
00:30:54.550 --> 00:30:58.700
What happens if you try to
draw a picture of complex
00:30:58.700 --> 00:31:00.460
two-dimensional vector space?
00:31:04.070 --> 00:31:06.680
Well, it becomes very
difficult to do.
00:31:06.680 --> 00:31:17.270
Because you're really talking
about the real part of, if
00:31:17.270 --> 00:31:22.050
you're dealing with a basis
which consists of two complex
00:31:22.050 --> 00:31:28.550
vectors, then instead of v1, you
need a real part of v1 and
00:31:28.550 --> 00:31:30.560
imaginary part of v1.
00:31:30.560 --> 00:31:33.800
Instead of v2, you need
real part of v2 and
00:31:33.800 --> 00:31:35.650
imaginary part of v2.
00:31:35.650 --> 00:31:39.090
And you can always draw this
in four dimensions.
00:31:39.090 --> 00:31:41.590
And I even have trouble drawing
in three dimensions,
00:31:41.590 --> 00:31:44.990
because somehow my pen doesn't
make marks in three
00:31:44.990 --> 00:31:46.060
dimensions.
00:31:46.060 --> 00:31:52.520
And in four dimensions,
I'm a blinking 12.
00:31:52.520 --> 00:31:56.650
And have no idea
of what to do.
00:31:56.650 --> 00:31:58.440
So you have to remember this.
00:32:01.040 --> 00:32:07.070
If you're dealing with rn or cn,
almost always you define
00:32:07.070 --> 00:32:12.090
the inner product of v and u as
the sum of the components
00:32:12.090 --> 00:32:14.900
with the second component
complex conjugated.
00:32:14.900 --> 00:32:19.690
This is just a standard thing
that we do all the time.
00:32:19.690 --> 00:32:23.690
When we do this, and we use
unit vectors, the inner
00:32:23.690 --> 00:32:29.510
product of v with the i'th unit
vector is just v sub i.
00:32:29.510 --> 00:32:31.110
That's what this formula says.
00:32:31.110 --> 00:32:37.020
Because e sub i is this vector
u, in which there's a 1 only
00:32:37.020 --> 00:32:41.220
in the i'th position, and
a 0 everywhere else.
00:32:41.220 --> 00:32:46.020
So v e i is always the v i,
and e i v is always v i
00:32:46.020 --> 00:32:47.240
complex conjugate.
00:32:47.240 --> 00:32:52.160
Again, this Hermitian nonsense
that comes up to
00:32:52.160 --> 00:32:54.720
plague us all the time.
00:32:54.720 --> 00:32:59.180
And from that, if you make v
equal to e sub j or e sub i,
00:32:59.180 --> 00:33:04.060
you get the inner product of two
of these basis vectors is
00:33:04.060 --> 00:33:07.050
equal to 0 for i unequal to j.
00:33:07.050 --> 00:33:11.240
In other words, the standard way
of drawing pictures when
00:33:11.240 --> 00:33:16.060
you make it into an inner
product space, those unit
00:33:16.060 --> 00:33:19.200
vectors become orthonormal.
00:33:19.200 --> 00:33:20.980
Because that's the way you
like to draw things.
00:33:20.980 --> 00:33:23.230
You like to draw one
here and one there.
00:33:23.230 --> 00:33:27.680
And that's what we mean by
perpendicular, which the
00:33:27.680 --> 00:33:32.160
two-dimensional or
three-dimensional word for
00:33:32.160 --> 00:33:33.410
orthogonal.
00:33:36.600 --> 00:33:40.010
So we have a couple
of definitions.
00:33:40.010 --> 00:33:45.160
The inner product of v with
itself is called inner product
00:33:45.160 --> 00:33:51.110
v squared, which is called the
squared norm of the vector.
00:33:51.110 --> 00:33:56.080
The squared norm has to be
non-negative, by axiom.
00:33:56.080 --> 00:34:00.650
It has to be greater than 0,
unless this vector, v, is in
00:34:00.650 --> 00:34:04.330
fact a 0 vector.
00:34:04.330 --> 00:34:07.050
And the length is
just the square
00:34:07.050 --> 00:34:09.530
root of the norm squared.
00:34:09.530 --> 00:34:12.020
In other words, the length and
the norm are the same thing.
00:34:12.020 --> 00:34:15.850
The norm of a vector is the
length of the vector.
00:34:15.850 --> 00:34:18.570
I've always called it the
length, but a lot of people
00:34:18.570 --> 00:34:21.190
like to call it the norm.
00:34:21.190 --> 00:34:26.530
v and u are orthogonal if the
inner product of v and u is
00:34:26.530 --> 00:34:27.900
equal to 0.
00:34:27.900 --> 00:34:29.850
How did I get that?
00:34:29.850 --> 00:34:31.730
I defined it that why.
00:34:31.730 --> 00:34:34.440
Everybody defines it that way.
00:34:34.440 --> 00:34:37.100
That's what you mean
by orthogonality.
00:34:37.100 --> 00:34:41.100
Now we have to go back and see
if it makes any sense in terms
00:34:41.100 --> 00:34:43.920
of these nice simple diagrams.
00:34:43.920 --> 00:34:47.690
But first I'm going to do
something called the
00:34:47.690 --> 00:34:50.820
one-dimensional projection
theorem.
00:34:50.820 --> 00:34:53.220
Which is something you all know
but you probably have
00:34:53.220 --> 00:34:56.150
never thought about.
00:34:56.150 --> 00:35:01.670
And what it says is, if you have
to vectors, v and u, you
00:35:01.670 --> 00:35:05.140
can always break v up
into two parts.
00:35:05.140 --> 00:35:09.530
One of which is on the
same line with u.
00:35:09.530 --> 00:35:12.060
In other words, is
colinear with u.
00:35:12.060 --> 00:35:15.330
I'm drawing a picture here for
real spaces, but when I say
00:35:15.330 --> 00:35:20.270
colinear, when I'm dealing with
complex spaces, I mean
00:35:20.270 --> 00:35:24.140
it's u times some scalar,
which could be complex.
00:35:24.140 --> 00:35:26.840
So it's somewhere
on this line.
00:35:26.840 --> 00:35:32.270
And the other part is
perpendicular to this line.
00:35:32.270 --> 00:35:35.620
And this theorem says in any
old inner product space at
00:35:35.620 --> 00:35:39.810
all, no matter how many
dimensions you have, infinite
00:35:39.810 --> 00:35:42.980
dimensional, finite dimensional,
anything, if it's
00:35:42.980 --> 00:35:46.880
an inner product space on either
the scalars r or the
00:35:46.880 --> 00:35:52.120
scalars c, you can always take
any old two vectors at all.
00:35:52.120 --> 00:35:55.630
And you can break one vector up
into a part that's colinear
00:35:55.630 --> 00:35:59.730
with the other, and another
part which is orthogonal.
00:35:59.730 --> 00:36:02.150
And you can always draw
a picture of it.
00:36:02.150 --> 00:36:06.210
If you don't mind just drawing
the picture for real vectors
00:36:06.210 --> 00:36:08.420
instead of complex vectors.
00:36:08.420 --> 00:36:11.520
This is an important idea.
00:36:11.520 --> 00:36:15.050
Because what we're going to use
it for in a while, is to
00:36:15.050 --> 00:36:18.450
be able to talk about functions
which are these
00:36:18.450 --> 00:36:21.160
incredibly complicated
objects.
00:36:21.160 --> 00:36:23.760
And we're going to talk about
two different functions.
00:36:23.760 --> 00:36:27.160
And we want to be able to draw
a picture in which those two
00:36:27.160 --> 00:36:30.990
functions are represented
just as points in a
00:36:30.990 --> 00:36:33.350
two-dimensional picture.
00:36:33.350 --> 00:36:36.060
And we're going to do that by
saying, OK, I take one of
00:36:36.060 --> 00:36:37.380
those functions.
00:36:37.380 --> 00:36:42.110
And I can represent it as partly
being colinear with
00:36:42.110 --> 00:36:43.420
this other function.
00:36:43.420 --> 00:36:47.980
And partly being orthogonal
to the other function.
00:36:47.980 --> 00:36:51.660
Which says, you can forget about
all of these functions
00:36:51.660 --> 00:36:57.010
which extend to infinity, in
time extend to infinity, in
00:36:57.010 --> 00:36:59.370
frequency and everything else.
00:36:59.370 --> 00:37:02.400
And, so long as you're only
interested in some small set
00:37:02.400 --> 00:37:06.310
of functions, you can just deal
with them as a finite
00:37:06.310 --> 00:37:08.330
dimensional vector space.
00:37:08.330 --> 00:37:12.810
You can get rid of all the mess,
and just think of them
00:37:12.810 --> 00:37:14.570
in this very simple sense.
00:37:14.570 --> 00:37:19.000
That's really why this vector
space idea, which is called
00:37:19.000 --> 00:37:22.070
signal space, is so popular
among engineers.
00:37:22.070 --> 00:37:25.420
It lets than get rid of all the
mess and think in terms of
00:37:25.420 --> 00:37:27.900
very, very simple things.
00:37:27.900 --> 00:37:32.710
So let's see why this
complicated
00:37:32.710 --> 00:37:37.870
theorem is in fact true.
00:37:37.870 --> 00:37:39.360
Let me state the
theorem first.
00:37:39.360 --> 00:37:43.800
It says for any inner products
space, v. And any vectors, u
00:37:43.800 --> 00:37:48.350
and v in v, with u
unequal to 0 --
00:37:48.350 --> 00:37:53.400
I hope I said that in the notes
-- the vector v can be
00:37:53.400 --> 00:37:59.720
broken up into a colinear term
plus an orthogonal term, where
00:37:59.720 --> 00:38:04.140
the colinear term is equal
to a scalar times u.
00:38:04.140 --> 00:38:05.980
That's what we mean
by colinear.
00:38:05.980 --> 00:38:09.070
That's just the definition
of colinear.
00:38:09.070 --> 00:38:13.210
And the other vector
is orthogonal to u.
00:38:13.210 --> 00:38:18.770
And alpha is uniquely given by
the inner product v u divided
00:38:18.770 --> 00:38:22.280
by the norm squared of u.
00:38:22.280 --> 00:38:24.400
Now, there's one thing
ugly about this.
00:38:24.400 --> 00:38:28.830
You see that norm squared, you
say, what is that doing there.
00:38:28.830 --> 00:38:31.730
It just looks like it's making
the formula complicated.
00:38:31.730 --> 00:38:33.900
Forgot about that for
the time being.
00:38:33.900 --> 00:38:37.480
We will get into it in a minute
and explain why we have
00:38:37.480 --> 00:38:39.190
that problem.
00:38:39.190 --> 00:38:41.670
But, for the moment, let's
just prove this
00:38:41.670 --> 00:38:43.540
and see what it says.
00:38:43.540 --> 00:38:48.040
So what we're going to do is
say, well, if I don't look at
00:38:48.040 --> 00:38:51.810
what the theorem is saying, what
I'd like to do is look at
00:38:51.810 --> 00:39:00.520
some generic element which is
a scalar multiple times u.
00:39:00.520 --> 00:39:06.780
So I'll say, OK let v parallel
to u be alpha times u.
00:39:06.780 --> 00:39:09.510
I don't know what alpha is yet,
but alpha's going to be
00:39:09.510 --> 00:39:11.340
whatever it has to be.
00:39:11.340 --> 00:39:15.570
We're going to choose alpha so
that this other vector, v
00:39:15.570 --> 00:39:19.960
minus v u, is a thing we're
calling v perp.
00:39:19.960 --> 00:39:22.680
So that that, the inner product
of that and u, is
00:39:22.680 --> 00:39:23.890
equal to 0.
00:39:23.890 --> 00:39:26.010
So what I'm trying to do
is to find a vector --
00:39:29.570 --> 00:39:35.070
strategy here, is to take any
old vector along this line and
00:39:35.070 --> 00:39:38.810
try to choose the scalar alpha
in such a way that the
00:39:38.810 --> 00:39:43.610
difference between this point
and this point is orthogonal
00:39:43.610 --> 00:39:45.370
to this line.
00:39:45.370 --> 00:39:47.930
That's why I started out
with alpha unknown.
00:39:47.930 --> 00:39:50.140
Alpha unknown just
says we have any
00:39:50.140 --> 00:39:51.760
point along this line.
00:39:51.760 --> 00:39:54.660
Now I'm going to find out
what alpha has to be.
00:39:54.660 --> 00:39:58.180
I hope I will find out that it
has to be only one thing, and
00:39:58.180 --> 00:40:00.980
it's uniquely chosen.
00:40:00.980 --> 00:40:04.550
And that's what we're
going to find.
00:40:04.550 --> 00:40:10.230
So v minus this projection
term, this is called a
00:40:10.230 --> 00:40:20.260
projection of v on u, is equal
to v u minus a projection
00:40:20.260 --> 00:40:22.410
inner product with u.
00:40:22.410 --> 00:40:24.330
So it's equal to this
difference here.
00:40:24.330 --> 00:40:27.870
This is equal to the
inner product.
00:40:27.870 --> 00:40:32.230
Since we have chosen this term
to be alpha times u, we can
00:40:32.230 --> 00:40:33.410
bring the alpha out.
00:40:33.410 --> 00:40:37.180
So it's alpha times the inner
product of u with itself.
00:40:37.180 --> 00:40:39.020
That's the norm squared
of alpha.
00:40:39.020 --> 00:40:42.850
So this is inner product of v
and u minus alpha times the
00:40:42.850 --> 00:40:44.100
norm squared.
00:40:46.550 --> 00:40:50.870
This is 0 if and only if you
set this equal to 0.
00:40:50.870 --> 00:40:54.720
And the only value alpha can
have to make this 0 is the
00:40:54.720 --> 00:41:01.000
inner product of v and u divided
by the norm squared.
00:41:01.000 --> 00:41:10.120
So I would think that if I ask
you to prove this without
00:41:10.120 --> 00:41:13.920
knowing the projection theorem,
I would hope that if
00:41:13.920 --> 00:41:16.640
you weren't afraid of it or
something, and you just sat
00:41:16.640 --> 00:41:18.980
down and tried to do it,
you would all do it in
00:41:18.980 --> 00:41:20.750
about half an hour.
00:41:20.750 --> 00:41:24.620
It would probably take most of
you longer than that, because
00:41:24.620 --> 00:41:27.360
everybody gets screwed up in the
notation when they first
00:41:27.360 --> 00:41:28.600
try to do this.
00:41:28.600 --> 00:41:30.900
But, in fact, this is not
a complicated thing.
00:41:30.900 --> 00:41:33.420
This is not rocket science.
00:41:38.170 --> 00:41:38.970
Now.
00:41:38.970 --> 00:41:40.860
What is this norm squared
doing here?
00:41:40.860 --> 00:41:45.100
The thing we have just proven is
that with any two vectors v
00:41:45.100 --> 00:41:51.390
and u, the projection of v on
u, namely that vector, there
00:41:51.390 --> 00:41:55.390
which has the property that v
minus v u is perpendicular to
00:41:55.390 --> 00:42:01.510
u, we showed, is this inner
product divided by the norm
00:42:01.510 --> 00:42:03.320
squared times u.
00:42:03.320 --> 00:42:06.070
Now, let me break up
this norm squared.
00:42:06.070 --> 00:42:07.790
Which is just some
positive number.
00:42:07.790 --> 00:42:10.360
It's a positive real number.
00:42:10.360 --> 00:42:12.970
Into the length times
the length.
00:42:12.970 --> 00:42:16.670
Namely, the norm u is just
some real number.
00:42:16.670 --> 00:42:21.950
And write it this way, as the
inner product of v with u
00:42:21.950 --> 00:42:27.930
divided by the length of
u times u divided by
00:42:27.930 --> 00:42:29.890
the length of u.
00:42:29.890 --> 00:42:33.540
Now, what is the vector u
divided by the length of u?
00:42:33.540 --> 00:42:35.860
AUDIENCE: [UNINTELLIGIBLE]
00:42:35.860 --> 00:42:36.140
PROFESSOR: What?
00:42:36.140 --> 00:42:38.620
AUDIENCE: [UNINTELLIGIBLE]
00:42:38.620 --> 00:42:41.220
PROFESSOR: It's the same
direction of u, but
00:42:41.220 --> 00:42:43.500
it has length 1.
00:42:43.500 --> 00:42:46.360
In other words, this is the
normalized form of u.
00:42:49.020 --> 00:42:52.430
And I have the normalized
form of u in here also.
00:42:52.430 --> 00:42:59.560
So what this is saying is that
this projection is also equal
00:42:59.560 --> 00:43:06.640
to the projection of v on the
normalized form of u, times
00:43:06.640 --> 00:43:09.560
the normalized form of u.
00:43:09.560 --> 00:43:13.530
Which says that it doesn't make
any difference what the
00:43:13.530 --> 00:43:15.300
length of u is.
00:43:15.300 --> 00:43:21.670
This projection is a function
only of the direction of u.
00:43:21.670 --> 00:43:25.070
I mean, this is obvious from
the picture, isn't it?
00:43:28.900 --> 00:43:33.230
But again, since we can't draw
pictures for complex valued
00:43:33.230 --> 00:43:36.200
things, it's nice to be able
to see it analytically.
00:43:36.200 --> 00:43:41.380
If I shorten u, or lengthen u,
this projection is still going
00:43:41.380 --> 00:43:43.080
to be exactly the same thing.
00:43:46.980 --> 00:43:49.730
And that's what the norm squared
of u is doing here.
00:43:49.730 --> 00:43:54.060
The norm squared of u is simply
sitting there so it
00:43:54.060 --> 00:43:57.430
does this normalization
function for us.
00:43:57.430 --> 00:44:03.520
It makes this projection equal
to the inner product of v with
00:44:03.520 --> 00:44:05.970
the normalized form
of u, times the
00:44:05.970 --> 00:44:07.730
normalized vector for u.
00:44:11.780 --> 00:44:16.090
The Pythagorean theorem, which
doesn't follow from this, it's
00:44:16.090 --> 00:44:20.170
something else, but it's simple
-- in fact, we can do
00:44:20.170 --> 00:44:24.660
it right away -- it says if v
and u are orthogonal, then the
00:44:24.660 --> 00:44:29.320
norm squared of u plus
v is equal to u
00:44:29.320 --> 00:44:31.990
squared plus v squared.
00:44:31.990 --> 00:44:33.120
I mean, this is something
you use
00:44:33.120 --> 00:44:35.090
geometrically all the time.
00:44:35.090 --> 00:44:36.270
And you're familiar with this.
00:44:36.270 --> 00:44:39.990
And the argument is, you just
break this up into the norm
00:44:39.990 --> 00:44:43.450
squared of u plus the norm
squared of v, plus the two
00:44:43.450 --> 00:44:48.440
cross-products, the inner
product of u times the inner
00:44:48.440 --> 00:44:52.410
product of u with v, which is 0,
and the inner product of v
00:44:52.410 --> 00:44:53.880
with u, which is 0.
00:44:53.880 --> 00:44:56.320
So the two cross-terms
go away and you're
00:44:56.320 --> 00:44:59.700
left with just this.
00:44:59.700 --> 00:45:03.870
And for any v and u, the Schwarz
inequality says that
00:45:03.870 --> 00:45:06.720
the inner product, the magnitude
of the inner product
00:45:06.720 --> 00:45:12.140
of v and u, is less than or
equal to the length of v times
00:45:12.140 --> 00:45:13.390
the length of u.
00:45:15.990 --> 00:45:20.290
The Schwarz inequality is
probably -- well, I'm not sure
00:45:20.290 --> 00:45:22.670
that it's probably, but it's
perhaps the most used
00:45:22.670 --> 00:45:25.680
inequality in mathematics.
00:45:25.680 --> 00:45:29.570
Any time you use vectors, you
use this all the time.
00:45:29.570 --> 00:45:31.020
And it's extremely useful.
00:45:31.020 --> 00:45:34.540
I'm not going to prove it here
because it's in the notes.
00:45:34.540 --> 00:45:37.940
It's a two-step proof from
what we've done.
00:45:37.940 --> 00:45:43.020
And the trouble is watching
two-step proofs in class.
00:45:43.020 --> 00:45:45.970
At a certain point you
saturate on them.
00:45:45.970 --> 00:45:48.160
And I have more important things
I want to do later
00:45:48.160 --> 00:45:50.860
today, so I don't want you
to saturate on this.
00:45:50.860 --> 00:45:53.590
You can read this at
your leisure and
00:45:53.590 --> 00:45:54.840
see why this is true.
00:45:57.610 --> 00:46:00.620
I did want to say something
about it, though.
00:46:00.620 --> 00:46:05.880
If you divide the left side by
the right side, you can write
00:46:05.880 --> 00:46:09.660
this as the magnitude of the
inner product of the
00:46:09.660 --> 00:46:15.780
normalized form of v with the
normalized form of u.
00:46:15.780 --> 00:46:20.610
If we're talking about real
vector space, this in fact is
00:46:20.610 --> 00:46:26.390
the cosine of the angle
between v and u.
00:46:26.390 --> 00:46:28.170
It's less than or equal to 1.
00:46:28.170 --> 00:46:31.410
So for real two-dimensional
vectors, the fact that the
00:46:31.410 --> 00:46:35.170
cosine is less than or equal to
1 is really equivalent to
00:46:35.170 --> 00:46:36.920
the Schwarz inequality.
00:46:36.920 --> 00:46:40.210
And this is the appropriate
generalization for any old
00:46:40.210 --> 00:46:41.460
vectors at all.
00:46:45.900 --> 00:46:51.500
And the notes would say more
about that if that went a
00:46:51.500 --> 00:46:54.900
little too quickly.
00:46:54.900 --> 00:47:01.430
OK, the inner product space of
interest to us is this thing
00:47:01.430 --> 00:47:02.720
we've called signal space.
00:47:02.720 --> 00:47:08.550
Namely, it's a space of
functions which are measurable
00:47:08.550 --> 00:47:10.010
n square integrals.
00:47:10.010 --> 00:47:15.730
In other words, finite value
when you take the square and
00:47:15.730 --> 00:47:18.410
integrate it.
00:47:18.410 --> 00:47:20.790
And we want to be able to talk
about the set of either real
00:47:20.790 --> 00:47:24.940
or complex L2 functions.
00:47:24.940 --> 00:47:28.290
So, either one of them, we're
going to define the inner
00:47:28.290 --> 00:47:30.770
product in the same
way for each.
00:47:30.770 --> 00:47:33.270
It really is the only natural
way to define an
00:47:33.270 --> 00:47:34.260
inner product here.
00:47:34.260 --> 00:47:38.640
And you'll see this more later
as we start doing other things
00:47:38.640 --> 00:47:40.090
with these inner products.
00:47:40.090 --> 00:47:45.070
But just like what when you're
dealing with n-tuples, there's
00:47:45.070 --> 00:47:47.830
only one sensible way to define
an inner product.
00:47:47.830 --> 00:47:51.880
You can define inner products
in other ways.
00:47:51.880 --> 00:47:54.480
But it's just a little
bizarre to do so.
00:47:54.480 --> 00:47:57.510
And here it's a little
bizarre also.
00:47:57.510 --> 00:48:01.500
There's a big technical
problem here.
00:48:01.500 --> 00:48:04.090
And if you look at, it can
anybody spot what it -- no,
00:48:04.090 --> 00:48:06.090
no, of course you can't spot
what it is unless you've read
00:48:06.090 --> 00:48:09.990
the notes and you
know what it is.
00:48:09.990 --> 00:48:17.070
One of the axioms of an inner
product space is that the only
00:48:17.070 --> 00:48:22.310
vector in the space which has an
inner product with itself,
00:48:22.310 --> 00:48:27.810
a squared norm equal to zero
is the zero vector.
00:48:27.810 --> 00:48:30.940
Now, we have all these crazy
vector we've been talking
00:48:30.940 --> 00:48:35.880
about, which are zero, except
on a set of measures zero.
00:48:35.880 --> 00:48:39.990
In other words, vectors'
functions which have zero
00:48:39.990 --> 00:48:45.430
energy but just pop up at
various isolated points and
00:48:45.430 --> 00:48:47.400
have values there.
00:48:47.400 --> 00:48:50.320
Which we really can't get rid
of if we view a function as
00:48:50.320 --> 00:48:54.320
something which is defined
at every value of t.
00:48:54.320 --> 00:48:56.760
You have to accept those things
as part of what you're
00:48:56.760 --> 00:48:57.800
dealing with.
00:48:57.800 --> 00:48:59.780
As soon as you started
integrating things, those
00:48:59.780 --> 00:49:01.600
things all disappear.
00:49:01.600 --> 00:49:05.930
But the trouble is, those
functions, which are zero
00:49:05.930 --> 00:49:08.900
almost everywhere,
are not zero.
00:49:08.900 --> 00:49:10.830
They're only zero almost
everywhere.
00:49:10.830 --> 00:49:13.240
They're zero for all engineering
purposes.
00:49:13.240 --> 00:49:17.700
But they're not zero, they're
only zero almost everywhere.
00:49:17.700 --> 00:49:24.330
Well, if you define the inner
product in this way and you
00:49:24.330 --> 00:49:28.790
want to satisfy the axioms of an
inner product space, you're
00:49:28.790 --> 00:49:30.250
out of luck.
00:49:30.250 --> 00:49:31.900
There's no way you can
do it, because this
00:49:31.900 --> 00:49:35.530
axiom just gets violated.
00:49:35.530 --> 00:49:37.010
So what do you do?
00:49:37.010 --> 00:49:39.040
Well, you do what we've
been doing all along.
00:49:39.040 --> 00:49:41.940
We've been sort of squinting a
little bit and saying, well,
00:49:41.940 --> 00:49:47.060
really, these functions of
measure 0 are really 0 for all
00:49:47.060 --> 00:49:49.400
practical purposes.
00:49:49.400 --> 00:49:52.820
Mathematically, what we have
to say is, we want to talk
00:49:52.820 --> 00:49:56.730
about an equivalence
class of functions.
00:49:56.730 --> 00:50:00.510
And two functions are in the
same equivalence class if
00:50:00.510 --> 00:50:03.630
their difference has 0 energy.
00:50:03.630 --> 00:50:08.340
Which is equivalent to saying
their difference is zero
00:50:08.340 --> 00:50:09.820
almost everywhere.
00:50:09.820 --> 00:50:12.580
It's one of these bizarre
functions which just jumps up
00:50:12.580 --> 00:50:15.890
at isolated points and doesn't
do anything else.
00:50:15.890 --> 00:50:19.610
Not impulses at isolated points,
just non-zero at
00:50:19.610 --> 00:50:21.450
isolated points.
00:50:21.450 --> 00:50:23.850
Impulses are not really
functions at all.
00:50:23.850 --> 00:50:27.010
So we're talking about things
that are functions, but
00:50:27.010 --> 00:50:30.290
they're these bizarre functions
which we talked
00:50:30.290 --> 00:50:31.840
about and we've said they're
unimportant.
00:50:31.840 --> 00:50:35.190
So, but they are there.
00:50:35.190 --> 00:50:38.450
So the solution is to associate
vectors with
00:50:38.450 --> 00:50:40.880
equivalence classes.
00:50:40.880 --> 00:50:44.500
And d of t and u of t are
equivalent if the v of t minus
00:50:44.500 --> 00:50:47.750
u of t is zero almost
everywhere.
00:50:47.750 --> 00:50:51.680
In other words, when we talk
about a vector, u, what we're
00:50:51.680 --> 00:50:53.050
talking about is an
00:50:53.050 --> 00:50:55.240
equivalence class of functions.
00:50:55.240 --> 00:51:01.640
It's the equivalence class of
functions for which two
00:51:01.640 --> 00:51:04.700
functions are in the same
equivalence class if they
00:51:04.700 --> 00:51:08.020
differ only on a set
of measure zero.
00:51:08.020 --> 00:51:10.100
In other words, these are the
things that gave us trouble
00:51:10.100 --> 00:51:12.490
when we were talking about
Fourier transforms.
00:51:12.490 --> 00:51:14.380
These are the things that gave
us trouble when we were
00:51:14.380 --> 00:51:18.090
talking about Fourier series.
00:51:18.090 --> 00:51:21.230
When you take anything in the
same equivalence class,
00:51:21.230 --> 00:51:24.570
time-limited functions, and
you form a Fourier series,
00:51:24.570 --> 00:51:26.830
what happens?
00:51:26.830 --> 00:51:30.000
All of the things in the same
equivalence class have the
00:51:30.000 --> 00:51:31.250
same Fourier coefficients.
00:51:33.710 --> 00:51:36.630
But when you go back from the
Fourier series coefficients
00:51:36.630 --> 00:51:40.530
back to the function, then you
might go back in a bunch of
00:51:40.530 --> 00:51:41.600
different ways.
00:51:41.600 --> 00:51:44.970
So, we started using this limit
in the mean notation and
00:51:44.970 --> 00:51:47.250
all of that stuff.
00:51:47.250 --> 00:51:50.020
And what we're doing here now
is, for these vectors, we're
00:51:50.020 --> 00:51:54.630
just saying, let's represent
a vector as this whole
00:51:54.630 --> 00:51:55.880
equivalence class.
00:52:03.300 --> 00:52:06.090
While we're talking about
vectors, we're almost always
00:52:06.090 --> 00:52:09.460
interested in orthogonal
expansions.
00:52:09.460 --> 00:52:14.340
And when we're interested in
orthogonal expansions, the
00:52:14.340 --> 00:52:18.400
coefficients in the orthogonal
expansions are found as
00:52:18.400 --> 00:52:21.040
integrals with the function.
00:52:21.040 --> 00:52:23.520
And the integrals with different
functions in the
00:52:23.520 --> 00:52:26.260
same equivalence class
are identical.
00:52:26.260 --> 00:52:28.710
In other words, any two
functions in the same
00:52:28.710 --> 00:52:33.500
equivalence class have the
same coefficients in any
00:52:33.500 --> 00:52:36.430
orthogonal expansion.
00:52:36.430 --> 00:52:40.410
So if you talk only about the
orthogonal expansion, and
00:52:40.410 --> 00:52:44.645
leave out these detailed notions
of what the function
00:52:44.645 --> 00:52:51.310
is doing at individual times,
then you don't have to worry
00:52:51.310 --> 00:52:52.860
about equivalence classes.
00:52:52.860 --> 00:52:56.190
In other words, when you --
00:52:56.190 --> 00:52:59.020
I'm going to say this again more
carefully later, but let
00:52:59.020 --> 00:53:01.910
me try to say it now a
little bit crudely.
00:53:01.910 --> 00:53:04.500
One of the things we're
interested in doing this
00:53:04.500 --> 00:53:08.740
taking this class of functions,
mapping each
00:53:08.740 --> 00:53:12.660
function into a set of
coefficients, where the set of
00:53:12.660 --> 00:53:14.880
coefficients are the
coefficients in some
00:53:14.880 --> 00:53:18.400
particular orthogonal expansion
that we're using.
00:53:18.400 --> 00:53:21.990
Namely, that's the whole way
that we're using to get from
00:53:21.990 --> 00:53:24.610
waveforms to sequences.
00:53:24.610 --> 00:53:27.600
It's the whole --
00:53:27.600 --> 00:53:30.620
it's the entire thing we're
doing when we start out on a
00:53:30.620 --> 00:53:35.260
channel with a sequence of
binary digits and then a
00:53:35.260 --> 00:53:36.790
sequence of symbols.
00:53:36.790 --> 00:53:40.110
And we modulate it
into a waveform.
00:53:40.110 --> 00:53:46.310
Again, it's the mapping from
sequence to waveform.
00:53:46.310 --> 00:53:49.320
Now, the important thing here
about these equivalence
00:53:49.320 --> 00:53:54.790
classes is, you can't tell
any of the members of the
00:53:54.790 --> 00:53:59.190
equivalence class apart within
the sequence that we're
00:53:59.190 --> 00:54:00.900
dealing with.
00:54:00.900 --> 00:54:02.760
Everything we're interested
in has to
00:54:02.760 --> 00:54:05.680
do with these sequences.
00:54:05.680 --> 00:54:08.180
I mean, if we could we'd just
ignore the waveforms
00:54:08.180 --> 00:54:09.970
altogether.
00:54:09.970 --> 00:54:14.970
Because all the processing that
we do is with sequences.
00:54:14.970 --> 00:54:19.340
So the only reason we have these
equivalence classes is
00:54:19.340 --> 00:54:23.970
because we need them to really
define what the functions are.
00:54:23.970 --> 00:54:27.280
So, we will come back
to that later.
00:54:31.780 --> 00:54:33.240
Boy, I think I'm going
to get done today.
00:54:33.240 --> 00:54:34.490
That's surprising.
00:54:37.130 --> 00:54:39.040
The next idea that
we want to talk
00:54:39.040 --> 00:54:41.940
about is vector subspaces.
00:54:41.940 --> 00:54:45.130
Again, that's an idea you've
probably heard of,
00:54:45.130 --> 00:54:46.390
for the most part.
00:54:46.390 --> 00:54:51.250
A subspace of a vector space
is a subset of the vector
00:54:51.250 --> 00:54:55.370
space such that that subspace
is a vector
00:54:55.370 --> 00:54:58.440
space in its own right.
00:54:58.440 --> 00:55:04.230
An equivalent definition is,
for all vectors u and v, in
00:55:04.230 --> 00:55:08.060
the subspace alpha
times u plus beta
00:55:08.060 --> 00:55:11.390
times v is in s also.
00:55:11.390 --> 00:55:15.060
In other words, a subspace is
something which you can't get
00:55:15.060 --> 00:55:17.140
out of by linear combinations.
00:55:19.940 --> 00:55:25.760
If I take one of these diagrams
back here that I keep
00:55:25.760 --> 00:55:29.230
looking at --
00:55:29.230 --> 00:55:30.480
I'll use this one.
00:55:34.590 --> 00:55:38.820
If I want to form a subspace of
this subspace, if I want to
00:55:38.820 --> 00:55:42.160
form a subspace of this
two-dimensional vector space
00:55:42.160 --> 00:55:45.150
here, one of the subspaces
includes
00:55:45.150 --> 00:55:48.500
u, but not v, perhaps.
00:55:48.500 --> 00:55:51.180
Now, if I want to make a
one-dimensional subspace
00:55:51.180 --> 00:55:55.760
including u, what is
that subspace?
00:55:55.760 --> 00:55:57.700
It's just all the
scalars times u.
00:55:57.700 --> 00:56:01.120
In other words, it's this line
that goes through the origin
00:56:01.120 --> 00:56:04.590
and through that vector, u.
00:56:04.590 --> 00:56:07.640
And a subspace has to include
the whole line.
00:56:07.640 --> 00:56:08.490
That's what we're saying.
00:56:08.490 --> 00:56:10.950
It's all scalar multiples
of u.
00:56:10.950 --> 00:56:14.760
If I want a subspace that
includes u and v, where u and
00:56:14.760 --> 00:56:19.790
v are just arbitrary vectors
I've chosen out of my hat,
00:56:19.790 --> 00:56:22.500
then I have this two-dimensional
subspace,
00:56:22.500 --> 00:56:24.880
which is what I've drawn here.
00:56:24.880 --> 00:56:26.480
In other words, this
idea is something
00:56:26.480 --> 00:56:28.900
we've been using already.
00:56:28.900 --> 00:56:32.010
It's just that we didn't need
to be explicit about it.
00:56:32.010 --> 00:56:37.440
The subspace which includes
u and v --
00:56:37.440 --> 00:56:40.500
well, a subspace which includes
u and v, is the
00:56:40.500 --> 00:56:44.610
subspace of all linear
combinations of u and v. And
00:56:44.610 --> 00:56:45.580
nothing else.
00:56:45.580 --> 00:56:47.830
So, it's all vectors
along here.
00:56:47.830 --> 00:56:49.570
It's all vectors along here.
00:56:49.570 --> 00:56:53.050
And you fill it all in with
anything here added to
00:56:53.050 --> 00:56:54.070
anything here.
00:56:54.070 --> 00:56:56.210
So you get this two-dimensional
space.
00:56:58.900 --> 00:57:03.050
Is 0 always in a subspace?
00:57:03.050 --> 00:57:04.140
Of course it is.
00:57:04.140 --> 00:57:07.120
I mean, you multiply
any vector by 0 and
00:57:07.120 --> 00:57:09.640
you get the 0 vector.
00:57:09.640 --> 00:57:13.220
So you sort of get
it as a linear --
00:57:17.410 --> 00:57:18.660
what more can I say?
00:57:21.530 --> 00:57:25.750
If we have a vector space which
is an inner product
00:57:25.750 --> 00:57:28.960
space; in other words, if we
add this inner product
00:57:28.960 --> 00:57:33.310
definition to our vector space,
and I take a subspace
00:57:33.310 --> 00:57:39.880
of it, that can be defined as
an inner product space also,
00:57:39.880 --> 00:57:42.010
with the same definition
of inner
00:57:42.010 --> 00:57:45.400
product that I had before.
00:57:45.400 --> 00:57:49.330
Because I can't get out of it
by linear combinations, and
00:57:49.330 --> 00:57:53.450
the inner product is defined for
every pair of vectors in
00:57:53.450 --> 00:57:54.770
that space.
00:57:54.770 --> 00:57:59.770
So we still have a nice,
well-defined vector space,
00:57:59.770 --> 00:58:05.850
which is an inner product space,
and which is a subspace
00:58:05.850 --> 00:58:09.030
of the space we started with.
00:58:09.030 --> 00:58:12.280
Everything I do from now on, I'm
going to assume that v is
00:58:12.280 --> 00:58:14.650
an inner product space.
00:58:14.650 --> 00:58:18.570
And want to look at how
we normalize vectors.
00:58:18.570 --> 00:58:20.200
We've already talked
about that.
00:58:23.170 --> 00:58:25.400
If I have a vector in this
vector space that's
00:58:25.400 --> 00:58:29.420
normalized, if its
norm equals 1.
00:58:29.420 --> 00:58:31.950
We already decided how to
normalize a vector.
00:58:31.950 --> 00:58:35.750
We took an arbitrary vector,
u, divided by its norm.
00:58:35.750 --> 00:58:41.380
And as soon as we divide by
its norm, that vector, v,
00:58:41.380 --> 00:58:48.540
divided by the norm of v, the
norm of that is just 1.
00:58:48.540 --> 00:58:52.570
So the projection, what the
projection theorem says, and
00:58:52.570 --> 00:58:55.020
all I need here is a
one-dimensional projection
00:58:55.020 --> 00:59:00.770
theorem, it says that v, in the
direction of this vector
00:59:00.770 --> 00:59:03.910
phi, is equal to the
inner product of u
00:59:03.910 --> 00:59:06.000
with phi times phi.
00:59:06.000 --> 00:59:06.520
That's what we said.
00:59:06.520 --> 00:59:10.180
As soon as we normalized these
vectors, that ugly denominator
00:59:10.180 --> 00:59:11.230
here disappears.
00:59:11.230 --> 00:59:16.020
Because the norm of
phi is equal to --
00:59:16.020 --> 00:59:18.760
because the norm
is equal to 1.
00:59:18.760 --> 00:59:23.920
So, an orthonormal set of
vectors is a set such that
00:59:23.920 --> 00:59:28.270
each pair of vectors is
orthogonal to each other, and
00:59:28.270 --> 00:59:30.330
where each vector
is normalized.
00:59:30.330 --> 00:59:34.230
In other words, it has norm
squared equal to 1.
00:59:34.230 --> 00:59:40.690
So, the inner product of these
vectors is just delta sub j k.
00:59:43.310 --> 00:59:50.390
If I have an orthogonal set, v
sub j, say, then phi sub j is
00:59:50.390 --> 00:59:54.230
an orthonormal set just by
taking each of these vectors
00:59:54.230 --> 00:59:55.890
and normalizing it.
00:59:55.890 --> 00:59:58.980
In other words, it's no big deal
to take an orthogonal set
00:59:58.980 --> 01:00:01.440
of functions and turn
them into an
01:00:01.440 --> 01:00:02.990
orthonormal set of functions.
01:00:02.990 --> 01:00:07.060
The Fourier series was natural
to define that, and most
01:00:07.060 --> 01:00:12.070
people define it, in such a way
that it's not orthonormal.
01:00:12.070 --> 01:00:15.650
Because we're defining it over
some interval of time, t, that
01:00:15.650 --> 01:00:20.250
has a norm squared of t and,
therefore, you have to divide
01:00:20.250 --> 01:00:23.920
by square root of t
to normalize it.
01:00:23.920 --> 01:00:27.310
If you want to put everything
in a common framework, it's
01:00:27.310 --> 01:00:30.200
nice to deal with orthonormal
series.
01:00:30.200 --> 01:00:33.510
And therefore, that's what we're
going to be stressing
01:00:33.510 --> 01:00:34.760
from now on.
01:00:38.360 --> 01:00:42.670
So I want to go on so the
real projection theorem.
01:00:42.670 --> 01:00:45.290
Actually, there are three
projection theorems.
01:00:45.290 --> 01:00:48.370
There's the one-dimensional
projection theorem.
01:00:48.370 --> 01:00:50.900
There's the n-dimensional
projection theorem, which is
01:00:50.900 --> 01:00:52.500
what this is.
01:00:52.500 --> 01:00:55.200
And then there's an
infinite-dimensional
01:00:55.200 --> 01:00:58.660
projection theorem, which is
not general for all inner
01:00:58.660 --> 01:01:04.860
product spaces, but is certainly
general for L2.
01:01:04.860 --> 01:01:10.070
So I'm going to assume that
phi 1 to phi sub n is an
01:01:10.070 --> 01:01:13.960
orthonormal basis for an
n-dimensional subspace, s,
01:01:13.960 --> 01:01:17.490
which is a subspace of v. How
do I know there is such an
01:01:17.490 --> 01:01:20.220
orthonormal basis?
01:01:20.220 --> 01:01:22.820
Well, I don't know that yet, and
I'm going to come back to
01:01:22.820 --> 01:01:24.010
that later.
01:01:24.010 --> 01:01:26.550
But for the time being, I'm just
going to assume that as
01:01:26.550 --> 01:01:28.510
part of the theorem.
01:01:28.510 --> 01:01:35.170
Assume I have some particular
subspace which has the
01:01:35.170 --> 01:01:39.320
property that it has an
orthonormal basis.
01:01:39.320 --> 01:01:43.180
So this is an orthonormal basis
for this n-dimensional
01:01:43.180 --> 01:01:50.760
subspace, s and v. For each
vector in v, s now is some
01:01:50.760 --> 01:01:52.290
small subspace.
01:01:52.290 --> 01:01:57.430
v is a big subspace
out around s.
01:01:57.430 --> 01:02:00.340
What I want to do now is, I want
to take some vector in
01:02:00.340 --> 01:02:01.920
the big subspace.
01:02:01.920 --> 01:02:04.170
I want to project it
onto the subspace.
01:02:06.800 --> 01:02:10.410
By projecting it onto the
subspace, what I mean is, I
01:02:10.410 --> 01:02:18.210
want to find some vector in
the subspace such that the
01:02:18.210 --> 01:02:22.620
difference between v and that
point in the subspace is
01:02:22.620 --> 01:02:25.980
orthogonal to the
subspace itself.
01:02:25.980 --> 01:02:33.020
In other words, it's the same
idea as we used before for
01:02:33.020 --> 01:02:36.950
this over-used picture.
01:02:36.950 --> 01:02:39.910
Which I keep looking at.
01:02:39.910 --> 01:02:43.330
Here, the subspace that I'm
looking at is just the
01:02:43.330 --> 01:02:48.000
subspace of vectors
colinear with u.
01:02:48.000 --> 01:02:52.620
And what I'm trying to do here
is to find, from v I'm trying
01:02:52.620 --> 01:02:56.880
to drop a perpendicular to this
subspace, which is just a
01:02:56.880 --> 01:02:58.370
straight line.
01:02:58.370 --> 01:03:02.490
In general, what I'm trying
to do is, I have an
01:03:02.490 --> 01:03:03.950
n-dimensional subspace.
01:03:07.040 --> 01:03:09.990
We can sort of visualize a
two-dimensional subspace if
01:03:09.990 --> 01:03:12.750
you think of this in
three dimensions.
01:03:12.750 --> 01:03:16.950
And think of replacing u
with some space, some
01:03:16.950 --> 01:03:20.950
two-dimensional space, which
is going through 0.
01:03:20.950 --> 01:03:23.410
And now I have this vector,
v, which is
01:03:23.410 --> 01:03:25.800
outside of that plane.
01:03:25.800 --> 01:03:29.710
And what I'm trying to do here
is to drop a perpendicular
01:03:29.710 --> 01:03:32.610
from v onto that subspace.
01:03:32.610 --> 01:03:36.850
The projection is where that
perpendicular lands.
01:03:36.850 --> 01:03:42.260
So, in other words, v, minus the
projection, is this v perp
01:03:42.260 --> 01:03:44.410
we're talking about.
01:03:44.410 --> 01:03:47.520
And what I want to do is exactly
the same thing that we
01:03:47.520 --> 01:03:48.510
did before with the
01:03:48.510 --> 01:03:51.590
one-dimensional projection theorem.
01:03:51.590 --> 01:03:54.160
And that's what we're
going to do.
01:03:54.160 --> 01:03:56.620
And it works.
01:03:56.620 --> 01:03:57.770
And it isn't really any more
01:03:57.770 --> 01:04:02.070
complicated, except for notation.
01:04:02.070 --> 01:04:05.550
So the theorem says, assume
you have this orthonormal
01:04:05.550 --> 01:04:09.180
basis for this subspace.
01:04:09.180 --> 01:04:14.610
And then you take any old v in
the entire vector space.
01:04:14.610 --> 01:04:18.710
This can be an infinite
dimensional vector space or
01:04:18.710 --> 01:04:19.470
anything else.
01:04:19.470 --> 01:04:23.470
And what we really want it to
be is some element in L2,
01:04:23.470 --> 01:04:27.330
which is some
infinite-dimensional element.
01:04:27.330 --> 01:04:34.950
It says there's a unique
projection in the subspace s.
01:04:34.950 --> 01:04:41.220
And it's given by the inner
product of v with each of
01:04:41.220 --> 01:04:43.810
those basis vectors.
01:04:43.810 --> 01:04:46.270
That inner product,
times v sub j.
01:04:46.270 --> 01:04:49.160
For the case of a
one-dimensional vector, you
01:04:49.160 --> 01:04:51.800
take the sum out and that
was exactly the
01:04:51.800 --> 01:04:54.190
projection we had before.
01:04:54.190 --> 01:04:57.540
Now we just have the
multi-dimensional projection.
01:04:57.540 --> 01:05:02.820
And it has a property that v
is equal to this projection
01:05:02.820 --> 01:05:06.910
plus the orthogonal thing.
01:05:06.910 --> 01:05:11.600
And the orthogonal thing, the
inner product of that with s
01:05:11.600 --> 01:05:15.790
equal to 0 for all s
in the subspace.
01:05:15.790 --> 01:05:18.940
In other words, it's just what
we got in this picture we were
01:05:18.940 --> 01:05:22.290
just trying to construct, of
a two-dimensional plane.
01:05:22.290 --> 01:05:25.410
You drop a perpendicular
two-dimensional plane.
01:05:25.410 --> 01:05:27.030
And when I drop a perpendicular
to a
01:05:27.030 --> 01:05:31.040
two-dimensional plane, and I
take any old vector in this
01:05:31.040 --> 01:05:34.150
two-dimensional plane,
we still have the
01:05:34.150 --> 01:05:36.890
perpendicularity.
01:05:36.890 --> 01:05:44.530
In other words, you have the
notion of this vector being
01:05:44.530 --> 01:05:48.240
perpendicular to a plane if it's
perpendicular whichever
01:05:48.240 --> 01:05:50.600
way you look at it.
01:05:50.600 --> 01:05:53.470
Let me outline the
proof of this.
01:05:53.470 --> 01:05:55.430
Actually, it's pretty much
a complete proof.
01:05:55.430 --> 01:06:01.970
But with a couple small
details left out.
01:06:01.970 --> 01:06:04.510
We're going to start out the
same way I did before.
01:06:04.510 --> 01:06:08.170
Namely, I don't know how
to choose this vector.
01:06:08.170 --> 01:06:12.390
But I know I want to choose
it to be in this subspace.
01:06:12.390 --> 01:06:14.940
And any element in this
subspace is a linear
01:06:14.940 --> 01:06:17.820
combination of these
p sub i's.
01:06:20.560 --> 01:06:24.730
So this is just a generic
element in s.
01:06:24.730 --> 01:06:28.040
And I want to find out what
element I have to use.
01:06:28.040 --> 01:06:32.050
I want to find the conditions
on these coefficients here
01:06:32.050 --> 01:06:36.310
such that v minus the projection
-- in other words,
01:06:36.310 --> 01:06:40.160
this v perp, as we've been
calling it -- is orthogonal to
01:06:40.160 --> 01:06:42.680
each phi sub i.
01:06:42.680 --> 01:06:45.980
Now, if it's orthogonal to
each phi sub i, it's also
01:06:45.980 --> 01:06:48.400
orthogonal to each linear
combination
01:06:48.400 --> 01:06:49.980
of the phi sub i's.
01:06:49.980 --> 01:06:53.190
So in fact, that solved
our problem for us.
01:06:53.190 --> 01:06:58.310
So what I want to do is, I want
to set 0 equal to v minus
01:06:58.310 --> 01:07:00.050
this projection.
01:07:00.050 --> 01:07:02.710
Where I don't yet know how to
make the projection, because I
01:07:02.710 --> 01:07:04.890
don't know what the
alpha sub i's are.
01:07:04.890 --> 01:07:09.265
But I'm trying to choose these
so that this, minus the
01:07:09.265 --> 01:07:14.590
projection, the inner product of
that with phi j is equal to
01:07:14.590 --> 01:07:18.410
0 for every j.
01:07:18.410 --> 01:07:25.750
This inner product here is equal
to the inner product of
01:07:25.750 --> 01:07:28.280
v with phi sub j.
01:07:28.280 --> 01:07:32.420
I have this difference here,
so the inner product is the
01:07:32.420 --> 01:07:37.460
inner product of v with phi sub
j minus the inner product
01:07:37.460 --> 01:07:40.590
of this with phi sub j.
01:07:40.590 --> 01:07:44.130
Let me write that here.
01:07:47.380 --> 01:08:03.020
v minus sum alpha i phi sub i
comma phi sub j is equal to v
01:08:03.020 --> 01:08:15.370
phi sub j minus summation
of i alpha i t sub i.
01:08:15.370 --> 01:08:18.110
phi sub j.
01:08:18.110 --> 01:08:20.730
Which is equal to this.
01:08:20.730 --> 01:08:25.540
All of these terms are 0 except
where j is equal to i.
01:08:30.270 --> 01:08:34.540
Where i is equal to j.
01:08:34.540 --> 01:08:38.590
So, alpha sub j has to be equal
to the inner product of
01:08:38.590 --> 01:08:41.560
v with this basis vector here.
01:08:41.560 --> 01:08:47.240
And, therefore, this projection
is equal, which we
01:08:47.240 --> 01:08:51.630
said was sum of alpha i phi sub
i, that's really the sum
01:08:51.630 --> 01:08:58.340
of v phi sub j, this inner
product, times p sub j,
01:08:58.340 --> 01:09:01.070
Now, if you really use your
imagination and you really
01:09:01.070 --> 01:09:05.580
think hard about the formula we
were using for the Fourier
01:09:05.580 --> 01:09:11.350
series coefficients, was
really the same formula
01:09:11.350 --> 01:09:12.952
without the normalization
in it.
01:09:12.952 --> 01:09:16.100
It's simplified by being already
normalized for us.
01:09:16.100 --> 01:09:19.470
We don't have that 1 over t in
here, which we had in the
01:09:19.470 --> 01:09:21.880
Fourier series because
now we've gone
01:09:21.880 --> 01:09:24.570
to orthonormal functions.
01:09:28.070 --> 01:09:30.400
So, in fact that sort of
proves the theorem.
01:09:42.200 --> 01:09:47.050
If we express v as some linear
combination of these
01:09:47.050 --> 01:09:52.960
orthonormal vectors, then if I
take the norm squared of v,
01:09:52.960 --> 01:09:55.860
this is something we've done
many times already.
01:09:55.860 --> 01:09:59.570
I just express the norm squared,
just by expanding
01:09:59.570 --> 01:10:08.530
this the sum of -- well, here
I've done it this way, so
01:10:08.530 --> 01:10:09.930
let's do it this way again.
01:10:09.930 --> 01:10:13.000
When I take the inner product
of v with all of these terms
01:10:13.000 --> 01:10:17.640
here, I get the sum of alpha
sub j complex conjugated,
01:10:17.640 --> 01:10:21.660
times the inner product
of v with phi sub j.
01:10:21.660 --> 01:10:25.940
But the inner product of v with
phi sub j is just alpha
01:10:25.940 --> 01:10:27.910
sub j times 1.
01:10:27.910 --> 01:10:31.670
So it's a sum of alpha
sub j squared.
01:10:31.670 --> 01:10:34.440
OK, this is this energy
relationship we've
01:10:34.440 --> 01:10:35.630
been using all along.
01:10:35.630 --> 01:10:37.400
We've been using it for
the Fourier series.
01:10:37.400 --> 01:10:41.330
We've been using it for
everything we've been doing.
01:10:41.330 --> 01:10:45.910
It's just a special case of
this relationship here, in
01:10:45.910 --> 01:10:49.120
this n-dimensional projection,
except that there we were
01:10:49.120 --> 01:10:51.050
dealing with infinite dimensions
and here we're
01:10:51.050 --> 01:10:53.160
dealing with finite
dimensions.
01:10:53.160 --> 01:10:56.500
But it's the same formula, and
you'll see how it generalizes
01:10:56.500 --> 01:10:59.340
in a little bit.
01:10:59.340 --> 01:11:02.400
We still have the Pythagorean
theorem, which in this case
01:11:02.400 --> 01:11:07.380
says that the norm squared of
vector v is equal to the norm
01:11:07.380 --> 01:11:11.200
squared of a projection, plus
the norm squared of the
01:11:11.200 --> 01:11:12.690
perpendicular part.
01:11:12.690 --> 01:11:15.550
In other words, when I start
to represent this vector
01:11:15.550 --> 01:11:23.640
outside of the space by a vector
inside the space, by
01:11:23.640 --> 01:11:27.140
this projection, I wind
up with two things.
01:11:27.140 --> 01:11:30.410
I wind up both with the part
that's outside of the space
01:11:30.410 --> 01:11:34.130
entirely, and is orthogonal to
the space, plus the part which
01:11:34.130 --> 01:11:35.510
is in the space.
01:11:35.510 --> 01:11:38.320
And each of those has a certain
amount of energy.
01:11:38.320 --> 01:11:45.630
When I expand this by this
relationship here --
01:11:45.630 --> 01:11:49.160
I'm not doing that yet -- what
I'm doing here is what's
01:11:49.160 --> 01:11:51.930
called a norm bound.
01:11:51.930 --> 01:11:57.230
Which says both of these
terms are non-negative.
01:11:57.230 --> 01:12:00.940
This term is non-negative in
particular, and therefore the
01:12:00.940 --> 01:12:05.780
difference between this and this
is always positive, or
01:12:05.780 --> 01:12:07.450
non-negative.
01:12:07.450 --> 01:12:11.200
Which says that 0 has to be less
than or equal to this,
01:12:11.200 --> 01:12:13.720
because it's non-negative.
01:12:13.720 --> 01:12:17.385
And this has to be less than or
equal to the norm of v. In
01:12:17.385 --> 01:12:22.020
other words, the projection
always has less energy then
01:12:22.020 --> 01:12:25.500
the vector itself.
01:12:25.500 --> 01:12:27.420
Which is not very surprising.
01:12:27.420 --> 01:12:30.000
So the norm bound
is no big deal.
01:12:30.000 --> 01:12:36.500
When I substitute this for the
actual value, what I get is
01:12:36.500 --> 01:12:43.310
the sum j equals 1 to n of the
norm of the inner product of
01:12:43.310 --> 01:12:48.690
v, with each one of these
basis vectors, magnitude
01:12:48.690 --> 01:12:53.280
squared, that's less than or
equal to the energy in v. In
01:12:53.280 --> 01:12:57.370
other words, if we start to
expand, as n gets bigger and
01:12:57.370 --> 01:13:01.670
bigger, and we look at these
terms, we take these inner
01:13:01.670 --> 01:13:03.780
products, square them.
01:13:03.780 --> 01:13:07.400
No matter how many terms I take
here, the sum is always
01:13:07.400 --> 01:13:12.620
less than or equal to the energy
in v. That's called
01:13:12.620 --> 01:13:21.030
Bessel's inequality, and it's a
nice, straightforward thing.
01:13:21.030 --> 01:13:27.436
And finally, the last
of these things --
01:13:32.500 --> 01:13:38.520
well, I'll use the other
one if I need it.
01:13:38.520 --> 01:13:40.770
The last is this thing
called the mean
01:13:40.770 --> 01:13:42.150
square error property.
01:13:44.670 --> 01:13:48.220
It says that if you take the
difference between the vector
01:13:48.220 --> 01:13:52.130
and its projection onto this
space, this is less than or
01:13:52.130 --> 01:13:55.290
equal to the difference between
the vector and the
01:13:55.290 --> 01:13:57.095
other s in the space.
01:13:57.095 --> 01:14:00.670
Any other --
01:14:00.670 --> 01:14:07.050
I can always represent v as
being equal to this plus the
01:14:07.050 --> 01:14:09.460
orthogonal component.
01:14:09.460 --> 01:14:13.010
So I wind up with a sum
here of two terms.
01:14:13.010 --> 01:14:18.910
One is the difference between
-- well, it's the --
01:14:22.140 --> 01:14:29.180
it's the length squared
of the projection.
01:14:29.180 --> 01:14:30.430
Write it out.
01:14:33.170 --> 01:14:51.410
v minus v s -- let me write this
term out. v minus s is
01:14:51.410 --> 01:15:04.220
equal to v, the projection, plus
v perpendicular to the
01:15:04.220 --> 01:15:06.920
subspace s minus
this vector s.
01:15:12.820 --> 01:15:19.155
This is perpendicular to this
and this, so -- ah,
01:15:19.155 --> 01:15:20.490
to hell with it.
01:15:20.490 --> 01:15:24.310
Excuse my language.
01:15:24.310 --> 01:15:25.770
I mean, this is proven
in the notes.
01:15:25.770 --> 01:15:30.600
I'm not going to go through it
now because I want to finish
01:15:30.600 --> 01:15:32.480
these other things.
01:15:32.480 --> 01:15:36.410
I don't want to play
around with it.
01:15:36.410 --> 01:15:39.750
We left something out of the
01:15:39.750 --> 01:15:42.690
n-dimensional projection theorem.
01:15:42.690 --> 01:15:46.800
How do you find an orthonormal
basis to start with?
01:15:46.800 --> 01:15:50.630
And there's this neat thing
called Gram-Schmidt, which I
01:15:50.630 --> 01:15:54.640
suspect most of you have
seen before also.
01:15:54.640 --> 01:15:57.310
Which is pretty simple
now in terms of
01:15:57.310 --> 01:15:59.350
the projection theorem.
01:15:59.350 --> 01:16:02.890
Gram-Schmidt is really a
bootstrap operation starting
01:16:02.890 --> 01:16:05.450
with the one-dimensional
projection theorem, working
01:16:05.450 --> 01:16:08.830
your way up to larger and
larger dimensions.
01:16:08.830 --> 01:16:12.630
And each case winding up with an
orthonormal basis for what
01:16:12.630 --> 01:16:13.490
you started with.
01:16:13.490 --> 01:16:16.420
Let's see how that happens.
01:16:16.420 --> 01:16:20.540
I start out with a basis for
an inner product subspace.
01:16:20.540 --> 01:16:24.390
So, s1 up to s sub n
as a basis for this
01:16:24.390 --> 01:16:26.540
inner product space.
01:16:26.540 --> 01:16:29.970
First thing I do is, I
start out with s1.
01:16:29.970 --> 01:16:32.490
I find the normalized
version of s1.
01:16:32.490 --> 01:16:34.500
I call that phi 1.
01:16:34.500 --> 01:16:44.740
So phi 1 is now an orthonormal
basis for the subspace whose
01:16:44.740 --> 01:16:47.670
basis is just phi 1 itself.
01:16:50.280 --> 01:16:54.210
phi 1 is the basis for the
subspace of all linear
01:16:54.210 --> 01:16:57.470
combinations of s1.
01:16:57.470 --> 01:17:01.490
So it's just a straight
line in space.
01:17:01.490 --> 01:17:04.520
The next thing I do
is, I take s2.
01:17:04.520 --> 01:17:09.310
I find the projection of
s2 on this subspace s1.
01:17:09.310 --> 01:17:11.480
I can do that.
01:17:11.480 --> 01:17:15.720
So I find a part which
is colinear with s1.
01:17:15.720 --> 01:17:18.240
I find the part which
is orthogonal.
01:17:18.240 --> 01:17:21.460
I take the orthogonal
part, and that's
01:17:21.460 --> 01:17:22.880
orthogonal, to phi 1.
01:17:22.880 --> 01:17:25.300
And I normalize it.
01:17:25.300 --> 01:17:29.540
So I then have two vectors, phi
1 and phi 2, which span
01:17:29.540 --> 01:17:32.810
the space of functions
of linear
01:17:32.810 --> 01:17:34.650
combinations of s1 and s2.
01:17:37.600 --> 01:17:45.040
And I call that subspace
S2, capital S2.
01:17:45.040 --> 01:17:46.700
And then I go on.
01:17:46.700 --> 01:17:50.930
So, given any orthonormal basis,
phi 1 up to phi sub k
01:17:50.930 --> 01:17:55.640
of the subspace s k generated
by s1 to s k, I'm going to
01:17:55.640 --> 01:18:00.730
project s k plus 1 onto this
subspace s sub k, and then I'm
01:18:00.730 --> 01:18:03.170
going to normalize it.
01:18:03.170 --> 01:18:07.210
And by going through this
procedure, I can in fact find
01:18:07.210 --> 01:18:10.780
an orthonormal basis to any set
of vectors that I want to,
01:18:10.780 --> 01:18:12.810
to any subspace that
I want to.
01:18:12.810 --> 01:18:15.730
Why is this important, is this
something you want to do?
01:18:15.730 --> 01:18:20.310
Well, it's something you can
program a computer to do
01:18:20.310 --> 01:18:22.770
almost trivially.
01:18:22.770 --> 01:18:24.980
But that's not why
we want it here.
01:18:24.980 --> 01:18:28.780
The thing we want it here is to
say that there's no reason
01:18:28.780 --> 01:18:31.870
to deal with bases other
than orthonormal bases.
01:18:31.870 --> 01:18:35.280
We can generate orthonormal
bases easily.
01:18:35.280 --> 01:18:38.760
The projection theorem now is
valid for any n-dimensional
01:18:38.760 --> 01:18:42.230
space because for any
n-dimensional space we can
01:18:42.230 --> 01:18:45.670
form this basis that we want.
01:18:45.670 --> 01:18:53.480
Let me just go on and finish
this, so we can start dealing
01:18:53.480 --> 01:18:56.840
with channels next time.
01:18:56.840 --> 01:19:01.370
So far, the projection
theorem is just for
01:19:01.370 --> 01:19:03.810
finite dimensional vectors.
01:19:03.810 --> 01:19:08.660
We want to now extend it to
infinite dimensional vectors.
01:19:08.660 --> 01:19:13.370
To accountably infinite
set of vectors.
01:19:13.370 --> 01:19:16.790
So I'm given any orthogonal set
of functions, status sub
01:19:16.790 --> 01:19:20.580
i, we can first generate
orthonormal functions as phi
01:19:20.580 --> 01:19:23.870
sub i, which are normalized.
01:19:23.870 --> 01:19:26.130
And that's old stuff.
01:19:26.130 --> 01:19:30.750
I can now think of doing the
same thing that we did before.
01:19:30.750 --> 01:19:36.800
Namely, starting out, taking any
old vector I want to, and
01:19:36.800 --> 01:19:39.880
projecting it first on
to the subspace with
01:19:39.880 --> 01:19:41.860
only phi 1 in it.
01:19:41.860 --> 01:19:45.820
Then the subspace generated by
phi 1 and phi 2, then the
01:19:45.820 --> 01:19:51.220
subspace generated by phi 1, phi
2 and phi 3, and so forth.
01:19:51.220 --> 01:19:55.360
When I do that successively,
which is successive
01:19:55.360 --> 01:20:00.420
approximations in a Fourier
expansion, or in any
01:20:00.420 --> 01:20:06.100
orthonormal expansion, what I'm
going to wind up with is
01:20:06.100 --> 01:20:09.910
the following theorem that says,
let phi sub m be a set
01:20:09.910 --> 01:20:12.090
of orthonormal functions.
01:20:12.090 --> 01:20:15.970
Let v be any L2 vector.
01:20:15.970 --> 01:20:20.140
Then there exists an L2 vector,
u, such that v minus u
01:20:20.140 --> 01:20:22.880
is orthogonal to
each phi sub n.
01:20:22.880 --> 01:20:26.720
In other words, this is the
projection theorem, carried on
01:20:26.720 --> 01:20:29.220
as n goes to infinity.
01:20:29.220 --> 01:20:31.690
But I can't quite state it
in the way I did before.
01:20:31.690 --> 01:20:33.180
I need a limit in here.
01:20:33.180 --> 01:20:40.280
Which says the limit as n goes
to infinity of u, namely what
01:20:40.280 --> 01:20:45.610
is now going to be this
projection, minus this term
01:20:45.610 --> 01:20:49.540
here, which is the term
in the subspace of
01:20:49.540 --> 01:20:51.570
these orthonormal functions.
01:20:51.570 --> 01:20:55.180
This difference goes to zero.
01:20:55.180 --> 01:20:56.090
What does this say?
01:20:56.090 --> 01:21:00.360
It doesn't say that I can take
any function, v, and expand it
01:21:00.360 --> 01:21:03.630
in an orthonormal expansion.
01:21:03.630 --> 01:21:06.840
I couldn't say that, because I
have nothing to know whether
01:21:06.840 --> 01:21:10.750
this arbitrary orthonormal
expansion I started with
01:21:10.750 --> 01:21:13.130
actually spans L2 or not.
01:21:13.130 --> 01:21:17.350
And without knowing that,
I can't state a
01:21:17.350 --> 01:21:18.210
theorem like this.
01:21:18.210 --> 01:21:21.450
But what the theorem does say
is, you take any orthonormal
01:21:21.450 --> 01:21:23.250
expansion you want to.
01:21:23.250 --> 01:21:26.900
Like the Fourier series, which
only spans functions which are
01:21:26.900 --> 01:21:28.480
time limited.
01:21:28.480 --> 01:21:30.390
I take an arbitrary function.
01:21:30.390 --> 01:21:32.960
I expand them in this
Fourier series.
01:21:32.960 --> 01:21:36.780
And, bingo, what I get is a
function u, which is the part
01:21:36.780 --> 01:21:40.430
of v, which is within these time
limits and what's left
01:21:40.430 --> 01:21:43.930
over, which is orthogonal, is
the stuff outside of those
01:21:43.930 --> 01:21:46.730
time limits.
01:21:46.730 --> 01:21:51.420
So this is the theorem that says
that in fact you can --
01:21:51.420 --> 01:21:52.650
AUDIENCE: [UNINTELLIGIBLE]
01:21:52.650 --> 01:21:52.950
PROFESSOR: What?
01:21:52.950 --> 01:21:55.550
AUDIENCE: [UNINTELLIGIBLE]
01:21:55.550 --> 01:21:59.290
PROFESSOR: It's similar
to Plancherel.
01:21:59.290 --> 01:22:04.300
Plancherel is done for the
Fourier integral, and in
01:22:04.300 --> 01:22:08.840
Plancherel you need this limit
in the mean on both sides.
01:22:08.840 --> 01:22:13.280
Here we're just dealing
with a series.
01:22:13.280 --> 01:22:18.230
I mean, we still have a limit in
the mean sort of thing, but
01:22:18.230 --> 01:22:20.720
we have a weaker theorem in
the sense that we're not
01:22:20.720 --> 01:22:24.030
asserting that this orthonormal
series actually
01:22:24.030 --> 01:22:26.750
spans all of L2.
01:22:26.750 --> 01:22:29.125
I mean, we found a couple of
orthonormal expansions
01:22:29.125 --> 01:22:32.380
that do span L2.
01:22:32.380 --> 01:22:33.980
So it's lacking in that.
01:22:33.980 --> 01:22:35.380
OK, going to stop there.