WEBVTT

00:00:00.000 --> 00:00:02.690
SPEAKER: The following content
is provided under a Creative

00:00:02.690 --> 00:00:03.630
Commons license.

00:00:03.630 --> 00:00:06.600
Your support will help MIT
OpeCourseWare continue to

00:00:06.600 --> 00:00:09.970
offer high quality educational
resources for free.

00:00:09.970 --> 00:00:12.815
To make a donation or to view
additional material from

00:00:12.815 --> 00:00:16.860
hundreds of MIT courses, visit
MIT OpenCourseWare at

00:00:16.860 --> 00:00:18.110
ocw.mit.edu.

00:00:21.490 --> 00:00:26.730
PROFESSOR: OK, we talked about
this a little bit last time.

00:00:26.730 --> 00:00:31.570
We were talking about the
detection of vectors in white

00:00:31.570 --> 00:00:33.210
Gaussian noise.

00:00:33.210 --> 00:00:35.670
When we talk about vectors we'll
often refer to white

00:00:35.670 --> 00:00:41.310
Gaussian noise as noise where
each component of the vector

00:00:41.310 --> 00:00:44.700
is independent of each
other when they're

00:00:44.700 --> 00:00:47.140
all the same variance.

00:00:47.140 --> 00:00:51.240
Usually we take the variance
to be n0 over 2,

00:00:51.240 --> 00:00:54.030
capital N0 over 2.

00:00:54.030 --> 00:00:59.530
And we'll say more about that as
we go on, but that's just--

00:00:59.530 --> 00:01:02.800
I guess one thing I ought
to say about it now--

00:01:02.800 --> 00:01:06.980
people keep wondering why we
call this N0 over 2 instead of

00:01:06.980 --> 00:01:08.440
something else.

00:01:08.440 --> 00:01:10.580
As I said before, there's
really no reason

00:01:10.580 --> 00:01:12.860
for it except custom.

00:01:12.860 --> 00:01:16.770
The one important thing that
you can always remember and

00:01:16.770 --> 00:01:22.380
which is always true is that any
time you're talking about

00:01:22.380 --> 00:01:28.650
a sequence of real noise
variables they all have

00:01:28.650 --> 00:01:34.320
variants N0 over 2, and the same
coordinate system that

00:01:34.320 --> 00:01:37.370
you're using to measure
the signals.

00:01:37.370 --> 00:01:39.650
OK, the only thing that ever
appears in any of these

00:01:39.650 --> 00:01:46.880
formulas is a ratio of signal
power or signal energy to

00:01:46.880 --> 00:01:49.430
noise signal power.

00:01:49.430 --> 00:01:58.530
And if we're up it passband,
we're dealing with the power,

00:01:58.530 --> 00:02:03.160
which is two times larger
than that at baseband.

00:02:03.160 --> 00:02:05.220
And because of that-- and
this is what really gets

00:02:05.220 --> 00:02:09.560
confusing-- is that when you
talk about N0 over 2 at

00:02:09.560 --> 00:02:13.070
passband, you are talking about
something which is twice

00:02:13.070 --> 00:02:17.240
as big as the N0 over 2, the
same N0 over 2 you were

00:02:17.240 --> 00:02:19.320
talking about at baseband.

00:02:19.320 --> 00:02:24.380
And the reason is since the
signal is twice as big there,

00:02:24.380 --> 00:02:28.120
the noise is also said
to be twice as big.

00:02:28.120 --> 00:02:29.710
I can't do anything
about that.

00:02:29.710 --> 00:02:31.950
It's just the way that everybody
does things.

00:02:31.950 --> 00:02:34.730
The other thing that everybody
does-- since everyone gets

00:02:34.730 --> 00:02:36.350
confused about that--

00:02:36.350 --> 00:02:40.230
is after they get all done
dealing with anything in a

00:02:40.230 --> 00:02:43.000
paper they're writing or
something, they always look at

00:02:43.000 --> 00:02:46.110
the signal to noise ratio that
they have and they remove all

00:02:46.110 --> 00:02:49.350
the factors of two that they
know shouldn't be there.

00:02:49.350 --> 00:02:53.190
So that you shouldn't trust
anything in the literature too

00:02:53.190 --> 00:02:56.870
much as far as factors
of two are concerned.

00:02:56.870 --> 00:02:59.630
And I try to be careful in the
notes about that, but you

00:02:59.630 --> 00:03:03.350
shouldn't trust the notes too
far along those lines, either.

00:03:03.350 --> 00:03:08.440
So well eventually I'll get the
notes straightened out on

00:03:08.440 --> 00:03:12.590
all of that but I think they're
pretty close now.

00:03:12.590 --> 00:03:16.240
But anyway, we were looking at
this question of how do you

00:03:16.240 --> 00:03:21.980
detect antipodal vectors in
white Gaussian noise?

00:03:21.980 --> 00:03:27.150
And the picture that we
can draw is this.

00:03:27.150 --> 00:03:31.820
Namely we have two signals.

00:03:31.820 --> 00:03:37.830
One is the vector, a, one
is the vector, minus a.

00:03:37.830 --> 00:03:41.910
It's in some finite dimensional
system, but we're

00:03:41.910 --> 00:03:45.310
viewing it as far as drawing
a picture as a

00:03:45.310 --> 00:03:46.940
two dimensional system.

00:03:46.940 --> 00:03:50.120
So a has some arbitrary
component

00:03:50.120 --> 00:03:51.440
in the first direction.

00:03:51.440 --> 00:03:54.330
Some arbitrary component in
the second direction.

00:03:54.330 --> 00:03:57.650
Minus a is, of course,
the reverse of that.

00:03:57.650 --> 00:04:00.160
This point right in here is
the zero point, which is

00:04:00.160 --> 00:04:01.530
halfway in between them.

00:04:06.290 --> 00:04:10.810
And the output that we observe
is either plus or minus a,

00:04:10.810 --> 00:04:16.260
plus this independent zero mean
noise, Z, which has the

00:04:16.260 --> 00:04:19.700
kind of circular symmetry
indicated here with these

00:04:19.700 --> 00:04:22.160
little circles.

00:04:22.160 --> 00:04:25.170
Each of the Z sub i have the
same variance, they're

00:04:25.170 --> 00:04:27.740
independent of each other.

00:04:27.740 --> 00:04:32.280
And when you write down the
likelihood of the probability

00:04:32.280 --> 00:04:37.110
density of this output, given
that the hypothesis was zero.

00:04:37.110 --> 00:04:41.050
Namely that plus a was the
signal which was chosen.

00:04:41.050 --> 00:04:44.290
OK, remember in all of these
things there's this process

00:04:44.290 --> 00:04:48.040
now going on that we usually
don't talk about anymore.

00:04:48.040 --> 00:04:51.730
But there's an input coming into
the communication channel

00:04:51.730 --> 00:04:53.810
which we're now calling
capital H. It's the

00:04:53.810 --> 00:04:56.620
hypothesis-- which is the thing
you're trying to detect

00:04:56.620 --> 00:04:57.970
when you're all done--

00:04:57.970 --> 00:05:03.110
that input which is one up to
capital N, or sometimes zero

00:05:03.110 --> 00:05:05.720
up to capital N minus 1.

00:05:05.720 --> 00:05:10.720
Is then mapped into a signal
from a single set of capital N

00:05:10.720 --> 00:05:11.980
different signals.

00:05:11.980 --> 00:05:15.560
So they're impossible signals
in this signal alphabet.

00:05:15.560 --> 00:05:20.600
You map the hypothesis
into one of those.

00:05:20.600 --> 00:05:24.400
From those, you generally
form a waveform.

00:05:24.400 --> 00:05:26.740
This waveform might be
modulated up to high

00:05:26.740 --> 00:05:30.410
frequency, back to low
frequency again.

00:05:30.410 --> 00:05:32.050
Detected or whatever.

00:05:32.050 --> 00:05:36.070
You got some vector,
v, at that point.

00:05:36.070 --> 00:05:42.570
Which is a sequence of samples
that you're going to be taking

00:05:42.570 --> 00:05:44.320
as far as most cases
are concerned.

00:05:44.320 --> 00:05:46.350
We'll talk more about
that later today.

00:05:46.350 --> 00:05:49.980
But anyway, v is a vector which
is plus or minus a at

00:05:49.980 --> 00:05:52.180
this point, plus this
Gaussian noise.

00:05:52.180 --> 00:05:55.110
We can write down the
probability density of that

00:05:55.110 --> 00:06:01.160
vector, v, which is if
hypothesis zero occurs.

00:06:01.160 --> 00:06:04.770
Namely if a zero enters the
communication channel, plus a

00:06:04.770 --> 00:06:06.860
is the signal which is chosen.

00:06:06.860 --> 00:06:13.090
Then what happens is that the
output is a plus Z. Which

00:06:13.090 --> 00:06:16.630
means that the probability
density of the

00:06:16.630 --> 00:06:19.800
noise is v minus a.

00:06:19.800 --> 00:06:23.480
So we have this probability
density here.

00:06:23.480 --> 00:06:27.110
When we look at the log
likelihood ratio, we're taking

00:06:27.110 --> 00:06:31.630
the logarithm of this
probability density divided by

00:06:31.630 --> 00:06:35.180
the probability density of the
alternative hypothesis.

00:06:35.180 --> 00:06:37.850
Namely v given one.

00:06:37.850 --> 00:06:40.230
Which is the same as this
formula except there's the

00:06:40.230 --> 00:06:42.830
plus a there instead
of a minus a.

00:06:42.830 --> 00:06:45.400
So you get this thing here.

00:06:45.400 --> 00:06:47.650
Now, why I wanted to talk about
this today is we're

00:06:47.650 --> 00:06:50.420
going to talk about the
complex case also and

00:06:50.420 --> 00:06:54.040
something very, very peculiar
and funny happens there.

00:06:54.040 --> 00:06:59.460
OK so the log likelihood ratio
is the scaled difference of

00:06:59.460 --> 00:07:05.120
the energy of the distance
between v and a.

00:07:05.120 --> 00:07:07.390
Which is this term here.

00:07:07.390 --> 00:07:09.820
This is just a squared distance
between the vector,

00:07:09.820 --> 00:07:11.970
v, and the vector, a.

00:07:11.970 --> 00:07:16.180
It's v minus a is that distance
there, squared.

00:07:16.180 --> 00:07:20.170
And the other term here is the
term that comes from the

00:07:20.170 --> 00:07:24.170
probability density
of v given one.

00:07:24.170 --> 00:07:27.330
Which turns out to be that
distance there squared.

00:07:27.330 --> 00:07:30.540
So you have the difference
between these two things.

00:07:30.540 --> 00:07:35.930
This is just the inner product
of v minus a with v minus a.

00:07:35.930 --> 00:07:38.770
And if you multiply that all out
it's the inner product v

00:07:38.770 --> 00:07:44.740
with itself, plus the inner
product of a with itself,

00:07:44.740 --> 00:07:50.660
minus the inner product of v and
a, minus the inner product

00:07:50.660 --> 00:07:52.990
of a with v.

00:07:52.990 --> 00:07:55.760
The only things that don't
cancel out between these two

00:07:55.760 --> 00:07:58.990
things is the inner product of v
with a, the inner product of

00:07:58.990 --> 00:08:02.650
a with v, the inner product of v
with a, the inner product of

00:08:02.650 --> 00:08:07.350
a with v. So those things last
because there's a minus sign

00:08:07.350 --> 00:08:09.270
here and a plus sign here.

00:08:09.270 --> 00:08:11.720
There's a minus sign here
and a plus sign here.

00:08:11.720 --> 00:08:15.280
So the plus and minus signs
cancel out, so you just get

00:08:15.280 --> 00:08:18.130
four of these terms here, which
is four times the inner

00:08:18.130 --> 00:08:20.230
product over n0.

00:08:20.230 --> 00:08:22.960
What happens to that
geometrically?

00:08:22.960 --> 00:08:27.020
What is the inner product
of v and a?

00:08:27.020 --> 00:08:32.210
Well it's the projection
of v on the vector a.

00:08:32.210 --> 00:08:37.110
Which happens to be the line
between minus a and plus a.

00:08:37.110 --> 00:08:39.890
Mainly the fact that it's the
line between minus a and plus

00:08:39.890 --> 00:08:42.230
a is the thing which is valuable
whether you're

00:08:42.230 --> 00:08:45.010
dealing with antipodal
communication or any other

00:08:45.010 --> 00:08:46.190
kind of communication.

00:08:46.190 --> 00:08:49.810
You're always looking at this
line between two points.

00:08:49.810 --> 00:08:53.940
So what this thing says is you
form the inner product, which

00:08:53.940 --> 00:08:58.950
says drop a perpendicular from
Z down to here, and in terms

00:08:58.950 --> 00:09:03.760
of where that perpendicular
lands here,

00:09:03.760 --> 00:09:04.890
you make your decision.

00:09:04.890 --> 00:09:07.770
Namely you compare that
with the threshold.

00:09:07.770 --> 00:09:08.200
OK?

00:09:08.200 --> 00:09:10.160
So we have two different
ways of doing this.

00:09:10.160 --> 00:09:14.700
One of them is compare this
distance with this distance.

00:09:14.700 --> 00:09:17.640
Or actually here you compare the
square distance here with

00:09:17.640 --> 00:09:19.310
the square distance there.

00:09:19.310 --> 00:09:23.380
Which says, if you're using
maximum likelihood then the

00:09:23.380 --> 00:09:26.440
threshold that you're dealing
with here is zero and you

00:09:26.440 --> 00:09:29.050
simply make your decision on
whether the log likelihood

00:09:29.050 --> 00:09:32.210
ratio is positive or minus.

00:09:32.210 --> 00:09:37.530
Which means in terms of the
projection theorem here, what

00:09:37.530 --> 00:09:40.940
you're doing is taking a
perpendicular bisector of the

00:09:40.940 --> 00:09:45.770
line between minus a and plus
a, and putting a plane there

00:09:45.770 --> 00:09:51.490
and that's the plane that
separates what goes into one

00:09:51.490 --> 00:09:53.290
and what goes into zero.

00:09:53.290 --> 00:09:54.930
This goes into zero.

00:09:54.930 --> 00:09:56.590
This goes into one.

00:09:56.590 --> 00:10:01.150
OK, so is it clear to all of you
that this is really saying

00:10:01.150 --> 00:10:02.100
the same thing as this?

00:10:02.100 --> 00:10:04.620
This inner product just
comes from here.

00:10:04.620 --> 00:10:08.310
You can either look at this as
minimum distance decoding.

00:10:08.310 --> 00:10:10.250
You just find the
point which is

00:10:10.250 --> 00:10:13.400
closest to what you receive.

00:10:15.950 --> 00:10:19.755
You find the hypothesized
signal, which is closest to

00:10:19.755 --> 00:10:21.830
the actual observation
that you make.

00:10:21.830 --> 00:10:24.600
You make your decision
in terms of that.

00:10:24.600 --> 00:10:27.690
Or you do this projection
and make your

00:10:27.690 --> 00:10:29.570
decision in terms of that.

00:10:29.570 --> 00:10:36.570
And if you use a triangle thing
which says that this

00:10:36.570 --> 00:10:39.400
squared distance plus this
squared distance is equal to

00:10:39.400 --> 00:10:40.500
that squared distance.

00:10:40.500 --> 00:10:44.090
We all remember that from third
grade or something.

00:10:44.090 --> 00:10:46.620
I don't know when.

00:10:46.620 --> 00:10:50.380
But that simply says the same
thing that this says.

00:10:50.380 --> 00:10:52.880
This just says it
more generally.

00:10:52.880 --> 00:10:58.690
In terms of an arbitrary finite
conventional vector,

00:10:58.690 --> 00:11:05.450
rather than just the case of
where you're looking at two

00:11:05.450 --> 00:11:07.110
dimensions.

00:11:07.110 --> 00:11:09.380
OK that's--

00:11:09.380 --> 00:11:13.670
we will probably come back to
look at that in a little bit,

00:11:13.670 --> 00:11:17.770
but now I want to look at
complex antipodal vectors in

00:11:17.770 --> 00:11:19.740
white Gaussian noise.

00:11:19.740 --> 00:11:25.780
So the set up there is that
the input is some vector.

00:11:25.780 --> 00:11:29.710
I usually use u's to mean
complex numbers.

00:11:29.710 --> 00:11:35.420
So the vector there is u1 up to
u sub j where u sub j is a

00:11:35.420 --> 00:11:36.440
complex number.

00:11:36.440 --> 00:11:39.110
So we're dealing with complex
vectors at this point.

00:11:39.110 --> 00:11:45.750
So we have two points, minus
a -- minus u and plus u.

00:11:45.750 --> 00:11:48.460
Where instead of being
a real vector

00:11:48.460 --> 00:11:51.060
they're complex vectors.

00:11:51.060 --> 00:11:55.560
And if you can't visualize
things geometrically, in terms

00:11:55.560 --> 00:11:58.160
of complex vectors,
join the crew.

00:11:58.160 --> 00:11:59.950
I can't either.

00:11:59.950 --> 00:12:04.330
The only thing I can never do
is talk about real vectors,

00:12:04.330 --> 00:12:07.740
try to get some idea of what's
going on from that, and use

00:12:07.740 --> 00:12:10.280
mathematics for the
complex vectors.

00:12:10.280 --> 00:12:13.540
Because the reason we use
complex vectors is that,

00:12:13.540 --> 00:12:16.570
analytically they're just as
simple as real vectors.

00:12:16.570 --> 00:12:19.160
The reason we use real vectors
is because we can draw

00:12:19.160 --> 00:12:20.370
pictures of them.

00:12:20.370 --> 00:12:24.230
I defy anybody to draw a picture
in four dimensions.

00:12:24.230 --> 00:12:27.960
Some books will do it, but
I can't understand them.

00:12:27.960 --> 00:12:30.140
And anyway.

00:12:30.140 --> 00:12:36.040
OK, so under hypothesis zero
what gets sent as u?

00:12:36.040 --> 00:12:39.530
And under hypothesis one, what
gets sent as minus u?

00:12:39.530 --> 00:12:46.080
So we're still talking about
binary communication and if

00:12:46.080 --> 00:12:48.790
you're talking about antipodal
vectors, you can't do much

00:12:48.790 --> 00:12:52.750
other than talk about binary
communication.

00:12:52.750 --> 00:12:56.430
Because if you're sending a, the
only vector antipodal to a

00:12:56.430 --> 00:12:57.750
is minus a.

00:12:57.750 --> 00:12:59.780
And then you're stuck
and you're done.

00:12:59.780 --> 00:13:03.750
So we're still talking
about binary vectors.

00:13:03.750 --> 00:13:07.280
Remember the reason why
we're doing this--

00:13:07.280 --> 00:13:09.690
because one of the things we
did last time, one of the

00:13:09.690 --> 00:13:13.170
things that's in the notes and
stressed again and again is

00:13:13.170 --> 00:13:16.250
that once you understand the
antipodal case, you can

00:13:16.250 --> 00:13:22.220
translate those two points
anywhere you want to and the

00:13:22.220 --> 00:13:28.470
maximum likelihood decision and
the MAP decision are still

00:13:28.470 --> 00:13:29.430
the same thing.

00:13:29.430 --> 00:13:33.960
You take those two points and
you just translate them in

00:13:33.960 --> 00:13:39.170
space until the mean between
them sits on the zero point.

00:13:39.170 --> 00:13:42.860
And then you're back to the
antipodal case again.

00:13:42.860 --> 00:13:49.170
OK, so the reason why we're
doing this is really so we can

00:13:49.170 --> 00:13:50.600
look at the more general case.

00:13:50.600 --> 00:13:52.870
But we don't have to have
that mean sitting

00:13:52.870 --> 00:13:54.630
around all the time.

00:13:54.630 --> 00:13:59.170
OK, Z then is going to be a
vector of j complex IID

00:13:59.170 --> 00:14:00.990
Gaussian random variables.

00:14:00.990 --> 00:14:04.900
IID real and imaginary parts.

00:14:04.900 --> 00:14:08.610
Namely the real part of each
Gaussian, complex Gaussian

00:14:08.610 --> 00:14:14.510
random variable has variance
n0 over 2 and the imaginary

00:14:14.510 --> 00:14:17.510
part also has variance
n0 over 2.

00:14:17.510 --> 00:14:21.650
These complex vectors, if you
look at the probability

00:14:21.650 --> 00:14:25.670
density for them and you draw it
in two dimensions, one for

00:14:25.670 --> 00:14:32.710
the real part one for the
imaginary part, you get this

00:14:32.710 --> 00:14:36.720
circular symmetry that we've
always associated.

00:14:36.720 --> 00:14:39.230
Those are supposed to be circles
and not ellipses.

00:14:42.260 --> 00:14:44.910
And those are sometimes
called proper complex

00:14:44.910 --> 00:14:46.860
Gaussian random variables.

00:14:46.860 --> 00:14:51.260
Because almost everywhere where
you see complex Gaussian

00:14:51.260 --> 00:14:55.020
random variables, the real
and imaginary parts are

00:14:55.020 --> 00:14:58.190
independent of each other and
both have the same variance.

00:15:01.460 --> 00:15:04.610
Again, when you look at formulas
in papers, formulas

00:15:04.610 --> 00:15:09.870
everywhere else, they are almost
always assuming this

00:15:09.870 --> 00:15:12.910
kind of circular symmetry.

00:15:12.910 --> 00:15:16.590
Or what's often called proper
complex random variables.

00:15:16.590 --> 00:15:19.380
Sort of accepting the fact that
anything else is very,

00:15:19.380 --> 00:15:22.230
very improper.

00:15:22.230 --> 00:15:26.450
It's improper because formulas
don't work in that case.

00:15:26.450 --> 00:15:31.150
OK, so we have a vector of these
complex IID Gaussian

00:15:31.150 --> 00:15:33.120
random variables.

00:15:33.120 --> 00:15:38.120
Under H equals zero, the
observation, v, is given by v

00:15:38.120 --> 00:15:42.680
equals u plus Z. And under
hypothesis one, the

00:15:42.680 --> 00:15:46.040
observation is given by minus
u plus Z. So I have exactly

00:15:46.040 --> 00:15:48.270
the same cases as
we had before.

00:15:48.270 --> 00:15:51.800
OK in other words almost all
formulas stay the same when

00:15:51.800 --> 00:15:54.680
you go from real to complex.

00:15:54.680 --> 00:15:58.370
But I don't trust the complex
case and you shouldn't either

00:15:58.370 --> 00:16:00.520
until you at least go
through it once.

00:16:00.520 --> 00:16:05.410
So what I'm going to do now is
translate this complex case to

00:16:05.410 --> 00:16:06.650
the real case.

00:16:06.650 --> 00:16:11.110
In other words, for each complex
variable I'm going to

00:16:11.110 --> 00:16:12.590
make two real variables.

00:16:12.590 --> 00:16:15.760
One the real part and one
the imaginary part.

00:16:15.760 --> 00:16:18.040
We know that the real part and
the imaginary part are

00:16:18.040 --> 00:16:19.700
independent of each other.

00:16:19.700 --> 00:16:22.820
And if these Gaussian random
variables are independent of

00:16:22.820 --> 00:16:25.750
each other, then the real and
imaginary parts of each of

00:16:25.750 --> 00:16:29.330
them are independent of the real
and imaginary parts of

00:16:29.330 --> 00:16:31.050
each of the other side.

00:16:31.050 --> 00:16:32.110
OK?

00:16:32.110 --> 00:16:37.610
So we can go from j Gaussian
random variables, independant

00:16:37.610 --> 00:16:40.300
Gaussian random variables,
which are complex.

00:16:40.300 --> 00:16:44.320
To 2j Gaussian random variables,
which at this point

00:16:44.320 --> 00:16:45.920
are going to be real.

00:16:45.920 --> 00:16:51.830
OK so it's just a translation
from j complex variables to 2j

00:16:51.830 --> 00:16:53.120
real variables.

00:16:53.120 --> 00:16:59.750
Again we can't draw pictures of
things in this j dimension,

00:16:59.750 --> 00:17:03.360
we can start to draw pictures
in 2j dimensions.

00:17:03.360 --> 00:17:06.220
If you talk about a probability
density for a

00:17:06.220 --> 00:17:10.310
complex random variable, what
are you talking about?

00:17:10.310 --> 00:17:14.030
How do you write the probability
density for just a

00:17:14.030 --> 00:17:18.030
plane Gaussian complex
random variable?

00:17:18.030 --> 00:17:19.280
What is it?

00:17:22.600 --> 00:17:23.650
Anybody know what it is?

00:17:23.650 --> 00:17:28.260
Is it one dimensional or
is it two dimensional?

00:17:28.260 --> 00:17:30.900
What does probability
density mean?

00:17:30.900 --> 00:17:34.060
It means probability
per unit area.

00:17:34.060 --> 00:17:35.970
What does area mean when
you're talking

00:17:35.970 --> 00:17:38.630
about complex numbers?

00:17:38.630 --> 00:17:43.000
Well you sort of mean what
you've drawn there, yes.

00:17:43.000 --> 00:17:48.800
And you're looking at areas in
this complex plane here.

00:17:48.800 --> 00:17:51.720
So that in fact when you write
the probability density for a

00:17:51.720 --> 00:17:55.460
complex random variable, what
you have already done whether

00:17:55.460 --> 00:17:59.360
you want to do it or not is
you've converted the problem

00:17:59.360 --> 00:18:01.660
to real and imaginary part.

00:18:01.660 --> 00:18:05.490
That's what the probability
densities are.

00:18:05.490 --> 00:18:08.640
Excuse me for a belaboring this
but, if I don't belabor

00:18:08.640 --> 00:18:12.370
it I mean there's a catch
here that comes

00:18:12.370 --> 00:18:13.870
along in a little bit.

00:18:13.870 --> 00:18:18.130
And you won't understand to
catch if you don't understand

00:18:18.130 --> 00:18:22.380
why these things are almost the
same as real variables up

00:18:22.380 --> 00:18:25.220
until the catch comes.

00:18:25.220 --> 00:18:30.020
OK, so we're going to
deal with these 2j

00:18:30.020 --> 00:18:32.530
dimensional real vectors.

00:18:32.530 --> 00:18:35.990
The components will be real part
of u j, imaginary part of

00:18:35.990 --> 00:18:39.660
u j for what goes into
the channel.

00:18:39.660 --> 00:18:44.730
And we'll let capital Y capital
Z prime be the two j

00:18:44.730 --> 00:18:48.780
dimensional real versions
of V and Z.

00:18:48.780 --> 00:18:53.650
OK so that we'll call Y the real
part, an imaginary part

00:18:53.650 --> 00:18:56.580
of Z. And you notice that what's
going on here is the

00:18:56.580 --> 00:19:00.910
same thing that's going on
when you modulate QAM.

00:19:00.910 --> 00:19:04.555
You take a complex signal, you
multiply it by either the 2pi

00:19:04.555 --> 00:19:11.920
j carrier frequency times t, and
then we started to look at

00:19:11.920 --> 00:19:14.810
orthonormal expansions, you
remember that what we looked

00:19:14.810 --> 00:19:17.560
at-- as far as the real signals
that were actually

00:19:17.560 --> 00:19:20.840
being transmitted on the
channel-- was the real part of

00:19:20.840 --> 00:19:24.890
that u of t times z to the
blah, blah, blah, and the

00:19:24.890 --> 00:19:28.880
imaginary part of u of t
times blah, blah, blah.

00:19:28.880 --> 00:19:32.310
So you've got two orthonormal
functions in place of one

00:19:32.310 --> 00:19:34.640
complex orthonormal function.

00:19:34.640 --> 00:19:36.620
And that's the same thing
that's going on here.

00:19:36.620 --> 00:19:41.490
It's just not with immodulation
put in, it's just

00:19:41.490 --> 00:19:45.480
dealing with the real parts and
imaginary parts directly.

00:19:45.480 --> 00:19:54.870
OK, so if we do that we get
a bunch of equations.

00:19:54.870 --> 00:19:56.040
They're sort of familiar
looking

00:19:56.040 --> 00:19:58.670
equations by now I hope.

00:19:58.670 --> 00:20:02.560
This is just the probability
density of this real 2j

00:20:02.560 --> 00:20:05.910
dimensional random variable.

00:20:05.910 --> 00:20:09.050
Which is all this junk that
we're used to seeing.

00:20:09.050 --> 00:20:15.910
We can collapse that into
e to the minus the norm

00:20:15.910 --> 00:20:18.190
squared of y minus a.

00:20:18.190 --> 00:20:20.790
This is the norm squared
in this 2j

00:20:20.790 --> 00:20:22.230
dimensional real space.

00:20:22.230 --> 00:20:26.590
It's not the norm squared
in this complex space.

00:20:26.590 --> 00:20:29.160
What gets confusing is
that those two norms

00:20:29.160 --> 00:20:31.150
are exactly the same.

00:20:31.150 --> 00:20:33.530
As we'll see in just
a few minutes.

00:20:33.530 --> 00:20:36.310
But anyway, what we're dealing
with now is this

00:20:36.310 --> 00:20:40.390
norm in real space.

00:20:40.390 --> 00:20:43.500
OK, note that's y--

00:20:43.500 --> 00:20:46.390
oh let me translate
this one for you.

00:20:49.540 --> 00:20:54.690
If we think of this v that we
received j complex random

00:20:54.690 --> 00:21:02.100
variables as being: real part
of v1, imaginary part of v1;

00:21:02.100 --> 00:21:07.510
real part of v2, imaginary part
of v2; and so forth, then

00:21:07.510 --> 00:21:13.140
y2j minus 1 minus
a2j minus one.

00:21:13.140 --> 00:21:15.120
I can't ever get these
formulas right.

00:21:18.110 --> 00:21:21.990
That should be a2j minus 1.

00:21:21.990 --> 00:21:23.240
There.

00:21:29.290 --> 00:21:32.200
This squared, plus
this squared--

00:21:32.200 --> 00:21:35.880
OK, in other words the real part
squared of the difference

00:21:35.880 --> 00:21:38.900
plus the imaginary part squared
of the difference-- is

00:21:38.900 --> 00:21:43.190
really just the same is
vj minus uj squared.

00:21:43.190 --> 00:21:47.080
OK, in other words you take the
complex number v sub j,

00:21:47.080 --> 00:21:50.970
you subtract off the real
number u sub j.

00:21:50.970 --> 00:21:55.380
And the way to do that, you
visualize this one complex

00:21:55.380 --> 00:21:59.870
variable in the complex plane,
and what you're doing is

00:21:59.870 --> 00:22:03.810
you're taking the square of the
real part of the distance,

00:22:03.810 --> 00:22:05.760
you're adding it to
the square of the

00:22:05.760 --> 00:22:07.960
imaginary part of the distance.

00:22:07.960 --> 00:22:11.160
OK, all of this is stuff you
learned in high school.

00:22:11.160 --> 00:22:14.390
Just viewed in a slightly
different way.

00:22:14.390 --> 00:22:18.790
OK, now when you take the
probability density with

00:22:18.790 --> 00:22:21.570
respect to these complex
vectors--

00:22:21.570 --> 00:22:24.310
which is what I want
to get at--

00:22:24.310 --> 00:22:28.000
probability density of these
complex variables really means

00:22:28.000 --> 00:22:30.890
the same thing as that with
the real variables.

00:22:30.890 --> 00:22:38.740
But then you wind up
with the magnitude

00:22:38.740 --> 00:22:41.750
of vj minus uj squared.

00:22:41.750 --> 00:22:44.800
And this term is really exactly
the same as these two

00:22:44.800 --> 00:22:45.940
terms there.

00:22:45.940 --> 00:22:49.150
So when you take this
probability density, you wind

00:22:49.150 --> 00:22:53.820
up with these terms the same and
with these terms the same.

00:22:53.820 --> 00:22:57.880
OK, in other words the complex
norm squared is the same as

00:22:57.880 --> 00:23:02.820
the real norm squared, when you
go from complex to real

00:23:02.820 --> 00:23:07.870
and imaginary parts of,
well-- as I said, this

00:23:07.870 --> 00:23:10.830
is the same as that.

00:23:10.830 --> 00:23:15.410
OK so when we look at the log
likelihood ratio, then, let's

00:23:15.410 --> 00:23:20.150
do the log likelihood ratio in
terms of the real parts first.

00:23:20.150 --> 00:23:24.820
We get the difference between
this norm squared of y minus

00:23:24.820 --> 00:23:28.030
a, and the norm squared
of y plus a.

00:23:28.030 --> 00:23:31.670
This is the part that comes from
the hypothesis zero, this

00:23:31.670 --> 00:23:35.120
is the part that comes from
the hypothesis one.

00:23:35.120 --> 00:23:37.350
OK, so we get these
two terms here.

00:23:37.350 --> 00:23:40.630
When we take the inner products
here, we get the same

00:23:40.630 --> 00:23:41.880
thing that we got before.

00:23:41.880 --> 00:23:45.440
Four times the inner
product of yna.

00:23:45.440 --> 00:23:47.560
Now, the whole reason for going
through all of this is

00:23:47.560 --> 00:23:49.340
this next formula.

00:23:49.340 --> 00:23:53.950
When you do this, you wind up
with this very bizarre four

00:23:53.950 --> 00:23:57.515
times the real part of the
inner product of v and u,

00:23:57.515 --> 00:23:59.400
divided by N0.

00:23:59.400 --> 00:24:01.700
And you get it in exactly
the same way

00:24:01.700 --> 00:24:03.050
that we got it before.

00:24:03.050 --> 00:24:06.010
Namely you take this norm
here, which is an inner

00:24:06.010 --> 00:24:09.080
product squared of v minus u.

00:24:09.080 --> 00:24:10.800
Let me write it out.

00:24:10.800 --> 00:24:12.050
I'll write it out here.

00:24:16.130 --> 00:24:26.730
Norm squared of y minus a, is
the norm squared of y plus the

00:24:26.730 --> 00:24:43.010
norm squared of a, plus the
inner product of minus ya,

00:24:43.010 --> 00:24:50.170
plus the inner product
of minus ay.

00:24:50.170 --> 00:24:52.320
What's the sum of these
two inner products?

00:24:52.320 --> 00:24:56.570
This inner product, by
definition in terms of

00:24:56.570 --> 00:25:02.630
integrals, is the integral
of minus y times a--

00:25:02.630 --> 00:25:04.770
complex conjugate.

00:25:04.770 --> 00:25:09.330
This is minus a times y--
complex conjugate.

00:25:09.330 --> 00:25:11.840
So in one case the complex
conjugate is here.

00:25:11.840 --> 00:25:13.860
In the other case it's
on the other term.

00:25:13.860 --> 00:25:16.320
In other words this and
this are complex

00:25:16.320 --> 00:25:18.920
conjugates of each other.

00:25:18.920 --> 00:25:21.970
What happens when you add
two complex conjugates?

00:25:21.970 --> 00:25:24.360
You get the real part
of the two of them.

00:25:24.360 --> 00:25:26.790
Okay so when you add these two
things you just get that real

00:25:26.790 --> 00:25:29.540
part there.

00:25:29.540 --> 00:25:31.300
OK, and then when you
do the other term,

00:25:31.300 --> 00:25:32.460
the same thing happens.

00:25:32.460 --> 00:25:36.590
The same cancellation that
we had before occurs.

00:25:36.590 --> 00:25:39.930
And these two inner products add
up so you wind up with the

00:25:39.930 --> 00:25:48.900
real part of vu over N0

00:25:48.900 --> 00:25:52.610
When we look at the picture
here, what does it mean?

00:25:52.610 --> 00:25:54.450
Well I suggest you first
look at the one

00:25:54.450 --> 00:25:56.910
dimensional case here.

00:25:56.910 --> 00:26:00.590
Namely on this one dimensional
case think of V as being a one

00:26:00.590 --> 00:26:03.620
dimensional complex
random variable.

00:26:03.620 --> 00:26:04.880
Then we can draw a picture.

00:26:04.880 --> 00:26:06.490
The picture make sense.

00:26:06.490 --> 00:26:10.070
And what we're dealing with is
the real and imaginary parts,

00:26:10.070 --> 00:26:17.170
and these distances here, when
you talk about the norm of V

00:26:17.170 --> 00:26:22.110
minus a-- namely what
corresponds to this line here,

00:26:22.110 --> 00:26:24.150
the length of this line--

00:26:24.150 --> 00:26:27.210
what do you really mean by it?

00:26:27.210 --> 00:26:32.940
If you took the inner product,
if you took the norm of v,

00:26:32.940 --> 00:26:38.990
with i times a-- namely that
the square root of minus 1

00:26:38.990 --> 00:26:41.800
times a-- would you get the same
thing or wouldn't you?

00:26:45.180 --> 00:26:48.970
If I take a complex number, and
I take the inner product

00:26:48.970 --> 00:26:53.090
of that complex number--
namely the product--

00:26:53.090 --> 00:27:06.050
of that, when I take the inner
product of ya, is this the

00:27:06.050 --> 00:27:12.710
same as the inner product
of y and i times a?

00:27:15.410 --> 00:27:16.190
Not at all.

00:27:16.190 --> 00:27:18.590
The two things are totally
different.

00:27:18.590 --> 00:27:22.240
Namely, inner products
are complex things.

00:27:22.240 --> 00:27:24.930
Norms are real things.

00:27:24.930 --> 00:27:28.080
And these norms, when you're
dealing with complex numbers,

00:27:28.080 --> 00:27:29.670
have real parts in them.

00:27:29.670 --> 00:27:33.900
In other words, this distance
that we're talking about here

00:27:33.900 --> 00:27:37.980
is not just the norm squared--

00:27:37.980 --> 00:27:40.840
well it is the norm squared--
because the norm has this

00:27:40.840 --> 00:27:43.580
complex feature built into it.

00:27:43.580 --> 00:27:46.580
Because people kept making that
mistake all the time.

00:27:46.580 --> 00:27:50.160
So they fudged the mathematics
to make it come out right.

00:27:50.160 --> 00:27:53.440
But after doing that, you have
to fudge the mathematics to

00:27:53.440 --> 00:27:56.210
come back to something that
makes sense here.

00:27:56.210 --> 00:27:59.350
So you have the real part of
this inner product, here.

00:27:59.350 --> 00:28:01.890
So in fact, what you're doing
when you're taking the inner

00:28:01.890 --> 00:28:05.590
product of two vectors and
you're trying to relate it to

00:28:05.590 --> 00:28:09.400
this plane here, this separation
plane, is you have

00:28:09.400 --> 00:28:11.300
to look at that separation
plane.

00:28:11.300 --> 00:28:13.620
You have to look at
that projection in

00:28:13.620 --> 00:28:15.050
terms of real numbers.

00:28:15.050 --> 00:28:17.880
Namely, you have to look
at the projection.

00:28:17.880 --> 00:28:23.110
First as being a complex
projection of v onto a, which

00:28:23.110 --> 00:28:25.520
gives you a complex number.

00:28:25.520 --> 00:28:29.470
And then after you do that you
have to you visualize yourself

00:28:29.470 --> 00:28:31.830
in a two dimensional
real space.

00:28:31.830 --> 00:28:34.870
And you have to project once
more from the two dimensional

00:28:34.870 --> 00:28:38.180
thing to the one dimensional
thing.

00:28:38.180 --> 00:28:40.860
And here where we're just
looking at one dimension to

00:28:40.860 --> 00:28:44.470
start with, we have to draw it
as a two dimensional space.

00:28:44.470 --> 00:28:48.490
And suddenly we are dealing
with this real part there

00:28:48.490 --> 00:28:50.440
while we're not dealing
with that here.

00:28:50.440 --> 00:28:53.740
Which is why when people say
minimum distance detection

00:28:53.740 --> 00:28:57.050
when they're dealing with
complex numbers it sounds

00:28:57.050 --> 00:28:59.100
very, very simple.

00:28:59.100 --> 00:29:00.960
But in fact, it's
not so simple.

00:29:03.500 --> 00:29:06.570
If you view this in the complex
plane, is this a

00:29:06.570 --> 00:29:11.410
linear operation or isn't it?

00:29:11.410 --> 00:29:15.000
When you're looking at things
as complex vectors.

00:29:15.000 --> 00:29:21.220
Is this thing a sub space of
the complex vector space?

00:29:21.220 --> 00:29:23.170
No, it's not.

00:29:23.170 --> 00:29:24.970
It's not a sub space.

00:29:24.970 --> 00:29:29.490
Because, to be a sub space you
have to be able to multiply by

00:29:29.490 --> 00:29:31.390
arbitrary scalors--

00:29:31.390 --> 00:29:33.960
which includes complex
numbers-- and

00:29:33.960 --> 00:29:35.840
stay in the same space.

00:29:35.840 --> 00:29:39.410
And here the complex numbers
are important.

00:29:39.410 --> 00:29:40.650
OK?

00:29:40.650 --> 00:29:43.110
You should go back and
think about that.

00:29:43.110 --> 00:29:46.650
You will be confused about it
for the first ten times you

00:29:46.650 --> 00:29:48.370
think about it.

00:29:48.370 --> 00:29:51.900
For those of you who stick with
it and carry through on

00:29:51.900 --> 00:29:53.990
it, you'll be happy because
you'll never be

00:29:53.990 --> 00:29:56.800
confused about it again.

00:29:56.800 --> 00:30:01.080
OK, anyway thats real
numbers there.

00:30:01.080 --> 00:30:06.860
And the most straightforward way
to deal with complex noise

00:30:06.860 --> 00:30:09.650
is to first turn it
into real noise.

00:30:09.650 --> 00:30:12.410
If you do that you never
get confused.

00:30:12.410 --> 00:30:16.690
And otherwise you only have
half the analytical work.

00:30:16.690 --> 00:30:19.300
You only have half the
writing to do.

00:30:19.300 --> 00:30:22.040
But you never know whether
you've done the right thing

00:30:22.040 --> 00:30:24.470
until you go back and check.

00:30:24.470 --> 00:30:28.830
OK the probability of error
for maximum likelihood

00:30:28.830 --> 00:30:33.170
detection, in other words where
the threshold for the

00:30:33.170 --> 00:30:37.820
log likelihood ratio is zero, is
simply the same thing that

00:30:37.820 --> 00:30:39.070
it was before.

00:30:39.070 --> 00:30:42.150
Namely it's the q function.

00:30:42.150 --> 00:30:47.530
This tale of the Gaussian
normal function.

00:30:47.530 --> 00:30:51.120
Of the norm of a divided by the
square root of N0 over 2.

00:30:51.120 --> 00:30:57.220
In other words it's the length
of a divided by the by the

00:30:57.220 --> 00:31:05.020
length of a one standard
deviation of the noise.

00:31:05.020 --> 00:31:12.260
When you put that in terms of,
well, if you write this as the

00:31:12.260 --> 00:31:16.220
square root of the norm squared
then you get this

00:31:16.220 --> 00:31:17.330
formula here.

00:31:17.330 --> 00:31:20.780
Because e sub b is just
the energy of

00:31:20.780 --> 00:31:22.690
these antipodal signals.

00:31:22.690 --> 00:31:25.580
When you look at this in terms
of the complex random

00:31:25.580 --> 00:31:28.860
variables, you get
the same thing.

00:31:28.860 --> 00:31:29.230
OK?

00:31:29.230 --> 00:31:33.740
You get u instead of a because,
in fact, in the

00:31:33.740 --> 00:31:37.110
complex plane and the real
plane, distances turn out to

00:31:37.110 --> 00:31:38.890
be the same.

00:31:38.890 --> 00:31:45.380
But again, in all cases it's
square root of 2eb over n0

00:31:45.380 --> 00:31:51.490
Now, that is true for any vector
at all where these

00:31:51.490 --> 00:31:53.450
norms are appropriate.

00:31:53.450 --> 00:31:58.910
When we start dealing with
functions what happens?

00:31:58.910 --> 00:32:01.910
When we start dealing with
functions, what we're going to

00:32:01.910 --> 00:32:04.680
do is we're going to take this
vector, we're going to turn it

00:32:04.680 --> 00:32:06.420
into a wave form.

00:32:06.420 --> 00:32:08.480
We're going to transmit
the wave form.

00:32:08.480 --> 00:32:10.650
The wave form is going
to come back to us.

00:32:10.650 --> 00:32:15.400
We're going to demodulate it,
get back to a number again.

00:32:15.400 --> 00:32:19.410
And that's the next thing that
I want to talk about.

00:32:19.410 --> 00:32:21.550
Because what I want to convince
you of is the

00:32:21.550 --> 00:32:26.150
property of white Gaussian noise
which is so important.

00:32:26.150 --> 00:32:31.990
Is that it doesn't make any
difference how you modulate.

00:32:31.990 --> 00:32:34.630
All modulation systems
are the same.

00:32:34.630 --> 00:32:36.990
All modulation schemes
are the same.

00:32:36.990 --> 00:32:39.110
All frequencies are the same.

00:32:39.110 --> 00:32:42.220
There is no way you can avoid
white Gaussian noise.

00:32:42.220 --> 00:32:44.550
There is no way you can
get screwed by it.

00:32:44.550 --> 00:32:48.610
No matter what you do, the
same thing happens.

00:32:48.610 --> 00:32:52.540
You can always take all these
problems where you're dealing

00:32:52.540 --> 00:32:53.540
with wave forms.

00:32:53.540 --> 00:32:58.070
You can convert them to finite
dimensional vector problems.

00:32:58.070 --> 00:33:00.410
And when you convert it into
a finite dimensional vector

00:33:00.410 --> 00:33:03.940
problem all of the orthonormal
functions that you're using

00:33:03.940 --> 00:33:05.590
all pass away.

00:33:05.590 --> 00:33:07.930
Because none of them are
relevant anymore.

00:33:07.930 --> 00:33:08.350
OK?

00:33:08.350 --> 00:33:11.580
That's the bottom line
of all of that.

00:33:14.100 --> 00:33:15.520
OK.

00:33:15.520 --> 00:33:20.370
We haven't really talked about
M-ARY hypothesis testing.

00:33:20.370 --> 00:33:22.590
So I want to talk about
it a little bit now.

00:33:22.590 --> 00:33:28.110
I talked about it just a
shade, but not much.

00:33:28.110 --> 00:33:32.010
When we want to detect between
m different hypothesis--

00:33:32.010 --> 00:33:34.560
namely in the vector case
we're going to now be

00:33:34.560 --> 00:33:37.860
detecting not between antipodal
signals but m

00:33:37.860 --> 00:33:41.910
signals which are placed
any place at all.

00:33:41.910 --> 00:33:47.610
We already said what the MAP,
optimal MAP test was.

00:33:47.610 --> 00:33:52.260
You see an observation, you're
trying to guess what the input

00:33:52.260 --> 00:33:55.380
was, or what the
hypothesis was.

00:33:55.380 --> 00:34:02.550
And in general, the MAP test
says try to find that j,

00:34:02.550 --> 00:34:06.970
namely that hypothesis, for
which the a priori probability

00:34:06.970 --> 00:34:11.570
of hypothesis j, times the
likelihood-- namely the

00:34:11.570 --> 00:34:14.910
probability that you
see v given h of

00:34:14.910 --> 00:34:19.130
v, given j is maximum.

00:34:19.130 --> 00:34:23.595
OK, this is just standard
formula for finding a

00:34:23.595 --> 00:34:25.330
posteriori probabilities.

00:34:25.330 --> 00:34:32.110
Where you factor out the
marginal on, when you cancel

00:34:32.110 --> 00:34:33.270
out the marginal on v.

00:34:33.270 --> 00:34:37.850
In other words, what this rule
says is to do MAP testing,

00:34:37.850 --> 00:34:41.930
what you do is you find the a
posteriori probabilities of

00:34:41.930 --> 00:34:45.260
each of the hypotheses and you
choose the a posteriori

00:34:45.260 --> 00:34:48.420
probability which is largest.

00:34:48.420 --> 00:34:50.540
Perfect common sense.

00:34:50.540 --> 00:34:54.130
The way we're going to do
that, the way which is

00:34:54.130 --> 00:34:58.330
particularly convenient, is you
do it the same way that

00:34:58.330 --> 00:35:01.810
we've been doing all along
for binary hypotheses.

00:35:01.810 --> 00:35:05.170
The way to do a MAP test, at
least one way to do a MAP

00:35:05.170 --> 00:35:09.980
test, is you compare every pair
of hypothesis and you

00:35:09.980 --> 00:35:12.180
choose the most likely
of each pair.

00:35:12.180 --> 00:35:15.290
And when you've got it all
done, you have a winner.

00:35:15.290 --> 00:35:16.310
OK?

00:35:16.310 --> 00:35:19.230
Mainly you can always compare
any objects if they're

00:35:19.230 --> 00:35:20.750
comparable.

00:35:20.750 --> 00:35:25.330
And after you compare each pair,
you take the one which

00:35:25.330 --> 00:35:28.240
beats every other one and
that's the winner.

00:35:28.240 --> 00:35:28.630
OK?

00:35:28.630 --> 00:35:33.900
So what you do is you do a
pairwise test between each

00:35:33.900 --> 00:35:34.790
hypothesis.

00:35:34.790 --> 00:35:38.920
Namely, the likelihood ratio
of m relative to m prime.

00:35:38.920 --> 00:35:43.670
Is the likelihood of the output
conditional on m,

00:35:43.670 --> 00:35:45.790
divided by the likelihood
of the output

00:35:45.790 --> 00:35:47.350
conditional on m prime.

00:35:47.350 --> 00:35:53.020
You compare it with the a priori
probabilities, and the

00:35:53.020 --> 00:35:56.820
point here is that nothing
really has been added.

00:35:56.820 --> 00:35:59.710
You have the same problem
you had before, OK?

00:35:59.710 --> 00:36:01.350
Nothing new.

00:36:01.350 --> 00:36:04.780
It's just gotten n square
times as complicated.

00:36:04.780 --> 00:36:09.870
The computation is free now, so
you have exactly the same

00:36:09.870 --> 00:36:12.840
problem that you had before.

00:36:12.840 --> 00:36:16.000
If you have to write it out on
paper, yeah it's much more

00:36:16.000 --> 00:36:17.050
complicated.

00:36:17.050 --> 00:36:20.600
But conceptually, it's not.

00:36:20.600 --> 00:36:23.070
You have to remember that
the signals are

00:36:23.070 --> 00:36:24.810
not antipodal here.

00:36:24.810 --> 00:36:28.740
But what we're dealing with
mostly at this point is this

00:36:28.740 --> 00:36:30.840
Gaussian noise case.

00:36:30.840 --> 00:36:35.490
And here, what you observe,
is signal plus noise.

00:36:35.490 --> 00:36:41.800
And Z is zero-mean jointly Gauss
and s is discrete with n

00:36:41.800 --> 00:36:43.530
possible values.

00:36:43.530 --> 00:36:47.540
OK, so let's see what
that means.

00:36:47.540 --> 00:36:48.790
Here's a picture of it.

00:36:51.230 --> 00:36:54.890
If you have three singles which
are each two dimensional

00:36:54.890 --> 00:36:59.170
vectors, suppose one of them is
s0, suppose one of them is

00:36:59.170 --> 00:37:02.880
s1, suppose one of them s2.

00:37:02.880 --> 00:37:04.320
OK?

00:37:04.320 --> 00:37:09.570
And now you want to pairwise,
see which one is most likely.

00:37:09.570 --> 00:37:12.850
And let's think of doing this
first for the maximum

00:37:12.850 --> 00:37:14.090
likelihood case.

00:37:14.090 --> 00:37:15.100
What do you do?

00:37:15.100 --> 00:37:21.200
You set up a perpendicular
bisector between s0 and s1.

00:37:21.200 --> 00:37:23.940
That's this line here.

00:37:23.940 --> 00:37:27.550
And if you weren't to worry
about s2, everything on this

00:37:27.550 --> 00:37:31.190
side would go into
H equals zero.

00:37:31.190 --> 00:37:34.620
And everything on this side
would go into H equals one.

00:37:34.620 --> 00:37:37.150
Namely, whatever's closest
to this point

00:37:37.150 --> 00:37:38.350
gets mapped into it.

00:37:38.350 --> 00:37:41.770
Whatever's closest to this point
gets mapped into it.

00:37:41.770 --> 00:37:46.160
If you're doing MAP testing,
what happens?

00:37:46.160 --> 00:37:52.820
In the test between this and
this you had the same

00:37:52.820 --> 00:37:56.320
orientation for this line, but
it just gets shifted a little

00:37:56.320 --> 00:38:00.140
bit this way or a little
bit this way.

00:38:00.140 --> 00:38:00.630
OK?

00:38:00.630 --> 00:38:02.720
Then you compare this
with this and you

00:38:02.720 --> 00:38:05.210
get this line here.

00:38:05.210 --> 00:38:07.110
Same argument as before.

00:38:07.110 --> 00:38:11.000
It's just comparing two things
are not antipodal they've just

00:38:11.000 --> 00:38:13.730
been shifted off from the
origin a little bit.

00:38:13.730 --> 00:38:16.470
But for the maximum likelihood
you still take the

00:38:16.470 --> 00:38:19.430
perpendicular bisector
between them.

00:38:19.430 --> 00:38:22.770
And then you compare these two
and you got a perpendicular

00:38:22.770 --> 00:38:24.410
bisector between those.

00:38:26.970 --> 00:38:30.050
And these perpendicular
bisectors, in two dimensions,

00:38:30.050 --> 00:38:32.660
always come together
at one point.

00:38:32.660 --> 00:38:33.920
And I don't know why.

00:38:38.010 --> 00:38:40.430
And if you looked at it often
enough, you probably know why.

00:38:40.430 --> 00:38:43.230
And you could probably prove
it in about ten minutes.

00:38:43.230 --> 00:38:46.740
But in fact these things always
come together somehow.

00:38:46.740 --> 00:38:49.830
If you do the MAP test, they
always come together also.

00:38:49.830 --> 00:38:54.670
You can shift each of them in
arbitrary ways and somehow

00:38:54.670 --> 00:38:58.230
they always come together
in this point.

00:38:58.230 --> 00:39:02.410
OK, the separators between
decision regions here are the

00:39:02.410 --> 00:39:08.290
set of points where the real
part of the inner product, vu,

00:39:08.290 --> 00:39:09.060
is constant.

00:39:09.060 --> 00:39:09.430
OK?

00:39:09.430 --> 00:39:13.670
Again, for dealing with complex
vectors, you got to

00:39:13.670 --> 00:39:17.020
both do the projection and then
do the projection again

00:39:17.020 --> 00:39:19.550
onto the real part of
this projection.

00:39:19.550 --> 00:39:21.580
So it's sort of a two
way projection.

00:39:21.580 --> 00:39:27.220
Because probability densities in
j dimensional complex space

00:39:27.220 --> 00:39:30.380
are really 2j dimensional
quantities.

00:39:30.380 --> 00:39:32.690
And when you're comparing
them, you really have to

00:39:32.690 --> 00:39:35.610
compare things in terms
of that 2j dimensional

00:39:35.610 --> 00:39:38.060
probability density.

00:39:38.060 --> 00:39:40.890
OK so that's why
that comes out.

00:39:40.890 --> 00:39:43.620
These are best visualized in
separate, real, and imaginary

00:39:43.620 --> 00:39:44.110
coordinates.

00:39:44.110 --> 00:39:48.850
And for maximum likelihood
detection, the regions are

00:39:48.850 --> 00:39:50.360
Voronoi regions.

00:39:50.360 --> 00:39:50.600
OK?

00:39:50.600 --> 00:39:58.520
We talked about Voronoi regions
in terms of doing

00:39:58.520 --> 00:40:00.010
quantization.

00:40:00.010 --> 00:40:02.840
And we found out if you wanted
to minimize the mean square

00:40:02.840 --> 00:40:06.620
error, what you did was you
set up regions, which are

00:40:06.620 --> 00:40:10.020
perpendicular bisectors between
all the points.

00:40:10.020 --> 00:40:12.810
And here you get the same
perpendicular bisectors

00:40:12.810 --> 00:40:14.880
between the points.

00:40:14.880 --> 00:40:16.800
And everybody-- because
of that-- thinks that

00:40:16.800 --> 00:40:21.740
quantization has a great deal
to do with error probability

00:40:21.740 --> 00:40:24.430
when you have large
sets of signals.

00:40:24.430 --> 00:40:28.230
And it probably has something
to do with it, but I don't

00:40:28.230 --> 00:40:30.530
know what other than the fact
that you've got Voronoi

00:40:30.530 --> 00:40:33.900
regions in each case.

00:40:33.900 --> 00:40:35.150
Which is what you get.

00:40:45.060 --> 00:40:55.440
OK, so that's where we are with
both complex vectors and

00:40:55.440 --> 00:40:56.640
real vectors.

00:40:56.640 --> 00:40:59.480
I want to now just restrict
attention to real wave forms

00:40:59.480 --> 00:41:02.770
so I don't have to keep going
back and forth between the

00:41:02.770 --> 00:41:04.640
real and imaginary case.

00:41:04.640 --> 00:41:07.800
If you're thinking in terms of
QAM, we're now thinking in

00:41:07.800 --> 00:41:10.920
terms of what goes
on at passband.

00:41:10.920 --> 00:41:13.300
Why do we want to think of
what goes on at passband?

00:41:13.300 --> 00:41:16.890
Because that's where
the noise hits us.

00:41:16.890 --> 00:41:21.620
And in a fundamental sense,
all of the stuff about QAM

00:41:21.620 --> 00:41:24.250
really isn't fundamental.

00:41:24.250 --> 00:41:27.520
I mean, it's all done-- all this
stuff down at passband--

00:41:27.520 --> 00:41:30.170
is all done because people
thought it was easier to

00:41:30.170 --> 00:41:32.020
implement things there.

00:41:32.020 --> 00:41:36.830
It's the only reason for all of
that mess with dealing with

00:41:36.830 --> 00:41:39.540
all of these complex signals.

00:41:39.540 --> 00:41:42.240
If we really want to deal with
the problem in a fundamental

00:41:42.240 --> 00:41:44.600
way, what we want to
do is to choose a

00:41:44.600 --> 00:41:47.220
signal set up at passband.

00:41:47.220 --> 00:41:49.890
Do detection up at passband.

00:41:49.890 --> 00:41:52.620
And then after we find out what
the optimal detection is

00:41:52.620 --> 00:41:56.430
up at passband, see if we can
actually implement that down

00:41:56.430 --> 00:41:58.200
at baseband.

00:41:58.200 --> 00:42:01.330
So the fundamental problem is
looking at single sets up at

00:42:01.330 --> 00:42:06.080
passband and analyze
what they all mean.

00:42:06.080 --> 00:42:11.290
OK, so we're going to generalize
both PAM and QAM.

00:42:11.290 --> 00:42:14.590
And now we're going to look at
the general problem where what

00:42:14.590 --> 00:42:18.580
we're dealing with
is a single set.

00:42:18.580 --> 00:42:21.020
Which is m signals.

00:42:21.020 --> 00:42:24.760
Each of them we're going to
visualize as a vector in j

00:42:24.760 --> 00:42:28.340
dimensional space. m different
signals, j dimensional space.

00:42:28.340 --> 00:42:31.590
Don't confuse the dimension
of space with

00:42:31.590 --> 00:42:33.420
the number of signals.

00:42:33.420 --> 00:42:33.770
OK?

00:42:33.770 --> 00:42:36.760
You can have an arbitrarily
large dimensional space and

00:42:36.760 --> 00:42:38.770
just binary signals.

00:42:38.770 --> 00:42:42.670
Or you can have an arbitrarily
large set of signals and you

00:42:42.670 --> 00:42:43.970
can be dealing with it.

00:42:43.970 --> 00:42:48.050
In PAM, for example, it's just
all done in one dimension.

00:42:48.050 --> 00:42:50.110
So j there is equal to 1.

00:42:50.110 --> 00:42:52.600
QAM j is equal to 2.

00:42:52.600 --> 00:42:54.540
We now want to look at
more general things.

00:42:54.540 --> 00:42:58.110
Partly because we want to look
at orthoginal wave forms.

00:42:58.110 --> 00:43:02.080
We want to look at orthoginal
wave forms for two reasons.

00:43:02.080 --> 00:43:05.890
One is that we would like to
show that by using orthoginal

00:43:05.890 --> 00:43:10.280
wave forms, you can reach what
we've called the capacity of a

00:43:10.280 --> 00:43:12.970
white Gaussian noise channel.

00:43:12.970 --> 00:43:17.480
And two, when we get to studying
wireless it's very,

00:43:17.480 --> 00:43:21.700
very useful to base
signal sets on

00:43:21.700 --> 00:43:23.510
orthonormal sets of functions.

00:43:23.510 --> 00:43:25.820
And we'll see why each
of those things

00:43:25.820 --> 00:43:28.710
happen as we move on.

00:43:28.710 --> 00:43:31.890
OK so we're going to denote the
single set as the set of

00:43:31.890 --> 00:43:35.170
vectors, a1 up to a sub m.

00:43:35.170 --> 00:43:38.480
And in that signal set, we will
denote the vector, a sub

00:43:38.480 --> 00:43:44.116
m, as a j dimensional vector,
a sub m1, a sub m2, up to a

00:43:44.116 --> 00:43:45.960
sub m capital j.

00:43:45.960 --> 00:43:47.950
So j is at the dimension.

00:43:47.950 --> 00:43:52.810
m is just a component
of these vectors.

00:43:52.810 --> 00:43:55.540
I'm going to create
a set of capital J

00:43:55.540 --> 00:43:57.930
orthonormal wave forms.

00:43:57.930 --> 00:43:59.150
They can be anything at all.

00:43:59.150 --> 00:44:02.220
I don't care what they are.

00:44:02.220 --> 00:44:05.080
I'm going to use those
orthonormal wave forms in

00:44:05.080 --> 00:44:07.290
order to modulate the signal--

00:44:07.290 --> 00:44:09.310
which is now a vector--

00:44:09.310 --> 00:44:13.240
up to some waveform.

00:44:13.240 --> 00:44:17.910
This is really the standard way
we've been turning signals

00:44:17.910 --> 00:44:20.820
into waveforms all along.

00:44:20.820 --> 00:44:23.910
It's just that now we're looking
at the general case

00:44:23.910 --> 00:44:26.820
instead of all of these specific
cases that we've been

00:44:26.820 --> 00:44:27.370
looking at.

00:44:27.370 --> 00:44:34.370
All of the special cases all fit
into this same category.

00:44:34.370 --> 00:44:38.620
So we have these capital
M different waveforms.

00:44:38.620 --> 00:44:45.110
And we're going to transmit one
of them and then at the

00:44:45.110 --> 00:44:49.370
receiver we're going to try to
decide which one was sent.

00:44:49.370 --> 00:44:52.240
OK, well one of the reasons why
I'm going through all of

00:44:52.240 --> 00:44:57.580
this generality is that there's
an issue we haven't

00:44:57.580 --> 00:45:00.210
talked about yet.

00:45:00.210 --> 00:45:05.020
All of the stuff we've done on
detection so far we have had

00:45:05.020 --> 00:45:07.030
one hypothesis.

00:45:07.030 --> 00:45:09.870
Could be M-ary could
be binary.

00:45:09.870 --> 00:45:11.500
We have sent something.

00:45:11.500 --> 00:45:13.190
We have received something.

00:45:13.190 --> 00:45:14.890
We have made a detection.

00:45:14.890 --> 00:45:15.270
OK?

00:45:15.270 --> 00:45:18.780
In other words, for all of the
antipodal stuff we've done, we

00:45:18.780 --> 00:45:20.640
built a communication system.

00:45:20.640 --> 00:45:22.230
We set it all up.

00:45:22.230 --> 00:45:24.400
We transmitted one bit.

00:45:24.400 --> 00:45:25.980
We've received the one bit.

00:45:25.980 --> 00:45:27.570
We've made a decision on it.

00:45:27.570 --> 00:45:29.890
Then we've torn down the
communication system

00:45:29.890 --> 00:45:31.930
and we've gone home.

00:45:31.930 --> 00:45:35.130
You really want to transmit a
whole sequence of symbols or

00:45:35.130 --> 00:45:37.870
signals or waveforms.

00:45:37.870 --> 00:45:42.600
So we want to deal
with that now.

00:45:42.600 --> 00:45:45.040
So we need some way
to transmit a

00:45:45.040 --> 00:45:48.770
succession of M-ary signals.

00:45:48.770 --> 00:45:52.930
And we'll call this succession
of signals-- mainly the

00:45:52.930 --> 00:45:55.780
signals are the things that
get chosen from the signal

00:45:55.780 --> 00:45:59.210
set-- we'll call them
x of k, k of z.

00:45:59.210 --> 00:46:01.030
Which is what we've been
calling them all along.

00:46:01.030 --> 00:46:04.540
We've been transmitting a
sequence of things when we're

00:46:04.540 --> 00:46:05.430
dealing with PAM.

00:46:05.430 --> 00:46:07.880
And aa in am.

00:46:07.880 --> 00:46:12.400
Why do I call them
xk instead of ak?

00:46:15.740 --> 00:46:18.940
Well I can't call them ak
because when I talk about ak

00:46:18.940 --> 00:46:23.000
I'm talking about the k'th
signal in the signal set.

00:46:23.000 --> 00:46:26.940
And here what I'm talking about
now is transmitting a

00:46:26.940 --> 00:46:28.940
sequence of choices.

00:46:28.940 --> 00:46:32.200
Each one of these choices, the
first choice is a choice from

00:46:32.200 --> 00:46:33.340
this set here.

00:46:33.340 --> 00:46:36.920
The second thing that I send
is a choice from this set.

00:46:36.920 --> 00:46:40.030
The third thing that I send
is a choice from this set.

00:46:40.030 --> 00:46:46.610
So x1, x2, x3, and x4 and so
forth are different choices

00:46:46.610 --> 00:46:48.870
among these M-ary signals.

00:46:48.870 --> 00:46:52.680
If m is 2 to the 6--

00:46:52.680 --> 00:46:55.770
OK in other words, every time
I transmit a signal I'm

00:46:55.770 --> 00:46:58.140
transmitting six bits.

00:46:58.140 --> 00:46:58.560
OK.

00:46:58.560 --> 00:47:01.980
In a communication system
we transmit six bits.

00:47:01.980 --> 00:47:04.060
Then we transmit another
six bits.

00:47:04.060 --> 00:47:09.200
Then we transmit another six
bits, and so on forever.

00:47:09.200 --> 00:47:14.030
OK, so I need to talk about
these things as ways of a

00:47:14.030 --> 00:47:15.610
succession of signals.

00:47:15.610 --> 00:47:19.670
The thing that I'm trying to
get at is, how do you know

00:47:19.670 --> 00:47:23.940
when you send one of these
signals that you don't have to

00:47:23.940 --> 00:47:27.180
worry about the other signals?

00:47:27.180 --> 00:47:28.450
How do you know that they don't

00:47:28.450 --> 00:47:29.660
interfere with each other?

00:47:29.660 --> 00:47:33.870
Well, we sort of solved the
problem of them interfering

00:47:33.870 --> 00:47:37.370
with each other in dealing with
Nyquist, but we haven't

00:47:37.370 --> 00:47:40.580
dealt with that problem at all
since we started to talk about

00:47:40.580 --> 00:47:42.180
random processes.

00:47:42.180 --> 00:47:45.350
So we don't know whether we've
really solved it or not.

00:47:45.350 --> 00:47:48.950
So at this point we have to
solve that problem, and that's

00:47:48.950 --> 00:47:50.200
what we're aiming at.

00:47:52.530 --> 00:47:57.190
So, the one way to be able to
transmit a whole sequence of

00:47:57.190 --> 00:48:02.510
signals is to have these choices
of vectors here and to

00:48:02.510 --> 00:48:07.210
develop a set of orthonormal
waveforms, v1 up to v sub j,

00:48:07.210 --> 00:48:13.040
which all have the property that
if you time shift them

00:48:13.040 --> 00:48:18.180
each by capital T,
they're stilll--

00:48:18.180 --> 00:48:20.950
if you time shift them by
capital T, they have to be

00:48:20.950 --> 00:48:22.900
orthonormal to each other.

00:48:22.900 --> 00:48:25.760
The question you're facing is
whether these things that

00:48:25.760 --> 00:48:28.400
you're transmitting are
orthoganol to all of these

00:48:28.400 --> 00:48:30.700
things that you're
transmitting.

00:48:30.700 --> 00:48:33.890
Now, in the Nyquist problem, we
dealt with the problem of

00:48:33.890 --> 00:48:38.470
how do you take one waveform
here and make it orthonormal

00:48:38.470 --> 00:48:40.040
to all of its time shifts.

00:48:40.040 --> 00:48:42.270
And we solved that problem.

00:48:42.270 --> 00:48:44.520
In the quiz you solved the
problem-- although you

00:48:44.520 --> 00:48:46.860
probably didn't recognize it--

00:48:46.860 --> 00:48:49.510
of dealing with orthonormal
functions both

00:48:49.510 --> 00:48:51.320
in time and in frequency.

00:48:51.320 --> 00:48:54.200
And that's the kind of thing
we would like to use here.

00:48:54.200 --> 00:48:57.990
If I take a set of orthonormal
pulses and then I modulate

00:48:57.990 --> 00:49:01.590
those orthonormal pulses up
to a higher frequency--

00:49:01.590 --> 00:49:04.560
which is out of the range of
this first frequency-- then I

00:49:04.560 --> 00:49:08.140
can send one sequence of
orthonormal functions down

00:49:08.140 --> 00:49:11.070
here, another set of orthonormal
functions up here

00:49:11.070 --> 00:49:12.810
in a different frequency
range.

00:49:12.810 --> 00:49:14.930
Another one up here in
a different frequency

00:49:14.930 --> 00:49:16.340
range and so forth.

00:49:16.340 --> 00:49:19.370
So then all of these orthonormal
functions are

00:49:19.370 --> 00:49:21.170
going to be orthonormal
to each other.

00:49:21.170 --> 00:49:21.580
Yeah?

00:49:21.580 --> 00:49:27.355
AUDIENCE: Are you saying
that each x of

00:49:27.355 --> 00:49:30.260
k is its own frequency?

00:49:30.260 --> 00:49:30.560
Because each x of k is
infinitely long.

00:49:30.560 --> 00:49:35.120
PROFESSOR: Each x of
k is going to--

00:49:35.120 --> 00:49:41.880
each x of k is just a vector
of j components.

00:49:41.880 --> 00:49:45.500
I'm going to modulate that
vector, x of k, into a

00:49:45.500 --> 00:49:51.230
waveform, x of t, which might be
finite duration or it might

00:49:51.230 --> 00:49:53.020
be infinite duration.

00:49:53.020 --> 00:49:57.670
I mean, it's going to go to zero
very, very fast, anyway.

00:49:57.670 --> 00:50:01.200
And whether it is absolutely
time limited or not is

00:50:01.200 --> 00:50:04.550
something I don't really care
about at this point.

00:50:04.550 --> 00:50:09.770
But the point is I can create
functions where, in fact, I

00:50:09.770 --> 00:50:13.240
have a whole sequence of
functions here and they're

00:50:13.240 --> 00:50:15.420
orthonormal to all
of the shifts.

00:50:15.420 --> 00:50:22.390
One way of doing this, for
example, is to make capital J

00:50:22.390 --> 00:50:25.540
a bunch of little time
shifts on functions.

00:50:25.540 --> 00:50:29.610
I can pick a function p of t,
which is orthonormal to all of

00:50:29.610 --> 00:50:33.240
its shifts, in terms of t1.

00:50:33.240 --> 00:50:38.530
I can send J of those pulses
to take care of x of k.

00:50:38.530 --> 00:50:43.000
And then I can use a capital T
in here, which is j times this

00:50:43.000 --> 00:50:44.730
little t that I was using.

00:50:44.730 --> 00:50:47.700
And I can send another
set of functions.

00:50:47.700 --> 00:50:51.720
So I can do that, I can move
up and down in frequency.

00:50:51.720 --> 00:50:53.710
I can choose any old
set of orthonormal

00:50:53.710 --> 00:50:55.460
functions that I want to.

00:50:55.460 --> 00:50:59.910
But the thing that I want to do
is I want to make sure that

00:50:59.910 --> 00:51:08.910
for each vector, x of k, that
I'm sending in time when I

00:51:08.910 --> 00:51:14.140
modulate it to a waveform, that
waveform is orthonormal

00:51:14.140 --> 00:51:16.860
to the waveforms for
every other k.

00:51:16.860 --> 00:51:18.970
And there are lots of
ways of doing that.

00:51:18.970 --> 00:51:21.190
OK, mainly there are
lots of choices

00:51:21.190 --> 00:51:23.900
of orthonormal functions.

00:51:23.900 --> 00:51:28.670
OK so anyway what I'm going to
be doing is making all of

00:51:28.670 --> 00:51:31.480
these signals orthogonal
to each other.

00:51:36.350 --> 00:51:39.880
OK, so the transmitting waveform
for this sequence of

00:51:39.880 --> 00:51:45.980
modulated signals is x of t,
which is the sum of x of k

00:51:45.980 --> 00:51:47.410
times t minus kt.

00:51:47.410 --> 00:51:50.860
Mainly the same thing we
were doing before.

00:51:50.860 --> 00:51:54.920
Except now I have also the
problem that each of these

00:51:54.920 --> 00:51:59.560
waveforms, x of k of t,
has to be some sum

00:51:59.560 --> 00:52:01.340
of orthonormal functions.

00:52:01.340 --> 00:52:03.380
So the problem becomes a little
more difficult than

00:52:03.380 --> 00:52:05.380
what it was before.

00:52:05.380 --> 00:52:06.570
But in fact it's--

00:52:06.570 --> 00:52:09.140
I mean this is just standard
communication.

00:52:09.140 --> 00:52:11.290
Every wireless system
in the world uses

00:52:11.290 --> 00:52:12.610
this kind of scheme.

00:52:12.610 --> 00:52:14.700
They don't use QAM or PAM.

00:52:14.700 --> 00:52:19.320
They use something much
more like this.

00:52:19.320 --> 00:52:23.780
OK, so now our problem is you
want to detect a generic x

00:52:23.780 --> 00:52:25.680
from this sequence.

00:52:25.680 --> 00:52:30.020
OK, in other words, one of these
x sub k in sequence, we

00:52:30.020 --> 00:52:33.670
want to be able to detect
what signal was sent.

00:52:33.670 --> 00:52:38.920
We want to detect which
hypothesis chose a signal

00:52:38.920 --> 00:52:42.940
which was then formed into
a waveform, x of k of t.

00:52:42.940 --> 00:52:45.180
And if I can do this
for one k, I can do

00:52:45.180 --> 00:52:46.720
it for all of them.

00:52:46.720 --> 00:52:51.630
So I want to solve the problem
for one generic value of k.

00:52:51.630 --> 00:52:53.690
OK, how is this problem
different from what I was

00:52:53.690 --> 00:52:55.120
looking at before?

00:52:55.120 --> 00:52:57.830
Before I was looking at the
problem where we built a

00:52:57.830 --> 00:53:02.450
communication system, we tuned
it all up, we sent one bit.

00:53:02.450 --> 00:53:06.300
We detected it, we tore it all
down and we went home.

00:53:06.300 --> 00:53:08.460
Now what we're doing is
we're building the

00:53:08.460 --> 00:53:09.680
communication system.

00:53:09.680 --> 00:53:10.870
We're tuning at all up.

00:53:10.870 --> 00:53:14.470
We're sending a sequence
of bits.

00:53:14.470 --> 00:53:18.860
And then all I'm interested in
at the moment is detecting the

00:53:18.860 --> 00:53:20.500
k'th of them.

00:53:20.500 --> 00:53:23.870
But if I find a way to detect
the k'th of them, I can then

00:53:23.870 --> 00:53:25.740
use it for every k.

00:53:25.740 --> 00:53:26.060
OK?

00:53:26.060 --> 00:53:29.450
So I'm going to build a detector
which is going to

00:53:29.450 --> 00:53:33.100
detect, in some optimal
way, each one of these

00:53:33.100 --> 00:53:35.030
signals that gets sent.

00:53:35.030 --> 00:53:35.300
OK?

00:53:35.300 --> 00:53:37.400
Is it clear how the problem
is different?

00:53:37.400 --> 00:53:40.000
Mainly I have to deal with the
fact that these other signals

00:53:40.000 --> 00:53:42.240
are floating around there.

00:53:42.240 --> 00:53:44.780
And that's my problem.

00:53:44.780 --> 00:53:49.380
Ok, so the input to the channel
is hypothesis H. That

00:53:49.380 --> 00:53:52.220
takes values one up the m.

00:53:52.220 --> 00:53:56.950
The symbol, m, is mapped into
the signal, vector a sub m,

00:53:56.950 --> 00:54:02.716
it's modulated into x of t
equals summation over j, a sub

00:54:02.716 --> 00:54:05.110
mj, phi j of t.

00:54:05.110 --> 00:54:09.980
OK, this waveform, now, is a
function of which particular

00:54:09.980 --> 00:54:11.380
signal I'm sending.

00:54:11.380 --> 00:54:15.040
Which is a function of which
particular hypothesis entered

00:54:15.040 --> 00:54:16.290
the encoder.

00:54:19.360 --> 00:54:25.860
The trouble with this material
is all the complication comes

00:54:25.860 --> 00:54:28.810
in this awful notation,
which you can't avoid.

00:54:28.810 --> 00:54:30.790
Because you're dealing with
sequences, you're dealing with

00:54:30.790 --> 00:54:33.860
vectors, and you're dealing
with wave forms

00:54:33.860 --> 00:54:35.820
all at the same time.

00:54:35.820 --> 00:54:40.070
What's going on, after you
understand it, you'll say why

00:54:40.070 --> 00:54:43.000
was it so difficult to
understand this?

00:54:43.000 --> 00:54:45.550
Because eventually when you see
it, it becomes very, very

00:54:45.550 --> 00:54:51.630
simple And I understand why
there's just too much stuff

00:54:51.630 --> 00:54:54.970
all going on at the same time.

00:54:54.970 --> 00:54:57.500
OK, so what I'm going to do now
is I'm going to take these

00:54:57.500 --> 00:55:02.420
J, capital J, orthonormal
waveforms.

00:55:02.420 --> 00:55:06.030
And we've already seen that you
can start out with any old

00:55:06.030 --> 00:55:09.990
orthonormal waveforms and if
you want to you can extend

00:55:09.990 --> 00:55:13.790
that set of waveforms into
an orthonormal set that

00:55:13.790 --> 00:55:16.530
spans all of L2.

00:55:16.530 --> 00:55:16.930
OK?

00:55:16.930 --> 00:55:20.030
So I'm going to imagine
that we've done that.

00:55:20.030 --> 00:55:23.050
It's taken us a long time,
but we've done it.

00:55:23.050 --> 00:55:24.320
We're all through with it.

00:55:24.320 --> 00:55:27.070
We have this orthonormal
set now.

00:55:27.070 --> 00:55:33.080
If I'm smart, that orthonormal
set, which I generated, will

00:55:33.080 --> 00:55:36.380
also include easy ways to
represent each of the other

00:55:36.380 --> 00:55:38.490
signals that we're
going to send.

00:55:38.490 --> 00:55:40.150
But I don't care about
that right now.

00:55:40.150 --> 00:55:44.200
All I'm dealing with is this one
hypothesis that came in.

00:55:44.200 --> 00:55:48.120
This one signal, a sub m--

00:55:50.720 --> 00:55:56.170
oh, the hypothesis m, the
signal a sub m, an the

00:55:56.170 --> 00:56:00.060
particular time instant, k, and
this waveform that gets

00:56:00.060 --> 00:56:05.490
sent, which can be represented
as the first J terms in this

00:56:05.490 --> 00:56:08.860
orthonormal sequence.

00:56:08.860 --> 00:56:12.010
OK, so what I'm going
to get then is the

00:56:12.010 --> 00:56:14.550
received random process.

00:56:14.550 --> 00:56:16.900
Is going to be a sum --

00:56:16.900 --> 00:56:19.040
and forgot about the
j prime now--

00:56:19.040 --> 00:56:24.890
I can represent it as a sum of
coefficients times these

00:56:24.890 --> 00:56:27.240
orthonormal waveforms.

00:56:27.240 --> 00:56:31.980
OK, that's what we've done for
arbitrary sequences, and then

00:56:31.980 --> 00:56:35.920
we've said we can also do it
for at least well defined

00:56:35.920 --> 00:56:38.380
random processes.

00:56:38.380 --> 00:56:41.560
I'm going to make, I mean,
instead of making this an

00:56:41.560 --> 00:56:45.880
infinite dimensional sum, I
want to make it a finite

00:56:45.880 --> 00:56:51.030
dimensional sum where J prime
is very, very large.

00:56:51.030 --> 00:56:53.520
Say, 10 to the fiftieth
if you want to.

00:56:53.520 --> 00:56:55.660
I don't want to make it
infinite, I want to look at

00:56:55.660 --> 00:56:59.280
what happens when I let it
get bigger or smaller.

00:56:59.280 --> 00:57:05.380
So I'm expanding Y of t over
an orthonormal expansion.

00:57:05.380 --> 00:57:07.700
But I'm not going all the way.

00:57:07.700 --> 00:57:13.160
I'm just going to try to do
maximum likelihood detection

00:57:13.160 --> 00:57:16.270
with this finite set
of observations.

00:57:16.270 --> 00:57:18.190
So I wont do quite as well
as if I have all the

00:57:18.190 --> 00:57:20.360
observations, but I'll
still do pretty well.

00:57:20.360 --> 00:57:22.120
We hope.

00:57:22.120 --> 00:57:26.200
OK, well so Y sub j--

00:57:26.200 --> 00:57:30.940
the output that I see in
this degree of freedom

00:57:30.940 --> 00:57:34.100
corresponding to
phi sub j of t.

00:57:34.100 --> 00:57:37.840
Is going to be xj-- what I
sent in that degree of

00:57:37.840 --> 00:57:40.130
freedom-- plus zj.

00:57:40.130 --> 00:57:43.270
And there are j degrees of--
capital J degrees of freedom

00:57:43.270 --> 00:57:44.270
that I'm using.

00:57:44.270 --> 00:57:50.010
So the outputs in those degrees
of freedom, namely in

00:57:50.010 --> 00:57:55.370
the phi1 of t, phi2 of t, phi3
of t directions in this L2

00:57:55.370 --> 00:57:59.960
space are going to be the
signal plus the noise.

00:57:59.960 --> 00:58:01.760
For all of these dimensions.

00:58:01.760 --> 00:58:05.810
And Yj is just going to
be equal to zj for

00:58:05.810 --> 00:58:08.440
all the other terms.

00:58:08.440 --> 00:58:12.160
OK, now I want to add one
extra thing here.

00:58:12.160 --> 00:58:16.810
What I should be putting in here
is all the other signals

00:58:16.810 --> 00:58:18.820
that are going to
be transmitted.

00:58:18.820 --> 00:58:21.430
I don't know how to do that.

00:58:21.430 --> 00:58:24.100
Notationally it gets
very confusing.

00:58:24.100 --> 00:58:29.190
So what I'm going to say is,
OK Z sub j here is not just

00:58:29.190 --> 00:58:31.420
Gaussian noise.

00:58:31.420 --> 00:58:37.190
Z sub j is Gaussian noise plus
all the signals from other

00:58:37.190 --> 00:58:40.410
time instance that
we're sending.

00:58:40.410 --> 00:58:43.170
Plus all of the signals that
anybody else is sending.

00:58:43.170 --> 00:58:44.910
If we're dealing with wireless
then we have

00:58:44.910 --> 00:58:46.920
interference from them.

00:58:46.920 --> 00:58:50.480
Plus any old other thing you
can think of. z sub j is

00:58:50.480 --> 00:58:55.360
everything but in these other
degrees of freedom.

00:58:55.360 --> 00:58:58.060
In these other coordinates.

00:58:58.060 --> 00:59:01.140
This solves another problem
for us, because when we

00:59:01.140 --> 00:59:05.410
defined white Gaussian noise
we had this problem.

00:59:05.410 --> 00:59:09.300
That we could only say it looked
white over the region

00:59:09.300 --> 00:59:10.470
of interest.

00:59:10.470 --> 00:59:16.220
We could only say it looked
white over some time span.

00:59:16.220 --> 00:59:19.440
Because the earth keeps
changing, you know.

00:59:19.440 --> 00:59:23.700
And over some frequency span
because different frequencies

00:59:23.700 --> 00:59:26.170
behave in different ways.

00:59:26.170 --> 00:59:32.040
So this also allows us to have
different Gaussian random

00:59:32.040 --> 00:59:32.740
variables here.

00:59:32.740 --> 00:59:35.890
So when we have arbitrary random
variables here, they

00:59:35.890 --> 00:59:38.180
can be Gaussian or
non-Gaussian.

00:59:38.180 --> 00:59:40.910
They don't have to have
the same variance.

00:59:40.910 --> 00:59:43.860
They don't have to
have anything.

00:59:43.860 --> 00:59:48.320
What I am going to assume is
that these out of band, out of

00:59:48.320 --> 00:59:53.740
sense, out of view random
variables are all independent

00:59:53.740 --> 00:59:56.880
of the things that
I am looking at.

00:59:56.880 --> 00:59:59.710
And for white Gaussian
noise, that's true.

00:59:59.710 --> 01:00:03.610
All of these random variables
here are independent of all of

01:00:03.610 --> 01:00:05.640
these random variables here.

01:00:05.640 --> 01:00:09.380
For these first capital J
different random variables

01:00:09.380 --> 01:00:10.790
that I'm interested in.

01:00:10.790 --> 01:00:16.280
And all these random variables
are independent of they input

01:00:16.280 --> 01:00:17.910
that I'm using.

01:00:17.910 --> 01:00:18.150
OK?

01:00:18.150 --> 01:00:24.380
In other words, a hypothesis
came into the transmitter that

01:00:24.380 --> 01:00:26.310
generated a signal.

01:00:26.310 --> 01:00:29.870
The signal got turned into a
waveform, which is defined

01:00:29.870 --> 01:00:34.800
solely in terms of these J
degrees of freedom, these J

01:00:34.800 --> 01:00:38.000
orthonormal functions.

01:00:38.000 --> 01:00:43.370
And now everything everywhere
else is independent of these J

01:00:43.370 --> 01:00:47.300
functions that I'm
interested in.

01:00:47.300 --> 01:00:49.130
Why is that a shaky
assumption?

01:00:52.050 --> 01:00:53.790
Anybody think of a
situation where

01:00:53.790 --> 01:00:55.410
that is absolute nonsense?

01:00:59.540 --> 01:00:59.790
Yeah?

01:00:59.790 --> 01:01:08.660
AUDIENCE: The stuff from the
other message had t--

01:01:08.660 --> 01:01:14.940
PROFESSOR: Mm hmm.

01:01:14.940 --> 01:01:16.650
AUDIENCE: You said t of j is
not just Gaussian noise--

01:01:16.650 --> 01:01:21.130
PROFESSOR: It also includes all
those other signals, yes.

01:01:21.130 --> 01:01:23.060
Well, but I want to assume that
those other signals are

01:01:23.060 --> 01:01:27.210
independent of this particular
signal that I'm sending.

01:01:27.210 --> 01:01:30.870
But in fact that is making
a pretty big assumption.

01:01:30.870 --> 01:01:34.660
Because one of the things that a
lot of people like to do is,

01:01:34.660 --> 01:01:38.640
when these bits come into a
channel, the first thing they

01:01:38.640 --> 01:01:42.500
do is they encode the bits
for error correction.

01:01:42.500 --> 01:01:45.440
And then they take those bits
that come out of the error

01:01:45.440 --> 01:01:50.020
correction device, error
encoding device, which are as

01:01:50.020 --> 01:01:52.260
correlated as could be.

01:01:52.260 --> 01:01:55.090
And they're all statistically
very dependent, because we

01:01:55.090 --> 01:01:57.080
want to use that statistical
dependence

01:01:57.080 --> 01:02:00.520
later to correct errors.

01:02:00.520 --> 01:02:03.670
And this assumption that I'm
making here says, "no that's

01:02:03.670 --> 01:02:04.740
not the case.

01:02:04.740 --> 01:02:07.660
I'm assuming that all that other
stuff is independent of

01:02:07.660 --> 01:02:10.290
what I'm transmitting here."
So I'm very specifically

01:02:10.290 --> 01:02:15.090
assuming at this point that all
of that stuff has not been

01:02:15.090 --> 01:02:16.620
encoded first.

01:02:16.620 --> 01:02:20.070
That I'm sending something
which is independent of

01:02:20.070 --> 01:02:21.780
everything else.

01:02:21.780 --> 01:02:25.280
Which is going to enter
this channel.

01:02:25.280 --> 01:02:28.190
We'll just assume that and after
we get done assuming it

01:02:28.190 --> 01:02:30.760
and seeing what the consequence
of it is, we'll go

01:02:30.760 --> 01:02:34.530
back and see what
it all means.

01:02:34.530 --> 01:02:38.750
OK, so for a little
more notation.

01:02:38.750 --> 01:02:43.020
I'm going to call the
vector, Y, the first

01:02:43.020 --> 01:02:45.950
J, capital J, outputs.

01:02:45.950 --> 01:02:50.800
I'm going to call the vector Y
prime all the other outputs.

01:02:50.800 --> 01:02:54.180
Now intuitively what were aiming
at is, we would like to

01:02:54.180 --> 01:02:57.930
say this stuff doesn't have
anything to do with it.

01:02:57.930 --> 01:02:59.900
We're going to base our
decision on this.

01:02:59.900 --> 01:03:04.060
But I want to prove that to you,
and show you why it works

01:03:04.060 --> 01:03:05.300
and why it doesn't.

01:03:05.300 --> 01:03:07.320
And the noise I'm going to
break up the same way.

01:03:07.320 --> 01:03:10.710
Z is this the first J components
of the noise.

01:03:10.710 --> 01:03:14.670
And Z prime is the other
components of noise.

01:03:14.670 --> 01:03:18.565
So what I have is that the
output, the output that I want

01:03:18.565 --> 01:03:21.230
to look at, namely the
output this vector

01:03:21.230 --> 01:03:24.860
of dimension J output.

01:03:24.860 --> 01:03:30.130
Which is equal to a vector of
dimension J input plus of a

01:03:30.130 --> 01:03:35.460
vector of dimension J
noise is equal to--

01:03:35.460 --> 01:03:39.600
well Y is equal X plus Z. And
the out of band stuff, the

01:03:39.600 --> 01:03:44.490
output, is just these noise
and other signals.

01:03:44.490 --> 01:03:46.000
OK?

01:03:46.000 --> 01:03:49.330
And I want to assume that
Z prime, X, and Z are

01:03:49.330 --> 01:03:52.120
statistically independent.

01:03:52.120 --> 01:03:55.360
Question, test your
probability.

01:03:55.360 --> 01:04:00.460
If I assume that Z prime is
independent of Z, and if I

01:04:00.460 --> 01:04:04.400
assume that Z prime is
independent of X, does that

01:04:04.400 --> 01:04:12.880
mean that Z prime is independant
of X and Z?

01:04:12.880 --> 01:04:17.150
If a is independent of b, and
a is independent of c, is a

01:04:17.150 --> 01:04:19.750
necessarily independent
of the pair bc?

01:04:22.800 --> 01:04:24.050
How many think that's true?

01:04:26.630 --> 01:04:29.520
Better go back and
study a little

01:04:29.520 --> 01:04:33.720
elementary probability again.

01:04:33.720 --> 01:04:37.240
And the notes are occasionally
wrong about that, too.

01:04:37.240 --> 01:04:42.860
So you shouldn't feel, you
shouldn't feel badly about it.

01:04:42.860 --> 01:04:45.840
No, the problem is you really
need this joint independence

01:04:45.840 --> 01:04:47.610
between all three of them.

01:04:47.610 --> 01:04:55.200
I could, for example, make X
plus Y be equal to Z. I could

01:04:55.200 --> 01:04:58.470
do this with discrete
random variables.

01:04:58.470 --> 01:05:01.330
Which are equally probably
zero and one.

01:05:01.330 --> 01:05:04.630
And make the plus equal
to a mod 2 operation.

01:05:04.630 --> 01:05:07.890
And if I did that, each pair
would be independent of each

01:05:07.890 --> 01:05:11.890
other, and the triple would be
very, very highly dependant.

01:05:11.890 --> 01:05:14.990
So anyway, I want to assume
that Z prime, X, and Z are

01:05:14.990 --> 01:05:18.770
statistically independent.

01:05:18.770 --> 01:05:21.710
In other words, what I'm doing
is saying, "If I assume that,

01:05:21.710 --> 01:05:24.770
what's the consequence of it?"

01:05:24.770 --> 01:05:29.600
OK, so the likelihood then, the
probability density of the

01:05:29.600 --> 01:05:33.360
output, Y-- this is the output
in the first j dimensions

01:05:33.360 --> 01:05:38.620
given a particular hypothesis
Y-- is equal to the

01:05:38.620 --> 01:05:45.410
probability density of the noise
evaluated at Y minus am.

01:05:45.410 --> 01:05:49.520
Where this is the signal that
goes with this hypothesis,

01:05:49.520 --> 01:05:51.450
well with am.

01:05:51.450 --> 01:05:59.080
Times the probability density
of Y prime for Z prime.

01:05:59.080 --> 01:05:59.600
OK?

01:05:59.600 --> 01:06:01.890
And I don't even have to assume
that this is Gaussian.

01:06:01.890 --> 01:06:04.480
All I've done is to assume that
these random variables

01:06:04.480 --> 01:06:06.760
are independent of these
random variables.

01:06:06.760 --> 01:06:12.400
And therefore this probability
density is multiplied by this

01:06:12.400 --> 01:06:15.370
probability density.

01:06:15.370 --> 01:06:16.660
OK, well that's kind of neat.

01:06:16.660 --> 01:06:22.420
Because if I put a different i
in here in place of m, I get

01:06:22.420 --> 01:06:23.240
this thing.

01:06:23.240 --> 01:06:24.430
Changes all around.

01:06:24.430 --> 01:06:26.420
F sub Z of Y minus a sub i.

01:06:26.420 --> 01:06:29.470
But this stuff, which
is out of band,

01:06:29.470 --> 01:06:32.950
doesn't change at all.

01:06:32.950 --> 01:06:38.030
When I form the likelihood
ratio, what I get then is this

01:06:38.030 --> 01:06:40.380
divided by that.

01:06:40.380 --> 01:06:43.430
What has happened to Y prime?

01:06:43.430 --> 01:06:46.340
Y prime has disappeared.

01:06:46.340 --> 01:06:51.160
In other words, Y1 to Y sub j
are a sufficient statistic for

01:06:51.160 --> 01:06:52.250
this problem.

01:06:52.250 --> 01:06:55.550
We've shown that sufficient
statistics are the only thing

01:06:55.550 --> 01:06:59.900
you need to use to do maximum
likelihood detection.

01:06:59.900 --> 01:07:00.220
OK?

01:07:00.220 --> 01:07:03.150
In other words, all those other
signals, all that other

01:07:03.150 --> 01:07:08.080
noise, all that stuff from out
of band has disappeared.

01:07:08.080 --> 01:07:10.980
Now let's go back to the fact
that we were looking at a

01:07:10.980 --> 01:07:12.620
finite dimensional problem.

01:07:17.010 --> 01:07:20.390
What happens now when I
make j prime bigger?

01:07:20.390 --> 01:07:22.850
When I start enlarging
j prime?

01:07:22.850 --> 01:07:24.930
What happens to all these
probabilities that we're

01:07:24.930 --> 01:07:26.670
talking about?

01:07:26.670 --> 01:07:32.580
This probability density goes
ape because of this term here.

01:07:32.580 --> 01:07:35.610
We're talking about a
probability density here which

01:07:35.610 --> 01:07:39.460
is involving more and
more and more terms.

01:07:39.460 --> 01:07:42.080
I can't talk about that.

01:07:42.080 --> 01:07:44.130
It doesn't go to any limit.

01:07:44.130 --> 01:07:48.980
It goes to absolute nonsense
as j prime gets big.

01:07:48.980 --> 01:07:52.010
But if I form the likelihood
ratio before I go to the

01:07:52.010 --> 01:07:55.200
limit, then I can go to the
limit quite easily.

01:07:55.200 --> 01:07:59.560
Because there isn't any
limit involved there.

01:07:59.560 --> 01:08:03.490
OK, in other words this is the
likelihood ratio between

01:08:03.490 --> 01:08:07.960
hypothesis m and hypothesis i,
if in fact I looked at this

01:08:07.960 --> 01:08:11.980
entire infinite amount
of observation.

01:08:11.980 --> 01:08:17.760
This is all I need to make
the optimal MAP decision.

01:08:17.760 --> 01:08:20.430
OK, so there's a theorem here
which is called the theorem of

01:08:20.430 --> 01:08:22.940
irrelevance.

01:08:22.940 --> 01:08:26.730
This is something that
Wosencraft and Jacobs in their

01:08:26.730 --> 01:08:30.140
book on communication many years
ago stressed a lot in

01:08:30.140 --> 01:08:33.360
trying to come up with a single
space viewpoint of

01:08:33.360 --> 01:08:34.530
communication.

01:08:34.530 --> 01:08:37.400
And you'll see why this really
does give you a single point

01:08:37.400 --> 01:08:39.100
viewpoint of communication.

01:08:39.100 --> 01:08:41.890
It says that assume that Z
prime is statistically

01:08:41.890 --> 01:08:46.740
independent of the pair X and Z.
Then the MAP detection of X

01:08:46.740 --> 01:08:51.570
from the observation of Y and Y
prime depends only on Y. The

01:08:51.570 --> 01:08:56.330
observed sample value of
Y prime is irrelevant.

01:08:56.330 --> 01:08:56.920
OK?

01:08:56.920 --> 01:09:01.050
So you can do all of detection
theory, you can do all of

01:09:01.050 --> 01:09:04.390
communication, simply forgetting
about that

01:09:04.390 --> 01:09:06.690
irrelevant stuff.

01:09:06.690 --> 01:09:09.390
Because of the theorem we can
stick to finite dimensional

01:09:09.390 --> 01:09:13.310
vectors and the other signals
can be viewed

01:09:13.310 --> 01:09:16.220
as part of Z prime.

01:09:16.220 --> 01:09:18.570
So you don't have to
worry about them.

01:09:18.570 --> 01:09:22.770
So long as each signal is
independent of each other--

01:09:22.770 --> 01:09:26.880
which means that these groups of
bits, the first group a bit

01:09:26.880 --> 01:09:30.000
is used to form a sub x sub 1.

01:09:33.100 --> 01:09:36.120
The next group, the form x sub
2, the next group the form x

01:09:36.120 --> 01:09:37.880
sub 3, and so forth.

01:09:37.880 --> 01:09:41.090
So long as those sequences of
bits are independent of each

01:09:41.090 --> 01:09:44.250
other, you're fine.

01:09:44.250 --> 01:09:45.470
Now, suppose they aren't?

01:09:45.470 --> 01:09:46.720
What happens then?

01:09:49.910 --> 01:09:52.900
Interesting question.

01:09:52.900 --> 01:09:56.480
We said that if they are
independent, I can really do

01:09:56.480 --> 01:10:00.180
maximum likelihood detection
on the whole sequence.

01:10:00.180 --> 01:10:04.120
If they aren't independent,
suppose I say, "oh I don't

01:10:04.120 --> 01:10:08.540
care about that." I'm just going
to use this portion of

01:10:08.540 --> 01:10:13.430
the output to make my decision
and not worry about whether

01:10:13.430 --> 01:10:15.570
this is independent
of anything else.

01:10:15.570 --> 01:10:20.900
I can do that, this still is
going to give me the optimal

01:10:20.900 --> 01:10:25.160
maximum likelihood detection in
terms of the observation y1

01:10:25.160 --> 01:10:26.410
up to y sub j.

01:10:28.240 --> 01:10:33.590
So in other words, whether I
have coding done before this

01:10:33.590 --> 01:10:36.170
or not doesn't make
any difference.

01:10:36.170 --> 01:10:39.080
I can still use maximum
likelihood detection on the

01:10:39.080 --> 01:10:41.840
basis of y1 up to y sub j.

01:10:41.840 --> 01:10:47.970
What the theorem says is if
the out of band stuff--

01:10:47.970 --> 01:10:51.120
both these inputs and the
noise-- are independent of

01:10:51.120 --> 01:10:55.130
what I'm trying to detect,
maximum likelihood becomes the

01:10:55.130 --> 01:10:59.100
optimum thing to do for
equally likely inputs.

01:10:59.100 --> 01:11:01.870
And otherwise, it's a perfectly
reasonable thing to

01:11:01.870 --> 01:11:04.520
do but it's not optimal.

01:11:04.520 --> 01:11:06.760
Now, a lot of people--

01:11:06.760 --> 01:11:08.390
and we'll see some
examples of this

01:11:08.390 --> 01:11:10.770
when we look at wireless--

01:11:10.770 --> 01:11:14.980
in fact, use coding.

01:11:14.980 --> 01:11:18.290
Then they use this particular
kind of detection where they

01:11:18.290 --> 01:11:20.140
forget about all of the added

01:11:20.140 --> 01:11:22.260
information from other signals.

01:11:22.260 --> 01:11:28.660
They make a decision on each of
these x sub k, namely each

01:11:28.660 --> 01:11:32.540
of the M-ary signals that
goes in, they make a

01:11:32.540 --> 01:11:34.210
hard decision on it.

01:11:34.210 --> 01:11:37.670
It's called a hard decision, not
because it's difficult it

01:11:37.670 --> 01:11:41.830
because they refuse to ever
go back and change it.

01:11:41.830 --> 01:11:46.590
If they just say likelihoods and
try to put things together

01:11:46.590 --> 01:11:49.540
in the final decoder, it's
called soft decoding.

01:11:49.540 --> 01:11:52.450
Otherwise it's called
hard decoding.

01:11:52.450 --> 01:11:54.470
If you do soft decoding,
it has to work better.

01:11:54.470 --> 01:11:57.310
Because you're making, in a
sense, a better decision

01:11:57.310 --> 01:12:02.450
because eventually you're
using more information.

01:12:02.450 --> 01:12:07.300
So soft decisions are better
than hard decisions.

01:12:07.300 --> 01:12:11.770
Used to be that everybody used
hard decisions because hard

01:12:11.770 --> 01:12:17.810
decisions were easy and soft
decisions were hard.

01:12:17.810 --> 01:12:21.320
Strange, strange thing.

01:12:21.320 --> 01:12:22.720
But anyway that's changed.

01:12:22.720 --> 01:12:23.730
Why?

01:12:23.730 --> 01:12:25.030
Well it ought to
be obvious why.

01:12:25.030 --> 01:12:28.830
Because anything you build now
cost a tenth of what it used

01:12:28.830 --> 01:12:31.140
to cost to build it.

01:12:31.140 --> 01:12:34.490
I heard Irwin Jacobs awhile
ago saying that one of the

01:12:34.490 --> 01:12:37.960
things that they always did when
they were designing new

01:12:37.960 --> 01:12:41.410
pieces of equipment is they
would look at how much it

01:12:41.410 --> 01:12:45.480
would cost to build
these devices.

01:12:45.480 --> 01:12:49.090
And then, as opposed to most
companies which would say

01:12:49.090 --> 01:12:52.440
that's too expensive let's find
the cheaper way to do it,

01:12:52.440 --> 01:12:55.870
they said OK how long is it
going to take for us to do it,

01:12:55.870 --> 01:12:58.530
and what is the price of
components going to be by time

01:12:58.530 --> 01:13:00.280
we got it done?

01:13:00.280 --> 01:13:02.310
And they would usually say, well
it's going to cost a year

01:13:02.310 --> 01:13:04.780
before we can go into
mass production.

01:13:04.780 --> 01:13:08.130
By that time, everything will
cost less than a half of what

01:13:08.130 --> 01:13:09.360
it's costing now.

01:13:09.360 --> 01:13:12.840
So let's go ahead and
do it the right way.

01:13:12.840 --> 01:13:16.520
So again the argument comes
that you can do

01:13:16.520 --> 01:13:19.340
the hard thing now.

01:13:19.340 --> 01:13:22.390
Which is soft decisions, and
that's what most people do at

01:13:22.390 --> 01:13:24.830
this point.

01:13:24.830 --> 01:13:26.840
Let me give you one more picture
to get ready for what

01:13:26.840 --> 01:13:29.200
we're doing next time.

01:13:29.200 --> 01:13:34.960
Because it's a nice picture
of different signal sets.

01:13:34.960 --> 01:13:39.190
Because we've just talked
abstractly of having multiple

01:13:39.190 --> 01:13:46.300
signals viewed as vectors, and
this will give us some idea of

01:13:46.300 --> 01:13:48.870
what all of these mean.

01:13:48.870 --> 01:13:54.110
I can have two signals, a binary
signal set, and I can

01:13:54.110 --> 01:13:57.880
insist on the signals being
orthogonal to each other.

01:13:57.880 --> 01:14:00.920
Which is a nice thing
to do some times.

01:14:00.920 --> 01:14:04.590
But then I can look at it and
I can say, "how can I make

01:14:04.590 --> 01:14:09.540
that a better signal system?"
The trouble with this signal

01:14:09.540 --> 01:14:12.730
system is it's not antipodal.

01:14:12.730 --> 01:14:16.790
It's not antipodal because
somehow by alternating between

01:14:16.790 --> 01:14:20.750
these two orthogonal signals--
there's a mean between them--

01:14:20.750 --> 01:14:23.880
and I'm transmitting that mean
plus the difference.

01:14:23.880 --> 01:14:28.580
And the difference between them
is minus 0.7 and plus 0.7

01:14:28.580 --> 01:14:32.220
in that direction that way.

01:14:32.220 --> 01:14:35.380
If you can, I guess you
can't see it that way.

01:14:35.380 --> 01:14:38.280
In this direction this way.

01:14:42.360 --> 01:14:46.820
Well anyway, OK.

01:14:46.820 --> 01:14:50.435
A better thing to do than this
is this, which is called bi

01:14:50.435 --> 01:14:51.420
orthogonal.

01:14:51.420 --> 01:14:55.980
So you take orthogonal signals
and then you have a signal set

01:14:55.980 --> 01:15:00.490
consisting of four signals and
two dimensional space.

01:15:00.490 --> 01:15:03.010
We then talk about orthogonal
signals and

01:15:03.010 --> 01:15:04.520
higher dimensional space.

01:15:04.520 --> 01:15:08.040
You can talk about three
orthogonal signals in three

01:15:08.040 --> 01:15:10.020
dimensional space here.

01:15:10.020 --> 01:15:11.220
There, there, and there.

01:15:11.220 --> 01:15:15.540
So that's m equals
3 and j equals 3.

01:15:15.540 --> 01:15:18.190
If you do the same thing
that we did here--

01:15:18.190 --> 01:15:20.650
we're going to make this into
an equilateral triangle.

01:15:20.650 --> 01:15:23.900
Namely we're going to center
it around the center.

01:15:23.900 --> 01:15:26.220
If we do the same thing that
we did here we're going to

01:15:26.220 --> 01:15:31.690
turn this into a set of six
waveforms which are still

01:15:31.690 --> 01:15:36.520
using the three degrees of
freedom, but at least get us

01:15:36.520 --> 01:15:39.710
more signals.

01:15:39.710 --> 01:15:42.110
For the same number of
degrees of freedom.

01:15:42.110 --> 01:15:45.560
So you can extend this picture
as far as you want to.

01:15:45.560 --> 01:15:49.060
You can talk about many, many
orthogonal signals going into

01:15:49.060 --> 01:15:52.530
many more degrees of freedom.

01:15:52.530 --> 01:15:54.860
For each one of them you
can come up with a

01:15:54.860 --> 01:15:56.800
simplex set of signals.

01:15:56.800 --> 01:16:00.280
The nice thing about the simplex
set of signals is that

01:16:00.280 --> 01:16:01.690
all of the signals are arranged

01:16:01.690 --> 01:16:03.240
around the center point.

01:16:03.240 --> 01:16:06.030
They're all equally distant
from each other.

01:16:06.030 --> 01:16:09.280
You can get these for every
dimension by starting with

01:16:09.280 --> 01:16:12.090
these and simply taking the mean
out, which loses you one

01:16:12.090 --> 01:16:16.440
dimension and makes this
sort of ideal set.

01:16:16.440 --> 01:16:20.980
Well tomorrow-- on Wednesday
what we're going to do is,

01:16:20.980 --> 01:16:25.640
we're going to talk about these
large sets here, and see

01:16:25.640 --> 01:16:26.450
what happens.

01:16:26.450 --> 01:16:28.070
And we'll see that you in
fact get to channel

01:16:28.070 --> 01:16:29.320
capacity this way.

01:16:31.830 --> 01:16:33.080
OK.