WEBVTT
00:00:00.460 --> 00:00:00.660
Hi.
00:00:00.660 --> 00:00:03.240
In this session, we're going to
cover a nice review problem
00:00:03.240 --> 00:00:05.210
that will look at how
to infer one random
00:00:05.210 --> 00:00:06.880
variable based on another.
00:00:06.880 --> 00:00:09.090
And in this problem, we're given
two random variables--
00:00:09.090 --> 00:00:10.090
X and Y--
00:00:10.090 --> 00:00:13.950
and we're also given their joint
pdf, which we're told is
00:00:13.950 --> 00:00:16.840
a constant 2/3 within the
region bounded by
00:00:16.840 --> 00:00:17.930
these orange lines.
00:00:17.930 --> 00:00:21.280
And outside of the region,
the joint pdf is 0.
00:00:21.280 --> 00:00:24.150
So the first thing we're going
to look at, or the first thing
00:00:24.150 --> 00:00:29.140
that we're asked to do, is find
the LMS estimator of Y
00:00:29.140 --> 00:00:36.230
based on X. Now, remember that
LMS estimator is really just a
00:00:36.230 --> 00:00:37.990
conditional expectation.
00:00:37.990 --> 00:00:42.860
So the LMS estimator of Y based
on X is the conditional
00:00:42.860 --> 00:00:45.950
expectation of Y, given X.
00:00:45.950 --> 00:00:48.960
Now, when we have a plot of
the joint pdf and we're
00:00:48.960 --> 00:00:53.200
dealing with these two random
variables, and especially when
00:00:53.200 --> 00:00:56.620
the joint pdf is constant like
this, it's often easy to
00:00:56.620 --> 00:00:59.680
calculate this conditional
expectation visually.
00:00:59.680 --> 00:01:02.810
So what we really need to do
is just say, given any
00:01:02.810 --> 00:01:07.520
particular value of X, what
is the conditional
00:01:07.520 --> 00:01:08.670
expectation of Y?
00:01:08.670 --> 00:01:12.390
So what we can do is we can just
pick some values of X and
00:01:12.390 --> 00:01:15.520
see visually what that initial
expectation is.
00:01:15.520 --> 00:01:24.900
So for example, if X is 1/2,
given that X is 1/2, and since
00:01:24.900 --> 00:01:29.510
this whole joint pdf is uniform,
then the conditional
00:01:29.510 --> 00:01:32.780
slice of Y will be from
here to here.
00:01:32.780 --> 00:01:36.260
And that slice, the conditional
distribution of Y,
00:01:36.260 --> 00:01:38.910
given that X is 1/2, will
also be uniform.
00:01:38.910 --> 00:01:41.450
So it'll be uniform
from here to here.
00:01:41.450 --> 00:01:42.990
And if it's uniform, we know
that the conditional
00:01:42.990 --> 00:01:46.150
expectation will just be
the midpoint here.
00:01:48.990 --> 00:01:51.670
And so that would be what the
conditional expectation of Y
00:01:51.670 --> 00:01:53.830
would be, given that X is 1/2.
00:01:53.830 --> 00:01:57.640
And we could do the same
thing for X equals 1.
00:01:57.640 --> 00:02:01.020
And we'll see that again,
because everything is uniform,
00:02:01.020 --> 00:02:03.030
this slice is also going
to be uniform.
00:02:03.030 --> 00:02:06.030
And so the conditional
expectation will again be the
00:02:06.030 --> 00:02:07.780
midpoint, which is there.
00:02:07.780 --> 00:02:12.220
And then if we just look at it
within this region, it's
00:02:12.220 --> 00:02:15.850
always going to be
the midpoint.
00:02:15.850 --> 00:02:18.820
And so we get that the initial
expectation of Y, given X,
00:02:18.820 --> 00:02:23.070
will just look like that line,
which you can think of it as
00:02:23.070 --> 00:02:29.420
just bisecting this angle formed
by these two parts of
00:02:29.420 --> 00:02:31.000
the region.
00:02:31.000 --> 00:02:33.230
But things are a little bit
different, though, when we
00:02:33.230 --> 00:02:37.160
move to the region where
X is between 1 and 2.
00:02:37.160 --> 00:02:44.480
Between 1 and 2, say
at 1 and 1/2, this
00:02:44.480 --> 00:02:45.370
line doesn't continue.
00:02:45.370 --> 00:02:51.690
Because now, the slice of Y goes
from here to here, and
00:02:51.690 --> 00:02:52.800
again, it's still uniform.
00:02:52.800 --> 00:02:58.400
So the midpoint would
be there.
00:02:58.400 --> 00:03:02.580
And similarly for X equals
2, it would be here.
00:03:02.580 --> 00:03:08.330
And so for X between 1 and 2,
the conditional expectation
00:03:08.330 --> 00:03:09.950
actually looks like this.
00:03:09.950 --> 00:03:12.720
So you see that there's actually
two linear parts of
00:03:12.720 --> 00:03:16.210
it, but there's a kink
at X equals 1.
00:03:16.210 --> 00:03:19.780
And so by looking at this
visually and taking advantage
00:03:19.780 --> 00:03:22.630
of the fact that everything is
uniform, we can pretty easily
00:03:22.630 --> 00:03:24.650
figure out what this conditional
expectation is.
00:03:24.650 --> 00:03:29.420
So now, let's actually just
write it out algebraically.
00:03:29.420 --> 00:03:34.620
So for X between 0 and 1, we
said that it's this line,
00:03:34.620 --> 00:03:41.490
which if we look at it, that's
just 1/2 of X. Now, this is
00:03:41.490 --> 00:03:44.800
for X between 0 and 1.
00:03:44.800 --> 00:03:49.000
And if X is between 1 and 2,
it's going to be this line,
00:03:49.000 --> 00:03:50.620
which is a slope of 1.
00:03:50.620 --> 00:03:53.600
And if we extend this
down, it hits the
00:03:53.600 --> 00:03:56.170
y-axis at negative 1/2.
00:03:56.170 --> 00:04:02.840
So it's X minus 1/2, if
X is between 1 and 2.
00:04:06.110 --> 00:04:08.840
And otherwise, it's undefined.
00:04:08.840 --> 00:04:11.100
So we'll focus on these
two cases here.
00:04:13.720 --> 00:04:21.209
Now, the second part of the
question, now that we know
00:04:21.209 --> 00:04:24.330
what the LMS estimator is, we're
asked to find what is
00:04:24.330 --> 00:04:27.860
the traditional mean squared
error of this estimator?
00:04:27.860 --> 00:04:29.590
So we want to know
how good is it.
00:04:29.590 --> 00:04:31.520
And one way of capturing
that is to look at the
00:04:31.520 --> 00:04:32.750
mean squared error.
00:04:32.750 --> 00:04:37.130
And so recall that the
conditional mean squared error
00:04:37.130 --> 00:04:41.755
is given by this expression.
00:04:48.930 --> 00:04:52.510
So what we're saying is this is
what we estimate Y to be.
00:04:52.510 --> 00:04:56.930
This is what y really is, so
this difference is how wrong,
00:04:56.930 --> 00:04:59.370
or the error in our estimate.
00:04:59.370 --> 00:05:04.090
We square it, because otherwise,
positive and
00:05:04.090 --> 00:05:05.580
negative errors might cancel
each other out,
00:05:05.580 --> 00:05:06.610
so we square it.
00:05:06.610 --> 00:05:10.880
And then this just looking
at each individual
00:05:10.880 --> 00:05:12.750
value of x for now.
00:05:12.750 --> 00:05:13.805
So this is why it's
the conditional
00:05:13.805 --> 00:05:15.540
mean squared error.
00:05:15.540 --> 00:05:17.290
So how do we calculate this?
00:05:17.290 --> 00:05:24.390
Well, remember that this g of
X, we said the LMS estimator
00:05:24.390 --> 00:05:26.900
is just a conditional
expectation.
00:05:26.900 --> 00:05:30.220
So it's just expectation
of Y, given X.
00:05:30.220 --> 00:05:32.945
Well, then if you look at this,
what this reminds you
00:05:32.945 --> 00:05:35.900
of, it reminds you of the
definition of what a
00:05:35.900 --> 00:05:37.190
conditional variance is.
00:05:37.190 --> 00:05:42.420
A variance is just, you take the
random variable, subtract
00:05:42.420 --> 00:05:45.230
its mean, square it, and take
the expectation of that.
00:05:45.230 --> 00:05:47.690
This is no different, except
that everything is now the
00:05:47.690 --> 00:05:50.060
conditional world of X.
00:05:50.060 --> 00:05:55.410
So this is actually the
conditional variance of Y,
00:05:55.410 --> 00:05:59.800
given X is little x.
00:05:59.800 --> 00:06:01.720
What is the conditional
variance of Y, given
00:06:01.720 --> 00:06:02.980
that X is little x?
00:06:02.980 --> 00:06:05.300
Now, we can again go back
to this plot to
00:06:05.300 --> 00:06:06.560
try to help us out.
00:06:06.560 --> 00:06:10.710
We can split this up
into regions again.
00:06:10.710 --> 00:06:15.130
So just take some little x as
an example and see what the
00:06:15.130 --> 00:06:16.120
variance is.
00:06:16.120 --> 00:06:22.590
So if little x is 1/2, then we
know that the conditional
00:06:22.590 --> 00:06:24.760
distribution of Y would
be uniform, we said,
00:06:24.760 --> 00:06:26.540
from 0 up to here.
00:06:26.540 --> 00:06:29.670
Well, that point is this
from 0 to 1/2.
00:06:29.670 --> 00:06:32.860
And remember, the variance of a
uniform distribution is just
00:06:32.860 --> 00:06:34.180
the width of the uniform
distribution
00:06:34.180 --> 00:06:36.150
squared, divided by 12.
00:06:36.150 --> 00:06:42.060
And so in this case, the width
would be 1/2 squared over 12.
00:06:42.060 --> 00:06:48.240
And in general, for the region
of X between 0 and 1, the
00:06:48.240 --> 00:06:51.960
width of the conditional
distribution of Y will always
00:06:51.960 --> 00:06:58.080
be X, because the width will
go from 0 to wherever X is.
00:06:58.080 --> 00:07:02.980
So because of that, the
conditional variance will just
00:07:02.980 --> 00:07:09.800
be X squared, the width squared,
over 12, when X is
00:07:09.800 --> 00:07:12.340
between 0 and 1.
00:07:12.340 --> 00:07:14.640
Now, let's think about the
other case, where X is
00:07:14.640 --> 00:07:16.130
between 1 and 2.
00:07:16.130 --> 00:07:18.900
Well, if X is between 1 and
2, we're over here.
00:07:18.900 --> 00:07:22.440
And now, if we take the
conditional distribution of Y,
00:07:22.440 --> 00:07:23.740
it's again uniform.
00:07:23.740 --> 00:07:26.760
But the width now, instead of
varying with Y, it's always
00:07:26.760 --> 00:07:30.090
going to be the same width.
00:07:30.090 --> 00:07:33.360
Each of these slices have the
same width, and the width goes
00:07:33.360 --> 00:07:39.610
from here-- this is X minus
1, and that's X.
00:07:39.610 --> 00:07:42.550
So if the width is always going
to be a constant of 1.
00:07:42.550 --> 00:07:45.310
And so this variance is
going to be 1/12.
00:07:48.360 --> 00:07:51.160
And from that, we get our answer
for the conditional
00:07:51.160 --> 00:07:52.110
mean squared error.
00:07:52.110 --> 00:07:55.820
Now, part c asks us to find the
mean squared error, which
00:07:55.820 --> 00:07:58.105
is given by this expression.
00:08:02.220 --> 00:08:04.490
And we'll see that it looks very
similar to this, which
00:08:04.490 --> 00:08:06.070
was the conditional mean
squared error.
00:08:06.070 --> 00:08:08.520
And now, given what we
know from part b,
00:08:08.520 --> 00:08:09.710
this is easy to calculate.
00:08:09.710 --> 00:08:13.330
We can just apply total
expectation, because this is
00:08:13.330 --> 00:08:22.130
just equal to the integral
of the conditional
00:08:22.130 --> 00:08:23.380
mean squared error.
00:08:27.430 --> 00:08:32.880
And then we need to also
multiply this by the pf of x,
00:08:32.880 --> 00:08:37.260
and then integrate over X. And
that integral will should be
00:08:37.260 --> 00:08:43.289
from X equals 0 to 2, because
that's the only range that
00:08:43.289 --> 00:08:46.240
applies for x, given
this joint pdf.
00:08:46.240 --> 00:08:48.310
Now, in order to do this first,
though, we need to
00:08:48.310 --> 00:08:54.610
figure out what the
pdf of X is.
00:08:57.190 --> 00:09:00.620
In order to do that, we can go
back to our original joint pdf
00:09:00.620 --> 00:09:02.600
of X and Y and marginalize it.
00:09:02.600 --> 00:09:05.810
So marginalizing, you could
think of it as taking this
00:09:05.810 --> 00:09:10.510
joint pdf and collapsing it onto
the x-axis so that you
00:09:10.510 --> 00:09:13.410
take everything and
integrate out Y.
00:09:13.410 --> 00:09:15.900
Now to do that, let's
do that up here.
00:09:20.310 --> 00:09:25.450
We can split it up into
two sections.
00:09:25.450 --> 00:09:32.540
So the section of X between 0
and 1, we integrate the joint
00:09:32.540 --> 00:09:38.620
pdf from Y equals 0 to Y
equals X, which is this
00:09:38.620 --> 00:09:40.320
portion of this line.
00:09:40.320 --> 00:09:50.790
So we integrate Y. The joint pdf
is 2/3, and we integrate Y
00:09:50.790 --> 00:10:05.180
out from Y equals 0 to X. And
then for the portion of X from
00:10:05.180 --> 00:10:07.890
1 to 2, we again integrate
Y out.
00:10:07.890 --> 00:10:20.620
Now we integrate Y from
X minus 1 up to X.
00:10:20.620 --> 00:10:30.810
So this is X between 0 and 1,
and this is X between 1 and 2.
00:10:33.580 --> 00:10:39.280
So we just do some little bit
of calculus, and we get that
00:10:39.280 --> 00:10:46.030
this is going to be 2/3 X when
X is between 0 and 1.
00:10:46.030 --> 00:10:55.270
And it's going to be 2/3 when
X is between 1 and 2.
00:10:55.270 --> 00:10:59.350
So now that we have what the
marginal pdf of X is, we can
00:10:59.350 --> 00:11:03.170
plug that into this, and plug in
what we had for b, and then
00:11:03.170 --> 00:11:04.910
calculate what this
actually is.
00:11:04.910 --> 00:11:08.810
So remember, we need to take
care of these two cases, these
00:11:08.810 --> 00:11:09.390
two regions--
00:11:09.390 --> 00:11:11.720
X between 0 and 1, and
X between 1 and 2.
00:11:11.720 --> 00:11:17.160
So the conditional mean squared
error for X between 0
00:11:17.160 --> 00:11:19.780
and 1 is X squared over 2.
00:11:19.780 --> 00:11:26.390
So between 0 and 1, this first
part is X squared over 12.
00:11:26.390 --> 00:11:32.745
The pdf of X in that same
region is 2/3 x.
00:11:32.745 --> 00:11:37.830
And we integrate that in the
region from x equals 0 to 1.
00:11:37.830 --> 00:11:41.340
And then, we also have the
second region which
00:11:41.340 --> 00:11:43.280
is X from 1 to 2.
00:11:43.280 --> 00:11:45.690
In that region, the traditional
mean squared error
00:11:45.690 --> 00:11:47.990
from part b is 1/12.
00:11:47.990 --> 00:11:55.560
The marginal pdf of X is 2/3,
and we do this integral.
00:11:55.560 --> 00:11:59.530
And if you just carry out some
calculus here, you'll get that
00:11:59.530 --> 00:12:02.525
the final answer is
equal to 5/72.
00:12:08.660 --> 00:12:14.420
Now, the last part of this
question asks us, is this mean
00:12:14.420 --> 00:12:17.380
squared error the same thing--
00:12:17.380 --> 00:12:24.475
does it equal the expectation
of the conditional variance?
00:12:27.000 --> 00:12:29.480
And it turns out that
yes, it does.
00:12:29.480 --> 00:12:36.980
And to see that, we can just
take this, and use the law of
00:12:36.980 --> 00:12:41.100
iterated expectations, because
iterated expectations tells us
00:12:41.100 --> 00:12:50.220
this is in fact equal to the
expectation of Y minus g of X
00:12:50.220 --> 00:12:56.580
squared, given X. That's just
applying law of iterated
00:12:56.580 --> 00:12:59.320
expectations.
00:12:59.320 --> 00:13:04.350
And then, if we look at this,
this part that's inside is
00:13:04.350 --> 00:13:07.200
exactly equal to the conditional
variance of Y,
00:13:07.200 --> 00:13:11.250
given X. And so these two
are, in fact, the same.
00:13:11.250 --> 00:13:15.980
In part c, we calculated what
the marginal pdf of X is, and
00:13:15.980 --> 00:13:18.530
it'll actually be used later
on in this problem.
00:13:18.530 --> 00:13:20.800
So for future reference, let's
just write it down here in
00:13:20.800 --> 00:13:22.140
this corner.
00:13:22.140 --> 00:13:24.490
Now, so far in this
problem, we've
00:13:24.490 --> 00:13:27.920
looked at the LMS estimator.
00:13:27.920 --> 00:13:29.590
And of course, there are
other estimators that
00:13:29.590 --> 00:13:30.550
you can use as well.
00:13:30.550 --> 00:13:35.240
And now in part d, let's look
at the linear LMS estimator.
00:13:35.240 --> 00:13:42.810
Now remember, the linear LMS
estimator is special, because
00:13:42.810 --> 00:13:48.040
it forces the estimator to have
a linear relationship.
00:13:48.040 --> 00:13:51.310
So the estimator is going to
be a linear function of X.
00:13:51.310 --> 00:13:53.820
Now, compare that
to what the LMS
00:13:53.820 --> 00:13:55.590
estimator was in this case.
00:13:55.590 --> 00:13:58.790
It was two linear pieces,
but there was a kink.
00:13:58.790 --> 00:14:03.150
And so the entire estimator
wasn't actually linear in X.
00:14:03.150 --> 00:14:07.360
Now, the LLMS estimator, or the
linear LMS estimator, will
00:14:07.360 --> 00:14:10.180
give us the linear estimator.
00:14:10.180 --> 00:14:11.610
It's going to be a linear
function of X.
00:14:11.610 --> 00:14:15.910
And we know that we have
a formula for this.
00:14:15.910 --> 00:14:23.930
Is the expectation of Y plus the
covariance of X and Y over
00:14:23.930 --> 00:14:32.770
the variance of X times X minus
expectation of X. All
00:14:32.770 --> 00:14:35.350
right, so in order to calculate
what this is, we
00:14:35.350 --> 00:14:38.810
just need to calculate
what four things are.
00:14:38.810 --> 00:14:43.080
Now, let's start with this last
one, the expected value
00:14:43.080 --> 00:14:47.900
of X. To calculate the expected
value of X, we just
00:14:47.900 --> 00:14:48.590
use a formula.
00:14:48.590 --> 00:14:52.010
And from before, we know
what the pdf of X is.
00:14:52.010 --> 00:14:56.910
And so we know that this
is just going to be X
00:14:56.910 --> 00:15:01.160
times fx of x dx.
00:15:01.160 --> 00:15:09.150
And in particular, this will
give us that from 0 to 1, it's
00:15:09.150 --> 00:15:13.200
going to be X times the
pdf of X is 2/3 X,
00:15:13.200 --> 00:15:16.486
so it's 2/3 X squared.
00:15:19.720 --> 00:15:24.950
And from 1 to 2, it's going to
be X times the pdf of X, which
00:15:24.950 --> 00:15:31.200
is just 2/3, so it's 2/3 X dx.
00:15:31.200 --> 00:15:34.000
And when you calculate
this out, you'll get
00:15:34.000 --> 00:15:38.060
that is equal to 11/9.
00:15:38.060 --> 00:15:41.250
Now, let's calculate the
variance of X next.
00:15:41.250 --> 00:15:43.760
In order to calculate that,
let's use the formula that
00:15:43.760 --> 00:15:46.920
variance is equal to the
expectation of X squared minus
00:15:46.920 --> 00:15:49.220
the expectation of X
quantity squared.
00:15:49.220 --> 00:15:52.440
We had the expectation of X,
so let's calculate what the
00:15:52.440 --> 00:15:56.940
expectation of X squared is.
00:15:56.940 --> 00:15:59.060
Now, it's the same idea.
00:15:59.060 --> 00:16:06.060
Instead, we have X squared
times f of X dx.
00:16:06.060 --> 00:16:08.860
And we'll get the same
sort of formula.
00:16:08.860 --> 00:16:11.880
We'll split it up again into
two different parts from X
00:16:11.880 --> 00:16:14.490
equals 0 to X equals 1.
00:16:14.490 --> 00:16:16.260
It's going to be X squared
times pdf, so
00:16:16.260 --> 00:16:20.430
it's 2/3 X cubed dx.
00:16:20.430 --> 00:16:23.490
And then from X equals 1 to
2, it's going to be X
00:16:23.490 --> 00:16:25.060
squared times 2/3.
00:16:25.060 --> 00:16:28.770
So it's 2/3 X squared dx.
00:16:28.770 --> 00:16:30.890
And when we calculate this
out, we'll get that
00:16:30.890 --> 00:16:32.750
it's equal to 31/18.
00:16:35.900 --> 00:16:38.730
From that, we know that the
variance is going to be equal
00:16:38.730 --> 00:16:44.680
to expectation of X squared
minus expectation of X
00:16:44.680 --> 00:16:46.860
quantity squared.
00:16:46.860 --> 00:16:52.940
Now, expectation of X
squared is 31/18.
00:16:52.940 --> 00:16:55.010
Expectation of X is 11/9.
00:16:55.010 --> 00:16:58.010
And when we calculate this,
we get that the
00:16:58.010 --> 00:17:06.400
variance is equal to 37/162.
00:17:06.400 --> 00:17:08.349
So now we have this,
and we have that.
00:17:08.349 --> 00:17:11.170
Let's calculate what expectation
of Y is.
00:17:11.170 --> 00:17:16.190
Expectation of Y, let's
calculate it using the law of
00:17:16.190 --> 00:17:19.310
iterated expectations.
00:17:19.310 --> 00:17:20.944
The law of iterated expectations
tells us that
00:17:20.944 --> 00:17:25.849
this is equal to the expectation
of Y conditioned
00:17:25.849 --> 00:17:27.700
on X.
00:17:27.700 --> 00:17:29.410
Now, we already know what
expectation of Y
00:17:29.410 --> 00:17:30.430
conditioned on X is.
00:17:30.430 --> 00:17:33.400
That was the LMS estimator that
we calculated earlier.
00:17:33.400 --> 00:17:35.770
It's this.
00:17:35.770 --> 00:17:39.760
So now we just need to
calculate this out,
00:17:39.760 --> 00:17:41.650
and we can do that.
00:17:41.650 --> 00:17:48.210
So we know that in the range
from X between 0 and 1, it's
00:17:48.210 --> 00:17:55.720
equal to 1/2 X. So in the range
from 0 to 1, it's equal
00:17:55.720 --> 00:18:04.130
to 1/2 X. But then, we have to
use total expectation, so we
00:18:04.130 --> 00:18:07.940
have to multiply by the pdf
of X in that region
00:18:07.940 --> 00:18:11.790
which is 2/3 X dx.
00:18:11.790 --> 00:18:16.900
And then in the range from
X equals 1 to 2, this
00:18:16.900 --> 00:18:18.840
conditional expectation
is X minus 1/2.
00:18:22.100 --> 00:18:25.440
And the pdf of X in that
region is 2/3.
00:18:30.200 --> 00:18:35.500
Now, when we calculate out this
value, we'll get that
00:18:35.500 --> 00:18:38.060
it's equal to 7/9.
00:18:40.620 --> 00:18:44.770
And now, the last piece is the
covariance of X and Y.
00:18:44.770 --> 00:18:47.760
Remember, the covariance, we
can calculate that as the
00:18:47.760 --> 00:18:52.810
expectation of X times Y minus
the expectation of X times the
00:18:52.810 --> 00:18:56.030
expectation of Y. We already
know the expectation of X and
00:18:56.030 --> 00:18:58.590
the expectation of Y, so we
just need to calculate the
00:18:58.590 --> 00:19:03.212
expectation of X times Y,
the product of the two.
00:19:03.212 --> 00:19:08.740
And for that, we'll use the
definition, and we'll use the
00:19:08.740 --> 00:19:11.990
joint pdf that we have.
00:19:11.990 --> 00:19:16.610
So this is going to be a double
integral of X times Y
00:19:16.610 --> 00:19:17.860
times the joint pdf.
00:19:22.360 --> 00:19:25.920
And the tricky part here is just
figuring out what these
00:19:25.920 --> 00:19:26.430
limits are.
00:19:26.430 --> 00:19:30.312
So we'll integrate
in this order--
00:19:30.312 --> 00:19:32.590
X and Y.
00:19:32.590 --> 00:19:35.880
Now, let's split this up.
00:19:35.880 --> 00:19:38.450
So let's focus on
splitting X up.
00:19:38.450 --> 00:19:46.990
So for X between 0 and 1, we
just need to figure out what's
00:19:46.990 --> 00:19:51.240
the rate right range of Y to
integrate over such that this
00:19:51.240 --> 00:19:52.120
is actually non-zero.
00:19:52.120 --> 00:19:55.160
Because remember, the
joint pdf is easy.
00:19:55.160 --> 00:19:56.230
It's just a constant 2/3.
00:19:56.230 --> 00:19:58.660
But it's only a constant
2/3 within this region.
00:19:58.660 --> 00:20:01.110
So the difficult part is just
figuring out what the limits
00:20:01.110 --> 00:20:03.100
are in order to specify
that region.
00:20:03.100 --> 00:20:11.410
So for X between 0 and 1, Y
has to be between 0 and X,
00:20:11.410 --> 00:20:18.100
because this line is Y equals
X. So we need to integrate
00:20:18.100 --> 00:20:19.926
from 0 to X--
00:20:19.926 --> 00:20:23.595
X times Y times the joint
pdf, which is 2/3.
00:20:28.260 --> 00:20:34.380
And now, let's do the other
part, which is X from 1 to 2.
00:20:34.380 --> 00:20:38.170
Well, if X is from 1 to 2, in
order to fall into this
00:20:38.170 --> 00:20:43.210
region, Y has to be between X
minus 1 and X. So we integrate
00:20:43.210 --> 00:20:48.110
Y from X minus 1 to X. Against,
it's X times Y times
00:20:48.110 --> 00:20:49.640
the joint pdf, which is 2/3.
00:20:54.070 --> 00:20:57.360
And now, once we have this set
up, the rest of it we can just
00:20:57.360 --> 00:20:58.400
do some calculus.
00:20:58.400 --> 00:21:00.680
And what we find is
that the final
00:21:00.680 --> 00:21:03.940
answer is equal to 41/36.
00:21:07.940 --> 00:21:13.640
Now, what that tells us is that
the covariance of X and
00:21:13.640 --> 00:21:19.430
Y, which is just expectation
of X times Y, the product,
00:21:19.430 --> 00:21:25.080
minus expectation of X times
expectation of Y.
00:21:25.080 --> 00:21:27.640
We know expectation
of X times Y now.
00:21:27.640 --> 00:21:29.440
It's 41/36.
00:21:29.440 --> 00:21:31.920
Expectation of X is 11/9.
00:21:31.920 --> 00:21:34.640
Expectation of Y is 7/9.
00:21:34.640 --> 00:21:37.050
So when we substitute all of
that in, we get that this
00:21:37.050 --> 00:21:43.830
covariance is 61/324.
00:21:43.830 --> 00:21:45.910
All right, so now we have
everything we need.
00:21:45.910 --> 00:21:48.760
Expectation of Y is here.
00:21:48.760 --> 00:21:51.620
Covariance is here.
00:21:51.620 --> 00:21:54.360
Variance of X is here.
00:21:54.360 --> 00:21:57.210
And expectation of X is here.
00:21:57.210 --> 00:22:01.970
So let's substitute that in, and
we can figure out what the
00:22:01.970 --> 00:22:05.410
actual LLMS estimator is.
00:22:05.410 --> 00:22:09.950
So expectation of Y
we know is 7/9.
00:22:09.950 --> 00:22:13.570
Expectation of X is 11/9.
00:22:16.800 --> 00:22:20.980
And when you divide the
covariance, which is 61/324,
00:22:20.980 --> 00:22:32.040
by the variance, which is
37/162, we'll get 61/74.
00:22:32.040 --> 00:22:36.290
And so that is the LLMS
estimator that we calculated.
00:22:36.290 --> 00:22:40.760
And notice that it is,
in fact, linear in X.
00:22:40.760 --> 00:22:44.720
So let's plot that and see
what it looks like.
00:22:44.720 --> 00:22:52.300
So it's going to be a
line, and it's going
00:22:52.300 --> 00:22:54.610
to look like this.
00:22:54.610 --> 00:23:00.700
So at X equals 2, it's actually
a little bit below 1
00:23:00.700 --> 00:23:05.360
and 1/2, which is what the
LMS estimator would be.
00:23:05.360 --> 00:23:13.010
At X equals 1, it's actually a
little bit above 1/2, which is
00:23:13.010 --> 00:23:15.150
what the LMS estimator
would be.
00:23:15.150 --> 00:23:21.120
And then it crosses 0 around
roughly 1/4, and it drops
00:23:21.120 --> 00:23:22.160
actually below 0.
00:23:22.160 --> 00:23:26.040
So if we connect the dots,
it's going to look
00:23:26.040 --> 00:23:28.820
something like this.
00:23:28.820 --> 00:23:32.230
So notice that it's actually not
too far away from the LMS
00:23:32.230 --> 00:23:33.850
estimator here.
00:23:33.850 --> 00:23:38.340
But it doesn't have the kink
because it is a line.
00:23:38.340 --> 00:23:41.700
And note also that it actually
drops below.
00:23:41.700 --> 00:23:45.070
So when X is very small, you
actually estimate negative
00:23:45.070 --> 00:23:51.880
values of Y, which is actually
impossible, given the joint
00:23:51.880 --> 00:23:53.410
pdf distribution that
we're given.
00:23:53.410 --> 00:23:57.950
And that is sometimes a feature
or artifact of the
00:23:57.950 --> 00:24:00.530
linear LMS estimator, that
you'll get values that don't
00:24:00.530 --> 00:24:03.820
necessarily seem
to make sense.
00:24:03.820 --> 00:24:06.720
So now that we've calculated
the linear LMS estimator in
00:24:06.720 --> 00:24:10.740
part d, which is this, and the
LMS estimator in part a, which
00:24:10.740 --> 00:24:14.420
is this, we've also compared
them visually.
00:24:14.420 --> 00:24:17.750
The linear LMS estimator
is the one in pink,
00:24:17.750 --> 00:24:18.680
the straight line.
00:24:18.680 --> 00:24:22.080
And the LMS estimator is the
one in black with the kink.
00:24:22.080 --> 00:24:24.170
It's an interesting question
to now ask, which
00:24:24.170 --> 00:24:25.480
one of these is better?
00:24:25.480 --> 00:24:27.390
And in order to judge that, we
need to come up with some sort
00:24:27.390 --> 00:24:30.060
of criterion to compare
the two with.
00:24:30.060 --> 00:24:34.280
And the one that we're going
to look at in part e is the
00:24:34.280 --> 00:24:35.040
mean squared error.
00:24:35.040 --> 00:24:37.420
Which one gives the lower
mean squared error.
00:24:37.420 --> 00:24:44.530
And so specifically, we're going
to ask ourselves which
00:24:44.530 --> 00:24:49.075
of these two estimators
gives us the smaller
00:24:49.075 --> 00:24:50.325
mean squared error?
00:24:54.000 --> 00:24:57.500
Is it the linear LMS estimator
given by l of X?
00:24:57.500 --> 00:25:02.180
Or is it the LMS estimator,
given by g of X?
00:25:02.180 --> 00:25:08.400
Now, we know that the LMS
estimator is the one that
00:25:08.400 --> 00:25:09.560
actually minimizes this.
00:25:09.560 --> 00:25:13.600
The LMS estimator is designed
to minimize the
00:25:13.600 --> 00:25:14.140
mean squared error.
00:25:14.140 --> 00:25:18.030
And so we know that given any
estimator of X, this one will
00:25:18.030 --> 00:25:20.470
have the smallest mean
squared error.
00:25:20.470 --> 00:25:24.430
And so the linear LMS estimator,
its mean squared
00:25:24.430 --> 00:25:29.480
error has to be at least as
large as the LMS estimators.
00:25:29.480 --> 00:25:32.220
And the last part of the
question now asks us to look
00:25:32.220 --> 00:25:37.680
at a third type of estimator,
which is the MEP estimator.
00:25:37.680 --> 00:25:41.390
Now, we want to ask, why is it
that we haven't been using the
00:25:41.390 --> 00:25:43.260
MEP estimator in this problem?
00:25:43.260 --> 00:25:45.360
Well, remember what the
MEP estimator does.
00:25:45.360 --> 00:25:50.730
In this case, what we would
do is it would take the
00:25:50.730 --> 00:25:54.030
conditional distribution ratio
of Y given any value of X. And
00:25:54.030 --> 00:25:59.200
then it would pick the value
of Y that gives the highest
00:25:59.200 --> 00:26:01.780
value in the conditional
distribution.
00:26:01.780 --> 00:26:04.190
And that would be the
MEP estimate of Y.
00:26:04.190 --> 00:26:08.060
But the problem in this case is
that if you take any slice
00:26:08.060 --> 00:26:11.700
here, so a condition on any
value of X, any of these
00:26:11.700 --> 00:26:18.240
slices, if you just take this
out and look at it, it's going
00:26:18.240 --> 00:26:20.040
to be uniform.
00:26:20.040 --> 00:26:28.910
This is what the conditional
distribution of Y given X is.
00:26:28.910 --> 00:26:33.662
It's going to be uniform between
0 and X. Now, what the
00:26:33.662 --> 00:26:37.610
MEP rule tells us is we're going
to pick the value of Y
00:26:37.610 --> 00:26:43.310
that gives us the highest point
in this conditional
00:26:43.310 --> 00:26:44.190
distribution.
00:26:44.190 --> 00:26:47.380
You can think of it as a
posterior distribution.
00:26:47.380 --> 00:26:49.370
Now, what's the problem here?
00:26:49.370 --> 00:26:52.860
Well, every single point gives
us exactly the same value for
00:26:52.860 --> 00:26:54.140
this conditional distribution.
00:26:54.140 --> 00:26:56.780
And so there's no
unique MEP rule.
00:26:56.780 --> 00:27:02.790
Every single value of Y has
just the same conditional
00:27:02.790 --> 00:27:03.510
distribution.
00:27:03.510 --> 00:27:08.260
So there's no sensible way of
choosing a value based on the
00:27:08.260 --> 00:27:10.490
MEP rule in this case.
00:27:10.490 --> 00:27:16.090
But compare that with the LMS
estimator, which is just get
00:27:16.090 --> 00:27:17.560
conditional expectation.
00:27:17.560 --> 00:27:19.600
In that case, we can always
find a conditional
00:27:19.600 --> 00:27:20.580
expectation.
00:27:20.580 --> 00:27:23.640
In this case, the conditional
expectation is the midpoint,
00:27:23.640 --> 00:27:27.690
which is X/2, just as
had found in part a.
00:27:27.690 --> 00:27:31.640
OK, so in this problem, we
reviewed a bunch of different
00:27:31.640 --> 00:27:35.823
ideas in terms of inference, and
we took a joint pdf of X
00:27:35.823 --> 00:27:39.600
and Y, and we used that to
calculate the LMS estimator,
00:27:39.600 --> 00:27:41.200
the linear LMS estimator.
00:27:41.200 --> 00:27:44.280
We compared the two, and then we
also looked at why in this
00:27:44.280 --> 00:27:47.670
case, the MEP estimator doesn't
really make sense.
00:27:47.670 --> 00:27:49.330
All right, so I hope that
was helpful, and we'll
00:27:49.330 --> 00:27:50.580
see you next time.