1
00:00:00,000 --> 00:00:00,460
2
00:00:00,460 --> 00:00:00,660
Hi.
3
00:00:00,660 --> 00:00:03,240
In this session, we're going to
cover a nice review problem
4
00:00:03,240 --> 00:00:05,210
that will look at how
to infer one random
5
00:00:05,210 --> 00:00:06,880
variable based on another.
6
00:00:06,880 --> 00:00:09,090
And in this problem, we're given
two random variables--
7
00:00:09,090 --> 00:00:10,090
X and Y--
8
00:00:10,090 --> 00:00:13,950
and we're also given their joint
pdf, which we're told is
9
00:00:13,950 --> 00:00:16,840
a constant 2/3 within the
region bounded by
10
00:00:16,840 --> 00:00:17,930
these orange lines.
11
00:00:17,930 --> 00:00:21,280
And outside of the region,
the joint pdf is 0.
12
00:00:21,280 --> 00:00:24,150
So the first thing we're going
to look at, or the first thing
13
00:00:24,150 --> 00:00:29,140
that we're asked to do, is find
the LMS estimator of Y
14
00:00:29,140 --> 00:00:36,230
based on X. Now, remember that
LMS estimator is really just a
15
00:00:36,230 --> 00:00:37,990
conditional expectation.
16
00:00:37,990 --> 00:00:42,860
So the LMS estimator of Y based
on X is the conditional
17
00:00:42,860 --> 00:00:45,950
expectation of Y, given X.
18
00:00:45,950 --> 00:00:48,960
Now, when we have a plot of
the joint pdf and we're
19
00:00:48,960 --> 00:00:53,200
dealing with these two random
variables, and especially when
20
00:00:53,200 --> 00:00:56,620
the joint pdf is constant like
this, it's often easy to
21
00:00:56,620 --> 00:00:59,680
calculate this conditional
expectation visually.
22
00:00:59,680 --> 00:01:02,810
So what we really need to do
is just say, given any
23
00:01:02,810 --> 00:01:07,520
particular value of X, what
is the conditional
24
00:01:07,520 --> 00:01:08,670
expectation of Y?
25
00:01:08,670 --> 00:01:12,390
So what we can do is we can just
pick some values of X and
26
00:01:12,390 --> 00:01:15,520
see visually what that initial
expectation is.
27
00:01:15,520 --> 00:01:24,900
So for example, if X is 1/2,
given that X is 1/2, and since
28
00:01:24,900 --> 00:01:29,510
this whole joint pdf is uniform,
then the conditional
29
00:01:29,510 --> 00:01:32,780
slice of Y will be from
here to here.
30
00:01:32,780 --> 00:01:36,260
And that slice, the conditional
distribution of Y,
31
00:01:36,260 --> 00:01:38,910
given that X is 1/2, will
also be uniform.
32
00:01:38,910 --> 00:01:41,450
So it'll be uniform
from here to here.
33
00:01:41,450 --> 00:01:42,990
And if it's uniform, we know
that the conditional
34
00:01:42,990 --> 00:01:46,150
expectation will just be
the midpoint here.
35
00:01:46,150 --> 00:01:48,990
36
00:01:48,990 --> 00:01:51,670
And so that would be what the
conditional expectation of Y
37
00:01:51,670 --> 00:01:53,830
would be, given that X is 1/2.
38
00:01:53,830 --> 00:01:57,640
And we could do the same
thing for X equals 1.
39
00:01:57,640 --> 00:02:01,020
And we'll see that again,
because everything is uniform,
40
00:02:01,020 --> 00:02:03,030
this slice is also going
to be uniform.
41
00:02:03,030 --> 00:02:06,030
And so the conditional
expectation will again be the
42
00:02:06,030 --> 00:02:07,780
midpoint, which is there.
43
00:02:07,780 --> 00:02:12,220
And then if we just look at it
within this region, it's
44
00:02:12,220 --> 00:02:15,850
always going to be
the midpoint.
45
00:02:15,850 --> 00:02:18,820
And so we get that the initial
expectation of Y, given X,
46
00:02:18,820 --> 00:02:23,070
will just look like that line,
which you can think of it as
47
00:02:23,070 --> 00:02:29,420
just bisecting this angle formed
by these two parts of
48
00:02:29,420 --> 00:02:31,000
the region.
49
00:02:31,000 --> 00:02:33,230
But things are a little bit
different, though, when we
50
00:02:33,230 --> 00:02:37,160
move to the region where
X is between 1 and 2.
51
00:02:37,160 --> 00:02:44,480
Between 1 and 2, say
at 1 and 1/2, this
52
00:02:44,480 --> 00:02:45,370
line doesn't continue.
53
00:02:45,370 --> 00:02:51,690
Because now, the slice of Y goes
from here to here, and
54
00:02:51,690 --> 00:02:52,800
again, it's still uniform.
55
00:02:52,800 --> 00:02:58,400
So the midpoint would
be there.
56
00:02:58,400 --> 00:03:02,580
And similarly for X equals
2, it would be here.
57
00:03:02,580 --> 00:03:08,330
And so for X between 1 and 2,
the conditional expectation
58
00:03:08,330 --> 00:03:09,950
actually looks like this.
59
00:03:09,950 --> 00:03:12,720
So you see that there's actually
two linear parts of
60
00:03:12,720 --> 00:03:16,210
it, but there's a kink
at X equals 1.
61
00:03:16,210 --> 00:03:19,780
And so by looking at this
visually and taking advantage
62
00:03:19,780 --> 00:03:22,630
of the fact that everything is
uniform, we can pretty easily
63
00:03:22,630 --> 00:03:24,650
figure out what this conditional
expectation is.
64
00:03:24,650 --> 00:03:29,420
So now, let's actually just
write it out algebraically.
65
00:03:29,420 --> 00:03:34,620
So for X between 0 and 1, we
said that it's this line,
66
00:03:34,620 --> 00:03:41,490
which if we look at it, that's
just 1/2 of X. Now, this is
67
00:03:41,490 --> 00:03:44,800
for X between 0 and 1.
68
00:03:44,800 --> 00:03:49,000
And if X is between 1 and 2,
it's going to be this line,
69
00:03:49,000 --> 00:03:50,620
which is a slope of 1.
70
00:03:50,620 --> 00:03:53,600
And if we extend this
down, it hits the
71
00:03:53,600 --> 00:03:56,170
y-axis at negative 1/2.
72
00:03:56,170 --> 00:04:02,840
So it's X minus 1/2, if
X is between 1 and 2.
73
00:04:02,840 --> 00:04:06,110
74
00:04:06,110 --> 00:04:08,840
And otherwise, it's undefined.
75
00:04:08,840 --> 00:04:11,100
So we'll focus on these
two cases here.
76
00:04:11,100 --> 00:04:13,720
77
00:04:13,720 --> 00:04:21,209
Now, the second part of the
question, now that we know
78
00:04:21,209 --> 00:04:24,330
what the LMS estimator is, we're
asked to find what is
79
00:04:24,330 --> 00:04:27,860
the traditional mean squared
error of this estimator?
80
00:04:27,860 --> 00:04:29,590
So we want to know
how good is it.
81
00:04:29,590 --> 00:04:31,520
And one way of capturing
that is to look at the
82
00:04:31,520 --> 00:04:32,750
mean squared error.
83
00:04:32,750 --> 00:04:37,130
And so recall that the
conditional mean squared error
84
00:04:37,130 --> 00:04:41,755
is given by this expression.
85
00:04:41,755 --> 00:04:48,930
86
00:04:48,930 --> 00:04:52,510
So what we're saying is this is
what we estimate Y to be.
87
00:04:52,510 --> 00:04:56,930
This is what y really is, so
this difference is how wrong,
88
00:04:56,930 --> 00:04:59,370
or the error in our estimate.
89
00:04:59,370 --> 00:05:04,090
We square it, because otherwise,
positive and
90
00:05:04,090 --> 00:05:05,580
negative errors might cancel
each other out,
91
00:05:05,580 --> 00:05:06,610
so we square it.
92
00:05:06,610 --> 00:05:10,880
And then this just looking
at each individual
93
00:05:10,880 --> 00:05:12,750
value of x for now.
94
00:05:12,750 --> 00:05:13,805
So this is why it's
the conditional
95
00:05:13,805 --> 00:05:15,540
mean squared error.
96
00:05:15,540 --> 00:05:17,290
So how do we calculate this?
97
00:05:17,290 --> 00:05:24,390
Well, remember that this g of
X, we said the LMS estimator
98
00:05:24,390 --> 00:05:26,900
is just a conditional
expectation.
99
00:05:26,900 --> 00:05:30,220
So it's just expectation
of Y, given X.
100
00:05:30,220 --> 00:05:32,945
Well, then if you look at this,
what this reminds you
101
00:05:32,945 --> 00:05:35,900
of, it reminds you of the
definition of what a
102
00:05:35,900 --> 00:05:37,190
conditional variance is.
103
00:05:37,190 --> 00:05:42,420
A variance is just, you take the
random variable, subtract
104
00:05:42,420 --> 00:05:45,230
its mean, square it, and take
the expectation of that.
105
00:05:45,230 --> 00:05:47,690
This is no different, except
that everything is now the
106
00:05:47,690 --> 00:05:50,060
conditional world of X.
107
00:05:50,060 --> 00:05:55,410
So this is actually the
conditional variance of Y,
108
00:05:55,410 --> 00:05:59,800
given X is little x.
109
00:05:59,800 --> 00:06:01,720
What is the conditional
variance of Y, given
110
00:06:01,720 --> 00:06:02,980
that X is little x?
111
00:06:02,980 --> 00:06:05,300
Now, we can again go back
to this plot to
112
00:06:05,300 --> 00:06:06,560
try to help us out.
113
00:06:06,560 --> 00:06:10,710
We can split this up
into regions again.
114
00:06:10,710 --> 00:06:15,130
So just take some little x as
an example and see what the
115
00:06:15,130 --> 00:06:16,120
variance is.
116
00:06:16,120 --> 00:06:22,590
So if little x is 1/2, then we
know that the conditional
117
00:06:22,590 --> 00:06:24,760
distribution of Y would
be uniform, we said,
118
00:06:24,760 --> 00:06:26,540
from 0 up to here.
119
00:06:26,540 --> 00:06:29,670
Well, that point is this
from 0 to 1/2.
120
00:06:29,670 --> 00:06:32,860
And remember, the variance of a
uniform distribution is just
121
00:06:32,860 --> 00:06:34,180
the width of the uniform
distribution
122
00:06:34,180 --> 00:06:36,150
squared, divided by 12.
123
00:06:36,150 --> 00:06:42,060
And so in this case, the width
would be 1/2 squared over 12.
124
00:06:42,060 --> 00:06:48,240
And in general, for the region
of X between 0 and 1, the
125
00:06:48,240 --> 00:06:51,960
width of the conditional
distribution of Y will always
126
00:06:51,960 --> 00:06:58,080
be X, because the width will
go from 0 to wherever X is.
127
00:06:58,080 --> 00:07:02,980
So because of that, the
conditional variance will just
128
00:07:02,980 --> 00:07:09,800
be X squared, the width squared,
over 12, when X is
129
00:07:09,800 --> 00:07:12,340
between 0 and 1.
130
00:07:12,340 --> 00:07:14,640
Now, let's think about the
other case, where X is
131
00:07:14,640 --> 00:07:16,130
between 1 and 2.
132
00:07:16,130 --> 00:07:18,900
Well, if X is between 1 and
2, we're over here.
133
00:07:18,900 --> 00:07:22,440
And now, if we take the
conditional distribution of Y,
134
00:07:22,440 --> 00:07:23,740
it's again uniform.
135
00:07:23,740 --> 00:07:26,760
But the width now, instead of
varying with Y, it's always
136
00:07:26,760 --> 00:07:30,090
going to be the same width.
137
00:07:30,090 --> 00:07:33,360
Each of these slices have the
same width, and the width goes
138
00:07:33,360 --> 00:07:39,610
from here-- this is X minus
1, and that's X.
139
00:07:39,610 --> 00:07:42,550
So if the width is always going
to be a constant of 1.
140
00:07:42,550 --> 00:07:45,310
And so this variance is
going to be 1/12.
141
00:07:45,310 --> 00:07:48,360
142
00:07:48,360 --> 00:07:51,160
And from that, we get our answer
for the conditional
143
00:07:51,160 --> 00:07:52,110
mean squared error.
144
00:07:52,110 --> 00:07:55,820
Now, part c asks us to find the
mean squared error, which
145
00:07:55,820 --> 00:07:58,105
is given by this expression.
146
00:07:58,105 --> 00:08:02,220
147
00:08:02,220 --> 00:08:04,490
And we'll see that it looks very
similar to this, which
148
00:08:04,490 --> 00:08:06,070
was the conditional mean
squared error.
149
00:08:06,070 --> 00:08:08,520
And now, given what we
know from part b,
150
00:08:08,520 --> 00:08:09,710
this is easy to calculate.
151
00:08:09,710 --> 00:08:13,330
We can just apply total
expectation, because this is
152
00:08:13,330 --> 00:08:22,130
just equal to the integral
of the conditional
153
00:08:22,130 --> 00:08:23,380
mean squared error.
154
00:08:23,380 --> 00:08:27,430
155
00:08:27,430 --> 00:08:32,880
And then we need to also
multiply this by the pf of x,
156
00:08:32,880 --> 00:08:37,260
and then integrate over X. And
that integral will should be
157
00:08:37,260 --> 00:08:43,289
from X equals 0 to 2, because
that's the only range that
158
00:08:43,289 --> 00:08:46,240
applies for x, given
this joint pdf.
159
00:08:46,240 --> 00:08:48,310
Now, in order to do this first,
though, we need to
160
00:08:48,310 --> 00:08:54,610
figure out what the
pdf of X is.
161
00:08:54,610 --> 00:08:57,190
162
00:08:57,190 --> 00:09:00,620
In order to do that, we can go
back to our original joint pdf
163
00:09:00,620 --> 00:09:02,600
of X and Y and marginalize it.
164
00:09:02,600 --> 00:09:05,810
So marginalizing, you could
think of it as taking this
165
00:09:05,810 --> 00:09:10,510
joint pdf and collapsing it onto
the x-axis so that you
166
00:09:10,510 --> 00:09:13,410
take everything and
integrate out Y.
167
00:09:13,410 --> 00:09:15,900
Now to do that, let's
do that up here.
168
00:09:15,900 --> 00:09:20,310
169
00:09:20,310 --> 00:09:25,450
We can split it up into
two sections.
170
00:09:25,450 --> 00:09:32,540
So the section of X between 0
and 1, we integrate the joint
171
00:09:32,540 --> 00:09:38,620
pdf from Y equals 0 to Y
equals X, which is this
172
00:09:38,620 --> 00:09:40,320
portion of this line.
173
00:09:40,320 --> 00:09:50,790
So we integrate Y. The joint pdf
is 2/3, and we integrate Y
174
00:09:50,790 --> 00:10:05,180
out from Y equals 0 to X. And
then for the portion of X from
175
00:10:05,180 --> 00:10:07,890
1 to 2, we again integrate
Y out.
176
00:10:07,890 --> 00:10:20,620
Now we integrate Y from
X minus 1 up to X.
177
00:10:20,620 --> 00:10:30,810
So this is X between 0 and 1,
and this is X between 1 and 2.
178
00:10:30,810 --> 00:10:33,580
179
00:10:33,580 --> 00:10:39,280
So we just do some little bit
of calculus, and we get that
180
00:10:39,280 --> 00:10:46,030
this is going to be 2/3 X when
X is between 0 and 1.
181
00:10:46,030 --> 00:10:55,270
And it's going to be 2/3 when
X is between 1 and 2.
182
00:10:55,270 --> 00:10:59,350
So now that we have what the
marginal pdf of X is, we can
183
00:10:59,350 --> 00:11:03,170
plug that into this, and plug in
what we had for b, and then
184
00:11:03,170 --> 00:11:04,910
calculate what this
actually is.
185
00:11:04,910 --> 00:11:08,810
So remember, we need to take
care of these two cases, these
186
00:11:08,810 --> 00:11:09,390
two regions--
187
00:11:09,390 --> 00:11:11,720
X between 0 and 1, and
X between 1 and 2.
188
00:11:11,720 --> 00:11:17,160
So the conditional mean squared
error for X between 0
189
00:11:17,160 --> 00:11:19,780
and 1 is X squared over 2.
190
00:11:19,780 --> 00:11:26,390
So between 0 and 1, this first
part is X squared over 12.
191
00:11:26,390 --> 00:11:32,745
The pdf of X in that same
region is 2/3 x.
192
00:11:32,745 --> 00:11:37,830
And we integrate that in the
region from x equals 0 to 1.
193
00:11:37,830 --> 00:11:41,340
And then, we also have the
second region which
194
00:11:41,340 --> 00:11:43,280
is X from 1 to 2.
195
00:11:43,280 --> 00:11:45,690
In that region, the traditional
mean squared error
196
00:11:45,690 --> 00:11:47,990
from part b is 1/12.
197
00:11:47,990 --> 00:11:55,560
The marginal pdf of X is 2/3,
and we do this integral.
198
00:11:55,560 --> 00:11:59,530
And if you just carry out some
calculus here, you'll get that
199
00:11:59,530 --> 00:12:02,525
the final answer is
equal to 5/72.
200
00:12:02,525 --> 00:12:08,660
201
00:12:08,660 --> 00:12:14,420
Now, the last part of this
question asks us, is this mean
202
00:12:14,420 --> 00:12:17,380
squared error the same thing--
203
00:12:17,380 --> 00:12:24,475
does it equal the expectation
of the conditional variance?
204
00:12:24,475 --> 00:12:27,000
205
00:12:27,000 --> 00:12:29,480
And it turns out that
yes, it does.
206
00:12:29,480 --> 00:12:36,980
And to see that, we can just
take this, and use the law of
207
00:12:36,980 --> 00:12:41,100
iterated expectations, because
iterated expectations tells us
208
00:12:41,100 --> 00:12:50,220
this is in fact equal to the
expectation of Y minus g of X
209
00:12:50,220 --> 00:12:56,580
squared, given X. That's just
applying law of iterated
210
00:12:56,580 --> 00:12:59,320
expectations.
211
00:12:59,320 --> 00:13:04,350
And then, if we look at this,
this part that's inside is
212
00:13:04,350 --> 00:13:07,200
exactly equal to the conditional
variance of Y,
213
00:13:07,200 --> 00:13:11,250
given X. And so these two
are, in fact, the same.
214
00:13:11,250 --> 00:13:15,980
In part c, we calculated what
the marginal pdf of X is, and
215
00:13:15,980 --> 00:13:18,530
it'll actually be used later
on in this problem.
216
00:13:18,530 --> 00:13:20,800
So for future reference, let's
just write it down here in
217
00:13:20,800 --> 00:13:22,140
this corner.
218
00:13:22,140 --> 00:13:24,490
Now, so far in this
problem, we've
219
00:13:24,490 --> 00:13:27,920
looked at the LMS estimator.
220
00:13:27,920 --> 00:13:29,590
And of course, there are
other estimators that
221
00:13:29,590 --> 00:13:30,550
you can use as well.
222
00:13:30,550 --> 00:13:35,240
And now in part d, let's look
at the linear LMS estimator.
223
00:13:35,240 --> 00:13:42,810
Now remember, the linear LMS
estimator is special, because
224
00:13:42,810 --> 00:13:48,040
it forces the estimator to have
a linear relationship.
225
00:13:48,040 --> 00:13:51,310
So the estimator is going to
be a linear function of X.
226
00:13:51,310 --> 00:13:53,820
Now, compare that
to what the LMS
227
00:13:53,820 --> 00:13:55,590
estimator was in this case.
228
00:13:55,590 --> 00:13:58,790
It was two linear pieces,
but there was a kink.
229
00:13:58,790 --> 00:14:03,150
And so the entire estimator
wasn't actually linear in X.
230
00:14:03,150 --> 00:14:07,360
Now, the LLMS estimator, or the
linear LMS estimator, will
231
00:14:07,360 --> 00:14:10,180
give us the linear estimator.
232
00:14:10,180 --> 00:14:11,610
It's going to be a linear
function of X.
233
00:14:11,610 --> 00:14:15,910
And we know that we have
a formula for this.
234
00:14:15,910 --> 00:14:23,930
Is the expectation of Y plus the
covariance of X and Y over
235
00:14:23,930 --> 00:14:32,770
the variance of X times X minus
expectation of X. All
236
00:14:32,770 --> 00:14:35,350
right, so in order to calculate
what this is, we
237
00:14:35,350 --> 00:14:38,810
just need to calculate
what four things are.
238
00:14:38,810 --> 00:14:43,080
Now, let's start with this last
one, the expected value
239
00:14:43,080 --> 00:14:47,900
of X. To calculate the expected
value of X, we just
240
00:14:47,900 --> 00:14:48,590
use a formula.
241
00:14:48,590 --> 00:14:52,010
And from before, we know
what the pdf of X is.
242
00:14:52,010 --> 00:14:56,910
And so we know that this
is just going to be X
243
00:14:56,910 --> 00:15:01,160
times fx of x dx.
244
00:15:01,160 --> 00:15:09,150
And in particular, this will
give us that from 0 to 1, it's
245
00:15:09,150 --> 00:15:13,200
going to be X times the
pdf of X is 2/3 X,
246
00:15:13,200 --> 00:15:16,486
so it's 2/3 X squared.
247
00:15:16,486 --> 00:15:19,720
248
00:15:19,720 --> 00:15:24,950
And from 1 to 2, it's going to
be X times the pdf of X, which
249
00:15:24,950 --> 00:15:31,200
is just 2/3, so it's 2/3 X dx.
250
00:15:31,200 --> 00:15:34,000
And when you calculate
this out, you'll get
251
00:15:34,000 --> 00:15:38,060
that is equal to 11/9.
252
00:15:38,060 --> 00:15:41,250
Now, let's calculate the
variance of X next.
253
00:15:41,250 --> 00:15:43,760
In order to calculate that,
let's use the formula that
254
00:15:43,760 --> 00:15:46,920
variance is equal to the
expectation of X squared minus
255
00:15:46,920 --> 00:15:49,220
the expectation of X
quantity squared.
256
00:15:49,220 --> 00:15:52,440
We had the expectation of X,
so let's calculate what the
257
00:15:52,440 --> 00:15:56,940
expectation of X squared is.
258
00:15:56,940 --> 00:15:59,060
Now, it's the same idea.
259
00:15:59,060 --> 00:16:06,060
Instead, we have X squared
times f of X dx.
260
00:16:06,060 --> 00:16:08,860
And we'll get the same
sort of formula.
261
00:16:08,860 --> 00:16:11,880
We'll split it up again into
two different parts from X
262
00:16:11,880 --> 00:16:14,490
equals 0 to X equals 1.
263
00:16:14,490 --> 00:16:16,260
It's going to be X squared
times pdf, so
264
00:16:16,260 --> 00:16:20,430
it's 2/3 X cubed dx.
265
00:16:20,430 --> 00:16:23,490
And then from X equals 1 to
2, it's going to be X
266
00:16:23,490 --> 00:16:25,060
squared times 2/3.
267
00:16:25,060 --> 00:16:28,770
So it's 2/3 X squared dx.
268
00:16:28,770 --> 00:16:30,890
And when we calculate this
out, we'll get that
269
00:16:30,890 --> 00:16:32,750
it's equal to 31/18.
270
00:16:32,750 --> 00:16:35,900
271
00:16:35,900 --> 00:16:38,730
From that, we know that the
variance is going to be equal
272
00:16:38,730 --> 00:16:44,680
to expectation of X squared
minus expectation of X
273
00:16:44,680 --> 00:16:46,860
quantity squared.
274
00:16:46,860 --> 00:16:52,940
Now, expectation of X
squared is 31/18.
275
00:16:52,940 --> 00:16:55,010
Expectation of X is 11/9.
276
00:16:55,010 --> 00:16:58,010
And when we calculate this,
we get that the
277
00:16:58,010 --> 00:17:06,400
variance is equal to 37/162.
278
00:17:06,400 --> 00:17:08,349
So now we have this,
and we have that.
279
00:17:08,349 --> 00:17:11,170
Let's calculate what expectation
of Y is.
280
00:17:11,170 --> 00:17:16,190
Expectation of Y, let's
calculate it using the law of
281
00:17:16,190 --> 00:17:19,310
iterated expectations.
282
00:17:19,310 --> 00:17:20,944
The law of iterated expectations
tells us that
283
00:17:20,944 --> 00:17:25,849
this is equal to the expectation
of Y conditioned
284
00:17:25,849 --> 00:17:27,700
on X.
285
00:17:27,700 --> 00:17:29,410
Now, we already know what
expectation of Y
286
00:17:29,410 --> 00:17:30,430
conditioned on X is.
287
00:17:30,430 --> 00:17:33,400
That was the LMS estimator that
we calculated earlier.
288
00:17:33,400 --> 00:17:35,770
It's this.
289
00:17:35,770 --> 00:17:39,760
So now we just need to
calculate this out,
290
00:17:39,760 --> 00:17:41,650
and we can do that.
291
00:17:41,650 --> 00:17:48,210
So we know that in the range
from X between 0 and 1, it's
292
00:17:48,210 --> 00:17:55,720
equal to 1/2 X. So in the range
from 0 to 1, it's equal
293
00:17:55,720 --> 00:18:04,130
to 1/2 X. But then, we have to
use total expectation, so we
294
00:18:04,130 --> 00:18:07,940
have to multiply by the pdf
of X in that region
295
00:18:07,940 --> 00:18:11,790
which is 2/3 X dx.
296
00:18:11,790 --> 00:18:16,900
And then in the range from
X equals 1 to 2, this
297
00:18:16,900 --> 00:18:18,840
conditional expectation
is X minus 1/2.
298
00:18:18,840 --> 00:18:22,100
299
00:18:22,100 --> 00:18:25,440
And the pdf of X in that
region is 2/3.
300
00:18:25,440 --> 00:18:30,200
301
00:18:30,200 --> 00:18:35,500
Now, when we calculate out this
value, we'll get that
302
00:18:35,500 --> 00:18:38,060
it's equal to 7/9.
303
00:18:38,060 --> 00:18:40,620
304
00:18:40,620 --> 00:18:44,770
And now, the last piece is the
covariance of X and Y.
305
00:18:44,770 --> 00:18:47,760
Remember, the covariance, we
can calculate that as the
306
00:18:47,760 --> 00:18:52,810
expectation of X times Y minus
the expectation of X times the
307
00:18:52,810 --> 00:18:56,030
expectation of Y. We already
know the expectation of X and
308
00:18:56,030 --> 00:18:58,590
the expectation of Y, so we
just need to calculate the
309
00:18:58,590 --> 00:19:03,212
expectation of X times Y,
the product of the two.
310
00:19:03,212 --> 00:19:08,740
And for that, we'll use the
definition, and we'll use the
311
00:19:08,740 --> 00:19:11,990
joint pdf that we have.
312
00:19:11,990 --> 00:19:16,610
So this is going to be a double
integral of X times Y
313
00:19:16,610 --> 00:19:17,860
times the joint pdf.
314
00:19:17,860 --> 00:19:22,360
315
00:19:22,360 --> 00:19:25,920
And the tricky part here is just
figuring out what these
316
00:19:25,920 --> 00:19:26,430
limits are.
317
00:19:26,430 --> 00:19:30,312
So we'll integrate
in this order--
318
00:19:30,312 --> 00:19:32,590
X and Y.
319
00:19:32,590 --> 00:19:35,880
Now, let's split this up.
320
00:19:35,880 --> 00:19:38,450
So let's focus on
splitting X up.
321
00:19:38,450 --> 00:19:46,990
So for X between 0 and 1, we
just need to figure out what's
322
00:19:46,990 --> 00:19:51,240
the rate right range of Y to
integrate over such that this
323
00:19:51,240 --> 00:19:52,120
is actually non-zero.
324
00:19:52,120 --> 00:19:55,160
Because remember, the
joint pdf is easy.
325
00:19:55,160 --> 00:19:56,230
It's just a constant 2/3.
326
00:19:56,230 --> 00:19:58,660
But it's only a constant
2/3 within this region.
327
00:19:58,660 --> 00:20:01,110
So the difficult part is just
figuring out what the limits
328
00:20:01,110 --> 00:20:03,100
are in order to specify
that region.
329
00:20:03,100 --> 00:20:11,410
So for X between 0 and 1, Y
has to be between 0 and X,
330
00:20:11,410 --> 00:20:18,100
because this line is Y equals
X. So we need to integrate
331
00:20:18,100 --> 00:20:19,926
from 0 to X--
332
00:20:19,926 --> 00:20:23,595
X times Y times the joint
pdf, which is 2/3.
333
00:20:23,595 --> 00:20:28,260
334
00:20:28,260 --> 00:20:34,380
And now, let's do the other
part, which is X from 1 to 2.
335
00:20:34,380 --> 00:20:38,170
Well, if X is from 1 to 2, in
order to fall into this
336
00:20:38,170 --> 00:20:43,210
region, Y has to be between X
minus 1 and X. So we integrate
337
00:20:43,210 --> 00:20:48,110
Y from X minus 1 to X. Against,
it's X times Y times
338
00:20:48,110 --> 00:20:49,640
the joint pdf, which is 2/3.
339
00:20:49,640 --> 00:20:54,070
340
00:20:54,070 --> 00:20:57,360
And now, once we have this set
up, the rest of it we can just
341
00:20:57,360 --> 00:20:58,400
do some calculus.
342
00:20:58,400 --> 00:21:00,680
And what we find is
that the final
343
00:21:00,680 --> 00:21:03,940
answer is equal to 41/36.
344
00:21:03,940 --> 00:21:07,940
345
00:21:07,940 --> 00:21:13,640
Now, what that tells us is that
the covariance of X and
346
00:21:13,640 --> 00:21:19,430
Y, which is just expectation
of X times Y, the product,
347
00:21:19,430 --> 00:21:25,080
minus expectation of X times
expectation of Y.
348
00:21:25,080 --> 00:21:27,640
We know expectation
of X times Y now.
349
00:21:27,640 --> 00:21:29,440
It's 41/36.
350
00:21:29,440 --> 00:21:31,920
Expectation of X is 11/9.
351
00:21:31,920 --> 00:21:34,640
Expectation of Y is 7/9.
352
00:21:34,640 --> 00:21:37,050
So when we substitute all of
that in, we get that this
353
00:21:37,050 --> 00:21:43,830
covariance is 61/324.
354
00:21:43,830 --> 00:21:45,910
All right, so now we have
everything we need.
355
00:21:45,910 --> 00:21:48,760
Expectation of Y is here.
356
00:21:48,760 --> 00:21:51,620
Covariance is here.
357
00:21:51,620 --> 00:21:54,360
Variance of X is here.
358
00:21:54,360 --> 00:21:57,210
And expectation of X is here.
359
00:21:57,210 --> 00:22:01,970
So let's substitute that in, and
we can figure out what the
360
00:22:01,970 --> 00:22:05,410
actual LLMS estimator is.
361
00:22:05,410 --> 00:22:09,950
So expectation of Y
we know is 7/9.
362
00:22:09,950 --> 00:22:13,570
Expectation of X is 11/9.
363
00:22:13,570 --> 00:22:16,800
364
00:22:16,800 --> 00:22:20,980
And when you divide the
covariance, which is 61/324,
365
00:22:20,980 --> 00:22:32,040
by the variance, which is
37/162, we'll get 61/74.
366
00:22:32,040 --> 00:22:36,290
And so that is the LLMS
estimator that we calculated.
367
00:22:36,290 --> 00:22:40,760
And notice that it is,
in fact, linear in X.
368
00:22:40,760 --> 00:22:44,720
So let's plot that and see
what it looks like.
369
00:22:44,720 --> 00:22:52,300
So it's going to be a
line, and it's going
370
00:22:52,300 --> 00:22:54,610
to look like this.
371
00:22:54,610 --> 00:23:00,700
So at X equals 2, it's actually
a little bit below 1
372
00:23:00,700 --> 00:23:05,360
and 1/2, which is what the
LMS estimator would be.
373
00:23:05,360 --> 00:23:13,010
At X equals 1, it's actually a
little bit above 1/2, which is
374
00:23:13,010 --> 00:23:15,150
what the LMS estimator
would be.
375
00:23:15,150 --> 00:23:21,120
And then it crosses 0 around
roughly 1/4, and it drops
376
00:23:21,120 --> 00:23:22,160
actually below 0.
377
00:23:22,160 --> 00:23:26,040
So if we connect the dots,
it's going to look
378
00:23:26,040 --> 00:23:28,820
something like this.
379
00:23:28,820 --> 00:23:32,230
So notice that it's actually not
too far away from the LMS
380
00:23:32,230 --> 00:23:33,850
estimator here.
381
00:23:33,850 --> 00:23:38,340
But it doesn't have the kink
because it is a line.
382
00:23:38,340 --> 00:23:41,700
And note also that it actually
drops below.
383
00:23:41,700 --> 00:23:45,070
So when X is very small, you
actually estimate negative
384
00:23:45,070 --> 00:23:51,880
values of Y, which is actually
impossible, given the joint
385
00:23:51,880 --> 00:23:53,410
pdf distribution that
we're given.
386
00:23:53,410 --> 00:23:57,950
And that is sometimes a feature
or artifact of the
387
00:23:57,950 --> 00:24:00,530
linear LMS estimator, that
you'll get values that don't
388
00:24:00,530 --> 00:24:03,820
necessarily seem
to make sense.
389
00:24:03,820 --> 00:24:06,720
So now that we've calculated
the linear LMS estimator in
390
00:24:06,720 --> 00:24:10,740
part d, which is this, and the
LMS estimator in part a, which
391
00:24:10,740 --> 00:24:14,420
is this, we've also compared
them visually.
392
00:24:14,420 --> 00:24:17,750
The linear LMS estimator
is the one in pink,
393
00:24:17,750 --> 00:24:18,680
the straight line.
394
00:24:18,680 --> 00:24:22,080
And the LMS estimator is the
one in black with the kink.
395
00:24:22,080 --> 00:24:24,170
It's an interesting question
to now ask, which
396
00:24:24,170 --> 00:24:25,480
one of these is better?
397
00:24:25,480 --> 00:24:27,390
And in order to judge that, we
need to come up with some sort
398
00:24:27,390 --> 00:24:30,060
of criterion to compare
the two with.
399
00:24:30,060 --> 00:24:34,280
And the one that we're going
to look at in part e is the
400
00:24:34,280 --> 00:24:35,040
mean squared error.
401
00:24:35,040 --> 00:24:37,420
Which one gives the lower
mean squared error.
402
00:24:37,420 --> 00:24:44,530
And so specifically, we're going
to ask ourselves which
403
00:24:44,530 --> 00:24:49,075
of these two estimators
gives us the smaller
404
00:24:49,075 --> 00:24:50,325
mean squared error?
405
00:24:50,325 --> 00:24:54,000
406
00:24:54,000 --> 00:24:57,500
Is it the linear LMS estimator
given by l of X?
407
00:24:57,500 --> 00:25:02,180
Or is it the LMS estimator,
given by g of X?
408
00:25:02,180 --> 00:25:08,400
Now, we know that the LMS
estimator is the one that
409
00:25:08,400 --> 00:25:09,560
actually minimizes this.
410
00:25:09,560 --> 00:25:13,600
The LMS estimator is designed
to minimize the
411
00:25:13,600 --> 00:25:14,140
mean squared error.
412
00:25:14,140 --> 00:25:18,030
And so we know that given any
estimator of X, this one will
413
00:25:18,030 --> 00:25:20,470
have the smallest mean
squared error.
414
00:25:20,470 --> 00:25:24,430
And so the linear LMS estimator,
its mean squared
415
00:25:24,430 --> 00:25:29,480
error has to be at least as
large as the LMS estimators.
416
00:25:29,480 --> 00:25:32,220
And the last part of the
question now asks us to look
417
00:25:32,220 --> 00:25:37,680
at a third type of estimator,
which is the MEP estimator.
418
00:25:37,680 --> 00:25:41,390
Now, we want to ask, why is it
that we haven't been using the
419
00:25:41,390 --> 00:25:43,260
MEP estimator in this problem?
420
00:25:43,260 --> 00:25:45,360
Well, remember what the
MEP estimator does.
421
00:25:45,360 --> 00:25:50,730
In this case, what we would
do is it would take the
422
00:25:50,730 --> 00:25:54,030
conditional distribution ratio
of Y given any value of X. And
423
00:25:54,030 --> 00:25:59,200
then it would pick the value
of Y that gives the highest
424
00:25:59,200 --> 00:26:01,780
value in the conditional
distribution.
425
00:26:01,780 --> 00:26:04,190
And that would be the
MEP estimate of Y.
426
00:26:04,190 --> 00:26:08,060
But the problem in this case is
that if you take any slice
427
00:26:08,060 --> 00:26:11,700
here, so a condition on any
value of X, any of these
428
00:26:11,700 --> 00:26:18,240
slices, if you just take this
out and look at it, it's going
429
00:26:18,240 --> 00:26:20,040
to be uniform.
430
00:26:20,040 --> 00:26:28,910
This is what the conditional
distribution of Y given X is.
431
00:26:28,910 --> 00:26:33,662
It's going to be uniform between
0 and X. Now, what the
432
00:26:33,662 --> 00:26:37,610
MEP rule tells us is we're going
to pick the value of Y
433
00:26:37,610 --> 00:26:43,310
that gives us the highest point
in this conditional
434
00:26:43,310 --> 00:26:44,190
distribution.
435
00:26:44,190 --> 00:26:47,380
You can think of it as a
posterior distribution.
436
00:26:47,380 --> 00:26:49,370
Now, what's the problem here?
437
00:26:49,370 --> 00:26:52,860
Well, every single point gives
us exactly the same value for
438
00:26:52,860 --> 00:26:54,140
this conditional distribution.
439
00:26:54,140 --> 00:26:56,780
And so there's no
unique MEP rule.
440
00:26:56,780 --> 00:27:02,790
Every single value of Y has
just the same conditional
441
00:27:02,790 --> 00:27:03,510
distribution.
442
00:27:03,510 --> 00:27:08,260
So there's no sensible way of
choosing a value based on the
443
00:27:08,260 --> 00:27:10,490
MEP rule in this case.
444
00:27:10,490 --> 00:27:16,090
But compare that with the LMS
estimator, which is just get
445
00:27:16,090 --> 00:27:17,560
conditional expectation.
446
00:27:17,560 --> 00:27:19,600
In that case, we can always
find a conditional
447
00:27:19,600 --> 00:27:20,580
expectation.
448
00:27:20,580 --> 00:27:23,640
In this case, the conditional
expectation is the midpoint,
449
00:27:23,640 --> 00:27:27,690
which is X/2, just as
had found in part a.
450
00:27:27,690 --> 00:27:31,640
OK, so in this problem, we
reviewed a bunch of different
451
00:27:31,640 --> 00:27:35,823
ideas in terms of inference, and
we took a joint pdf of X
452
00:27:35,823 --> 00:27:39,600
and Y, and we used that to
calculate the LMS estimator,
453
00:27:39,600 --> 00:27:41,200
the linear LMS estimator.
454
00:27:41,200 --> 00:27:44,280
We compared the two, and then we
also looked at why in this
455
00:27:44,280 --> 00:27:47,670
case, the MEP estimator doesn't
really make sense.
456
00:27:47,670 --> 00:27:49,330
All right, so I hope that
was helpful, and we'll
457
00:27:49,330 --> 00:27:50,580
see you next time.
458
00:27:50,580 --> 00:27:51,430