WEBVTT
00:00:01.150 --> 00:00:03.900
We now look at an application
of the Bayes rule that's
00:00:03.900 --> 00:00:08.039
involves a continuous unknown
random variable, which we try
00:00:08.039 --> 00:00:11.730
to estimate based on a
discrete measurement.
00:00:11.730 --> 00:00:14.120
Our model will be as follows.
00:00:14.120 --> 00:00:17.790
We observe the discrete random
variable K, which is
00:00:17.790 --> 00:00:22.870
Bernoulli, so it can take
two values, 1 or 0.
00:00:22.870 --> 00:00:27.010
And it takes those values with
probabilities y and 1 minus y,
00:00:27.010 --> 00:00:27.740
respectively.
00:00:27.740 --> 00:00:32.750
This is our model of K. The
catch is that the value of y
00:00:32.750 --> 00:00:34.110
is not known.
00:00:34.110 --> 00:00:38.710
And it is modeled as a random
variable by itself.
00:00:38.710 --> 00:00:41.970
You can think of a situation
where we are dealing with a
00:00:41.970 --> 00:00:43.390
single coin flip.
00:00:43.390 --> 00:00:46.250
We observe the outcome
of the coin flip,
00:00:46.250 --> 00:00:48.260
but the coin is biased.
00:00:48.260 --> 00:00:52.090
The probability of heads is
some unknown number, y.
00:00:52.090 --> 00:00:56.290
And we try to infer or say
something about the bias of
00:00:56.290 --> 00:01:00.290
the coin on the basis of the
observation that we have made.
00:01:00.290 --> 00:01:03.350
So what do we assume
about this y or
00:01:03.350 --> 00:01:05.950
the bias of the coin?
00:01:05.950 --> 00:01:08.850
If we know nothing about this
random variable, we might as
00:01:08.850 --> 00:01:11.300
well model it as a
uniform random
00:01:11.300 --> 00:01:14.350
variable on the unit interval.
00:01:14.350 --> 00:01:18.510
And the question now is, given
that we made one observation
00:01:18.510 --> 00:01:23.400
and the outcome was 1, what
can we say about the
00:01:23.400 --> 00:01:27.010
probability distribution of
Y given this particular
00:01:27.010 --> 00:01:29.340
information?
00:01:29.340 --> 00:01:32.350
So the question that we're
asking is, what we can tell
00:01:32.350 --> 00:01:38.180
about the density of Y
given that the value
00:01:38.180 --> 00:01:42.170
of 1 has been observed.
00:01:42.170 --> 00:01:45.740
The way to approach this problem
is by using a version
00:01:45.740 --> 00:01:47.930
of the Bayes rule.
00:01:47.930 --> 00:01:52.030
We want to calculate this
quantity for the special case
00:01:52.030 --> 00:01:56.470
where k is equal to 1.
00:01:56.470 --> 00:02:00.060
So let us calculate the various
pieces on the right
00:02:00.060 --> 00:02:01.990
hand side of this equation.
00:02:01.990 --> 00:02:05.570
The first piece is the density
of Y. This is the prior
00:02:05.570 --> 00:02:08.389
density before we obtain
any measurement.
00:02:08.389 --> 00:02:12.750
And since the random variable is
uniform, this is equal to 1
00:02:12.750 --> 00:02:17.760
for y in the unit interval.
00:02:17.760 --> 00:02:21.423
And of course, it
is 0 otherwise.
00:02:26.329 --> 00:02:31.195
The next piece that we need is
the distribution of K given
00:02:31.195 --> 00:02:38.550
the value of Y. Well, given Y,
K takes a value of 1, with
00:02:38.550 --> 00:02:41.350
probability equal to Y--
00:02:41.350 --> 00:02:45.350
so the probability of 1, if
we're told the value of y is
00:02:45.350 --> 00:02:46.890
just a y itself.
00:02:46.890 --> 00:02:51.770
y is the bias of the coin
that we're dealing with.
00:02:51.770 --> 00:02:56.410
The next term that we need
is the denominator.
00:02:56.410 --> 00:02:58.460
We will use this formula.
00:02:58.460 --> 00:03:03.070
It is the integral of
the density of Y,
00:03:03.070 --> 00:03:04.910
which is equal to 1.
00:03:04.910 --> 00:03:10.330
And it is equal to 1 only on the
range from 0 to 1, times
00:03:10.330 --> 00:03:16.430
this probability that K takes
a value, a certain value.
00:03:16.430 --> 00:03:19.770
In this case, we're dealing
with a value of 1, so here
00:03:19.770 --> 00:03:23.640
we're going to put
1 instead of k.
00:03:23.640 --> 00:03:26.310
And therefore, we're dealing
with this expression here,
00:03:26.310 --> 00:03:28.180
which is just y.
00:03:28.180 --> 00:03:31.200
And we integrated over y's.
00:03:31.200 --> 00:03:37.690
So this is y squared over 2,
evaluated at 0 and 1, which
00:03:37.690 --> 00:03:40.220
gives us 1/2.
00:03:40.220 --> 00:03:42.850
So this is the unconditional
probability that
00:03:42.850 --> 00:03:44.140
K is equal to 1.
00:03:44.140 --> 00:03:48.920
If we know nothing about Y, by
symmetry, higher biases are
00:03:48.920 --> 00:03:51.230
equally likely as
lower biases.
00:03:51.230 --> 00:03:55.760
So we should expect that it's
equally likely to give us a 1
00:03:55.760 --> 00:03:58.130
as it is to give us a 0.
00:03:58.130 --> 00:04:01.470
Now, we have in our hands all
the pieces that go into this
00:04:01.470 --> 00:04:02.940
particular formula.
00:04:02.940 --> 00:04:05.920
And we can go ahead with
the final calculation.
00:04:05.920 --> 00:04:13.180
So in the numerator, we have 1
times this term, evaluated at
00:04:13.180 --> 00:04:16.890
k equal to 1, which
is equal to y.
00:04:16.890 --> 00:04:19.810
And then in the denominator,
we have a term that
00:04:19.810 --> 00:04:21.950
evaluates to 1/2.
00:04:21.950 --> 00:04:25.110
So the final answer is 2y.
00:04:25.110 --> 00:04:28.520
Over what range of y's
is this correct?
00:04:28.520 --> 00:04:30.800
Only for those y's that
are possible.
00:04:30.800 --> 00:04:33.909
So this is for y's in
the unit interval.
00:04:36.409 --> 00:04:41.159
If we are to plot this PDF,
it has this shape.
00:04:46.620 --> 00:04:52.820
This is a plot of the PDF of Y
given that the random variable
00:04:52.820 --> 00:04:55.990
K takes on a value of 1.
00:04:55.990 --> 00:05:01.090
Initially, we started with a
uniform for Y. So all values
00:05:01.090 --> 00:05:05.370
of Y were equally likely.
00:05:05.370 --> 00:05:12.920
But once we observed an outcome
of 1, this tells us
00:05:12.920 --> 00:05:18.410
that perhaps Y is on
the higher end
00:05:18.410 --> 00:05:20.180
rather than lower end.
00:05:20.180 --> 00:05:23.390
So after we obtain our
observation, the random
00:05:23.390 --> 00:05:26.840
variable Y has this
distribution, with higher
00:05:26.840 --> 00:05:30.640
values being more likely
than lower values.
00:05:30.640 --> 00:05:33.960
This example is a prototype of
situations where we want to
00:05:33.960 --> 00:05:37.210
estimate a continuous random
variable based on discrete
00:05:37.210 --> 00:05:38.690
measurements.
00:05:38.690 --> 00:05:42.020
Essentially it is the same as
trying to estimate the bias of
00:05:42.020 --> 00:05:46.820
a coin based on a single
measurement of the
00:05:46.820 --> 00:05:48.860
result of a coin flip.
00:05:48.860 --> 00:05:52.070
As you can imagine, there are
generalizations in which we
00:05:52.070 --> 00:05:54.950
observe multiple coin flips.
00:05:54.950 --> 00:05:57.040
And this is an example
that we will see
00:05:57.040 --> 00:05:58.440
later on in this class.