WEBVTT
00:00:00.810 --> 00:00:03.200
In this exercise, we'll be
working with the notion of
00:00:03.200 --> 00:00:06.100
convergence in probability, as
well as some other notion of
00:00:06.100 --> 00:00:09.810
converge of random variables
that we'll introduce later.
00:00:09.810 --> 00:00:14.930
First type of random variable
is xn, where xn has
00:00:14.930 --> 00:00:24.570
probability 1 minus 1 minus
over n to be as 0 and
00:00:24.570 --> 00:00:28.010
probability of 1 over
n to be a 1.
00:00:30.776 --> 00:00:37.230
And graphically, we see that
we have a pretty big mess.
00:00:37.230 --> 00:00:42.470
1 minus 1 over n at location
0, and a tiny bit somewhere
00:00:42.470 --> 00:00:45.040
here, only 1 over n.
00:00:45.040 --> 00:00:48.315
So this will be the PMF for x.
00:00:52.330 --> 00:00:54.110
On the other hand, we
have the sequence of
00:00:54.110 --> 00:00:56.890
random variables, yn.
00:00:56.890 --> 00:01:00.410
Fairly similar to xn with
a slight tweak.
00:01:00.410 --> 00:01:04.410
The similar part says it also
has a very high probability of
00:01:04.410 --> 00:01:08.230
being at 0, mass 1
over 1 minus n.
00:01:08.230 --> 00:01:12.670
But on the off chance that yn
is not at 0, it has a pretty
00:01:12.670 --> 00:01:14.080
big value n.
00:01:14.080 --> 00:01:17.810
So it has probability 1 over
n of somewhere out there.
00:01:17.810 --> 00:01:21.800
So to contrast the two graphs,
we see at 0, they have the
00:01:21.800 --> 00:01:26.400
same amount of mass, 1 over 1
minus n, but for y, it's all
00:01:26.400 --> 00:01:29.820
the way out there that has
a small mass 1 over n.
00:01:29.820 --> 00:01:33.260
So this will be our Pyn of y.
00:01:37.770 --> 00:01:40.180
And for the remainder of the
problem, we'll be looking at
00:01:40.180 --> 00:01:44.980
the regime where the number n
tends to infinity, and study
00:01:44.980 --> 00:01:49.990
what will happen to these two
sequences of random variables.
00:01:49.990 --> 00:01:53.250
In part A, we're to compute the
expected value n variance
00:01:53.250 --> 00:01:55.230
for both xn and yn.
00:01:55.230 --> 00:01:56.650
Let's get started.
00:01:56.650 --> 00:02:03.330
The expected value of xn is
given by the probability--
00:02:03.330 --> 00:02:08.669
it's at one, which is 1 over n
times 1 plus the probability
00:02:08.669 --> 00:02:12.670
being at 0, 1 over
n times value 0.
00:02:12.670 --> 00:02:15.920
And that gives us 1 over n.
00:02:15.920 --> 00:02:21.550
To calculate the variance of
xn, see that variance is
00:02:21.550 --> 00:02:27.280
simply the expected value of xn
minus expected value of xn,
00:02:27.280 --> 00:02:31.890
which in this case is 1 over n
from the previous calculation
00:02:31.890 --> 00:02:33.130
we have here.
00:02:33.130 --> 00:02:35.490
We take the square of this value
and compute the whole
00:02:35.490 --> 00:02:44.560
expectation, and this gives us
1 over n, 1 minus 1 over n
00:02:44.560 --> 00:02:49.700
plus the remainder probability 1
over 1 minus n of x being at
00:02:49.700 --> 00:02:52.956
0, so 0 minus 1 over
n squared.
00:02:55.560 --> 00:03:01.310
And if we carry out the
calculations here, we'll get n
00:03:01.310 --> 00:03:03.730
minus 1 over n squared.
00:03:10.440 --> 00:03:13.080
Now, let's turn to yn.
00:03:13.080 --> 00:03:17.530
The expected value of yn is
equal to probability of being
00:03:17.530 --> 00:03:25.270
at 0, 0 plus the probability
of being at n and
00:03:25.270 --> 00:03:27.000
times the value n.
00:03:27.000 --> 00:03:30.060
And this gives us 1.
00:03:30.060 --> 00:03:33.274
The variance of yn.
00:03:33.274 --> 00:03:39.840
We do the same thing as before,
we have 1 minus 1 over
00:03:39.840 --> 00:03:44.950
n probability of being at 0,
multiplied 0 minus 1 squared,
00:03:44.950 --> 00:03:49.100
where 1 is the expected
value of y.
00:03:49.100 --> 00:03:54.150
And with probability 1 over n,
out there, equal to n, and
00:03:54.150 --> 00:03:58.620
this is multiplied by
n minus 1 squared.
00:03:58.620 --> 00:04:02.560
And this gives us n minus 1
00:04:02.560 --> 00:04:07.070
Already, we can see that while
the expected value for x was 1
00:04:07.070 --> 00:04:11.490
over n, the expected value for
y is sitting right at 1.
00:04:11.490 --> 00:04:15.400
It does not decrease
as it increases.
00:04:15.400 --> 00:04:20.310
And also, while the variance
for x is n minus 1 over n
00:04:20.310 --> 00:04:23.860
squared, the variance for
y is much bigger.
00:04:23.860 --> 00:04:27.930
It is actually increasing to
infinity as n goes infinity.
00:04:27.930 --> 00:04:30.180
So these intuitions will be
helpful for the remainder of
00:04:30.180 --> 00:04:31.370
the problem.
00:04:31.370 --> 00:04:34.700
In part B, we're asked to use
Chebyshev's Inequality and see
00:04:34.700 --> 00:04:38.660
whether xn or yn converges to
any number in probability.
00:04:38.660 --> 00:04:41.430
Let's first recall what the
inequality is about.
00:04:41.430 --> 00:04:46.020
It says that if we have random
variable x, in our case, xn,
00:04:46.020 --> 00:04:51.500
then the probability of xn minus
the expected value of
00:04:51.500 --> 00:04:57.040
xn, in our case, 1 over n,
that this deviation, the
00:04:57.040 --> 00:05:00.380
absolute value of this
difference is greater than
00:05:00.380 --> 00:05:08.290
epsilon is bounded above by the
variance of xn divided by
00:05:08.290 --> 00:05:11.290
the value of epsilon squared.
00:05:11.290 --> 00:05:16.100
Well, in our case, we know the
variance is n minus 1 over n
00:05:16.100 --> 00:05:22.300
squared, hence this whole term
is this term divided by
00:05:22.300 --> 00:05:24.530
epsilon squared.
00:05:24.530 --> 00:05:27.700
Now, we know that as n gets
really big, the probability of
00:05:27.700 --> 00:05:30.100
xn being at 0 is very big.
00:05:30.100 --> 00:05:32.270
It's 1 minus 1 over n.
00:05:32.270 --> 00:05:35.750
So a safe bet to guess is that
if xn work to converge
00:05:35.750 --> 00:05:38.590
anywhere on the real line,
it might just converge
00:05:38.590 --> 00:05:39.940
to the point 0.
00:05:39.940 --> 00:05:41.940
And let's see if that is true.
00:05:41.940 --> 00:05:45.860
Now, to show that xn converges
to 0 in probability, formally
00:05:45.860 --> 00:05:50.800
we need to show that for every
fixed epsilon greater than 0,
00:05:50.800 --> 00:05:58.600
the probability that xn minus 0
greater than epsilon has to
00:05:58.600 --> 00:06:02.690
be 0, and the limit has
n going to infinity.
00:06:02.690 --> 00:06:05.660
And hopefully, the inequalities
above will help
00:06:05.660 --> 00:06:07.480
us achieve this goal.
00:06:07.480 --> 00:06:09.350
And let's see how
that is done.
00:06:09.350 --> 00:06:12.860
I would like to have an
estimate, in fact, an upper
00:06:12.860 --> 00:06:17.850
bound of the probability xn
absolute value greater or
00:06:17.850 --> 00:06:20.040
equal to epsilon.
00:06:20.040 --> 00:06:21.850
And now, we're going to do
some massaging to this
00:06:21.850 --> 00:06:25.730
equation so that it looks like
what we know before, which is
00:06:25.730 --> 00:06:27.990
right here.
00:06:27.990 --> 00:06:33.420
Now, we see that this equation
is in fact, less than
00:06:33.420 --> 00:06:40.190
probability xn minus 1 over n
greater or equal to epsilon
00:06:40.190 --> 00:06:42.000
plus 1 over n.
00:06:42.000 --> 00:06:45.180
Now, I will justify this
inequality in one second.
00:06:45.180 --> 00:06:48.490
But suppose that you believe me
for this inequality, we can
00:06:48.490 --> 00:06:53.020
simply plug-in the value right
here, namely substituting
00:06:53.020 --> 00:06:57.560
epsilon plus 1 over n, in the
place of epsilon right here
00:06:57.560 --> 00:07:00.880
and use the Chebyshev Inequality
we did earlier to
00:07:00.880 --> 00:07:05.060
arrive at the following
inequality, which is n minus 1
00:07:05.060 --> 00:07:09.840
over n squared times, instead
of epsilon, now we have
00:07:09.840 --> 00:07:12.955
epsilon plus 1 over n squared.
00:07:16.710 --> 00:07:20.280
Now, if we take n to
infinity in this
00:07:20.280 --> 00:07:22.410
equation, see what happens.
00:07:22.410 --> 00:07:27.230
Well, this term here converges
to 0 because n squared is much
00:07:27.230 --> 00:07:29.490
bigger than n minus 1.
00:07:29.490 --> 00:07:32.180
And this term here converges
to number 1
00:07:32.180 --> 00:07:34.650
over epsilon squared.
00:07:34.650 --> 00:07:38.420
So it becomes 0 times 1 over
epsilon squared, hence the
00:07:38.420 --> 00:07:40.830
whole term converges to 0.
00:07:40.830 --> 00:07:46.640
And this proves that indeed, the
limit of the term here as
00:07:46.640 --> 00:07:52.220
n going to infinity is equal
to 0, and that implies xn
00:07:52.220 --> 00:07:54.410
converges to 0 in probability.
00:07:58.810 --> 00:08:01.050
Now, there is the one thing
I did not justify in the
00:08:01.050 --> 00:08:06.650
process, which is why is
probability of absolute value
00:08:06.650 --> 00:08:12.460
xn greater than epsilon less
then the term right here?
00:08:12.460 --> 00:08:14.480
So let's take a look.
00:08:14.480 --> 00:08:18.080
Well, the easiest way to see
this is to see what ranges of
00:08:18.080 --> 00:08:21.010
xn are we talking about
in each case.
00:08:21.010 --> 00:08:26.220
Well, in the first case, we're
looking at interval around 0
00:08:26.220 --> 00:08:30.850
plus minus epsilon and xn
can lie anywhere here.
00:08:33.539 --> 00:08:36.740
While in the second case, right
here, we can see that
00:08:36.740 --> 00:08:41.440
the set of range values for xn
is precisely this interval
00:08:41.440 --> 00:08:45.280
here, which was the same as
before, but now, we actually
00:08:45.280 --> 00:08:49.810
have less on this side, where
the starting point and the
00:08:49.810 --> 00:08:54.720
interval on the right is
epsilon plus 2 over n.
00:08:54.720 --> 00:08:58.890
And therefore, the right hand
style captures strictly less
00:08:58.890 --> 00:09:02.390
values of xn than the left
hand side, hence the
00:09:02.390 --> 00:09:04.820
inequality is true.
00:09:04.820 --> 00:09:07.490
Now, we wonder if we can use
the same trick, Chebyshev
00:09:07.490 --> 00:09:10.870
Inequality, to derive the
result for yn as well.
00:09:10.870 --> 00:09:12.390
Let's take a look.
00:09:12.390 --> 00:09:17.880
The probability of yn minus it's
mean, 1, greater or equal
00:09:17.880 --> 00:09:19.490
to epsilon.
00:09:19.490 --> 00:09:25.700
From the Chebyshev Inequality,
we have variance of yn divided
00:09:25.700 --> 00:09:28.420
by epsilon squared.
00:09:28.420 --> 00:09:29.410
Now, there is a problem.
00:09:29.410 --> 00:09:31.420
The variance of yn
is very big.
00:09:31.420 --> 00:09:34.510
In fact, it is equal
to n minus 1.
00:09:34.510 --> 00:09:37.870
And we calculated in part A,
divided by epsilon squared.
00:09:37.870 --> 00:09:41.940
And this quantity here diverges
as n going to
00:09:41.940 --> 00:09:45.360
infinity to infinity itself.
00:09:45.360 --> 00:09:48.810
So in this case, the Chebyshev
Inequality does not tell us
00:09:48.810 --> 00:09:54.220
much information of whether
yn converges or not.
00:09:54.220 --> 00:09:58.420
Now, going to part C, the
question is although we don't
00:09:58.420 --> 00:10:02.030
know anything about yn from just
the Chebyshev Inequality,
00:10:02.030 --> 00:10:04.660
does yn converge to
anything at all?
00:10:04.660 --> 00:10:07.520
Well, it turns out it does.
00:10:07.520 --> 00:10:09.270
In fact, we don't have to
go through anything more
00:10:09.270 --> 00:10:12.830
complicated than distribution
yn itself.
00:10:12.830 --> 00:10:17.600
So from the distribution yn, we
know that absolute value of
00:10:17.600 --> 00:10:22.980
yn greater or equal to epsilon
is equal to 1 over n whenever
00:10:22.980 --> 00:10:25.135
epsilon is less than n.
00:10:25.135 --> 00:10:30.150
And this is true because we know
yn has a lot of mass at 0
00:10:30.150 --> 00:10:36.320
and a tiny bit a mass at value
1 over n at location n.
00:10:36.320 --> 00:10:39.670
So if we draw the cutoff here
at epsilon, then the
00:10:39.670 --> 00:10:44.420
probability of yn landing to the
right of epsilon is simply
00:10:44.420 --> 00:10:46.250
equal to 1 over n.
00:10:46.250 --> 00:10:49.800
And this tells us, if we take
the limit of n going to
00:10:49.800 --> 00:10:54.640
infinity and measure the
probability that yn--
00:10:54.640 --> 00:10:55.870
just to write it clearly--
00:10:55.870 --> 00:11:00.800
deviates from 0 by more than
epsilon, this is equal to the
00:11:00.800 --> 00:11:04.600
limit as n going to infinity
of 1 over n.
00:11:04.600 --> 00:11:07.560
And that is equal to 0.
00:11:07.560 --> 00:11:12.270
From this calculation, we know
that yn does converge to 0 in
00:11:12.270 --> 00:11:16.260
probability as n going
to infinity.
00:11:20.350 --> 00:11:24.540
For part D, we'd like to know
whether the convergence in
00:11:24.540 --> 00:11:28.970
probability implies the
convergence in expectation.
00:11:28.970 --> 00:11:32.170
That is, if we have a sequence
of random variables, let's
00:11:32.170 --> 00:11:39.980
call it zn, that converges to
number c in probability as n
00:11:39.980 --> 00:11:45.960
going to infinity, does it also
imply that the limit as n
00:11:45.960 --> 00:11:52.930
going to infinity of the
expected value of zn also
00:11:52.930 --> 00:11:55.180
converges to c.
00:11:55.180 --> 00:11:56.250
Is that true?
00:11:56.250 --> 00:11:59.520
Well, intuitively it is true,
because in the limit, zn
00:11:59.520 --> 00:12:02.890
almost looks like it
concentrates on c solely,
00:12:02.890 --> 00:12:06.530
hence we might expect that the
expected value is also going
00:12:06.530 --> 00:12:08.040
to c itself.
00:12:08.040 --> 00:12:10.430
Well, unfortunately, that
is not quite true.
00:12:10.430 --> 00:12:14.580
In fact, we have the proof right
here by looking at yn.
00:12:14.580 --> 00:12:19.140
For yn, we know that the
expected value of yn is equal
00:12:19.140 --> 00:12:21.210
to 1 for all n.
00:12:21.210 --> 00:12:23.840
It does not matter
how big n gets.
00:12:23.840 --> 00:12:28.900
But we also know front part C
that yn does converge to 0 in
00:12:28.900 --> 00:12:31.200
probability.
00:12:31.200 --> 00:12:35.180
And this means somehow, yn can
get very close to 0, yet it's
00:12:35.180 --> 00:12:39.500
expected value still
stays one away.
00:12:39.500 --> 00:12:42.220
And the reason again, we go
back to the way yn was
00:12:42.220 --> 00:12:43.710
constructed.
00:12:43.710 --> 00:12:47.810
Now, as n goes to infinity, the
probability of yn being at
00:12:47.810 --> 00:12:52.960
0, 1 minus 1 over
n, approaches 1.
00:12:52.960 --> 00:12:58.740
So it's very likely that yn is
having a value 0, but whenever
00:12:58.740 --> 00:13:02.600
on the off chance that yn takes
a value other than 0,
00:13:02.600 --> 00:13:03.640
it's a huge number.
00:13:03.640 --> 00:13:08.560
It is n, even though it has a
small probability of 1 over n.
00:13:08.560 --> 00:13:11.110
Adding these two factors
together, it tells us the
00:13:11.110 --> 00:13:15.320
expected value of yn
always stays at 1.
00:13:15.320 --> 00:13:18.460
And however, in probability,
it's very likely
00:13:18.460 --> 00:13:21.100
that y is around 0.
00:13:21.100 --> 00:13:24.070
So this example tells us
converges in probability is
00:13:24.070 --> 00:13:25.020
not that strong.
00:13:25.020 --> 00:13:28.010
That tells us something about
the random variables but it
00:13:28.010 --> 00:13:31.420
does not tell us whether the
mean value of the random
00:13:31.420 --> 00:13:34.150
variables converge to
the same number.