WEBVTT

00:00:00.810 --> 00:00:03.520
We will now go through two
examples of convergence in

00:00:03.520 --> 00:00:04.590
probability.

00:00:04.590 --> 00:00:07.370
Our first example is
quite trivial.

00:00:07.370 --> 00:00:11.090
We're dealing with a sequence
of random variables Yn that

00:00:11.090 --> 00:00:12.220
are discrete.

00:00:12.220 --> 00:00:15.680
Most of the probability
is concentrated at 0.

00:00:15.680 --> 00:00:19.690
But there is also a small
probability of a large value.

00:00:19.690 --> 00:00:22.040
Because the bulk of the
probability mass is

00:00:22.040 --> 00:00:25.540
concentrated at 0, it is a good
guess that this sequence

00:00:25.540 --> 00:00:27.070
converges to 0.

00:00:27.070 --> 00:00:29.380
Do we have, indeed,
convergence in

00:00:29.380 --> 00:00:30.760
probability to 0?

00:00:30.760 --> 00:00:32.710
We need to check
the definition.

00:00:32.710 --> 00:00:36.930
So we fix some epsilon, which
is a positive number.

00:00:36.930 --> 00:00:40.550
And we look at the probability
of the event that our random

00:00:40.550 --> 00:00:46.360
variable is epsilon or more away
than what we think is the

00:00:46.360 --> 00:00:48.460
limit of that sequence.

00:00:48.460 --> 00:00:50.370
We look at that probability.

00:00:50.370 --> 00:00:54.590
And in this example, it is equal
to 1 over n, which goes

00:00:54.590 --> 00:00:57.820
to 0 as n goes to infinity.

00:00:57.820 --> 00:01:01.990
And this verifies that, indeed,
in this example, Yn

00:01:01.990 --> 00:01:07.065
converges to 0, as n goes to
infinity in probability.

00:01:10.330 --> 00:01:12.900
Now, we make the following
observation.

00:01:12.900 --> 00:01:16.220
If we are to calculate the
expected value of this random

00:01:16.220 --> 00:01:19.390
variable, what we get
is the following.

00:01:19.390 --> 00:01:23.060
We get a value of 0 with this
probability, no contribution

00:01:23.060 --> 00:01:24.280
to the expectation.

00:01:24.280 --> 00:01:27.970
But we also get a value
of n squared with

00:01:27.970 --> 00:01:30.490
probability 1 over n.

00:01:30.490 --> 00:01:34.630
And so the expected value is
equal to n, which, actually,

00:01:34.630 --> 00:01:38.440
goes to infinity, as
n goes to infinity.

00:01:38.440 --> 00:01:43.120
So we have a situation where
the sequence of the random

00:01:43.120 --> 00:01:45.220
variables converges to 0.

00:01:45.220 --> 00:01:48.090
But the expectation does
not converge to 0.

00:01:48.090 --> 00:01:50.120
In fact, it goes to infinity.

00:01:50.120 --> 00:01:52.920
And this example serves
to make the point that

00:01:52.920 --> 00:01:56.509
convergence in probability does
not imply convergence of

00:01:56.509 --> 00:01:57.890
expectations.

00:01:57.890 --> 00:02:01.940
The reason is that convergence
in probability has to do with

00:02:01.940 --> 00:02:04.360
the bulk of the distribution.

00:02:04.360 --> 00:02:08.259
It only cares that the tail of
the distribution has small

00:02:08.259 --> 00:02:09.340
probability.

00:02:09.340 --> 00:02:11.980
On the other hand, the
expectation is highly

00:02:11.980 --> 00:02:15.410
sensitive to the tail
of the distribution.

00:02:15.410 --> 00:02:19.150
It might be that the tail only
has a small probability.

00:02:19.150 --> 00:02:21.900
But if that probability is
assigned to a very large

00:02:21.900 --> 00:02:26.140
value, then the expectation will
be strongly affected and

00:02:26.140 --> 00:02:29.020
can be quite different
from the limit

00:02:29.020 --> 00:02:32.270
of the random variable.

00:02:32.270 --> 00:02:35.570
Our second example is going to
be less trivial and more

00:02:35.570 --> 00:02:36.860
interesting.

00:02:36.860 --> 00:02:40.290
Consider random variables
that are independent and

00:02:40.290 --> 00:02:44.050
identically distributed and
whose common distribution is

00:02:44.050 --> 00:02:47.350
uniform on the unit interval,
so that the

00:02:47.350 --> 00:02:50.160
PDF takes this form.

00:02:50.160 --> 00:02:55.150
Are these random variables
convergent to something?

00:02:55.150 --> 00:02:56.480
The answer is no.

00:02:56.480 --> 00:03:00.980
And the reason is that as i
increases, the distribution

00:03:00.980 --> 00:03:02.020
does not change.

00:03:02.020 --> 00:03:04.380
And it does not to
get concentrated

00:03:04.380 --> 00:03:06.010
around a certain number.

00:03:06.010 --> 00:03:09.130
The distribution remains spread
out over the entire

00:03:09.130 --> 00:03:10.960
unit interval.

00:03:10.960 --> 00:03:14.330
But let us look now at some
related random variables.

00:03:14.330 --> 00:03:19.940
Let us define Yn to be the
minimum of the first n of the

00:03:19.940 --> 00:03:21.530
X's that we get.

00:03:21.530 --> 00:03:25.540
So if n is equal to 4, and we
obtain these four values, Yn

00:03:25.540 --> 00:03:28.100
would be equal to this value.

00:03:28.100 --> 00:03:33.560
Notice that if we draw more
values, then the new values

00:03:33.560 --> 00:03:36.810
might be above the minimum, in
which case the minimum does

00:03:36.810 --> 00:03:37.890
not change.

00:03:37.890 --> 00:03:41.610
But we might also get a value
that's below the minimum, in

00:03:41.610 --> 00:03:44.100
which case the minimum
moves down.

00:03:44.100 --> 00:03:48.400
The only thing that can happen
is that the minimum goes down.

00:03:48.400 --> 00:03:49.960
It cannot go up.

00:03:49.960 --> 00:03:52.680
And this gives us
this inequality.

00:03:52.680 --> 00:03:56.420
So the random variables
Yn tends to go down.

00:03:56.420 --> 00:03:58.720
How far down?

00:03:58.720 --> 00:04:03.640
If n is very large, we expect
that we will obtain some X's

00:04:03.640 --> 00:04:07.570
whose value happens to be very
close to 0, which means that

00:04:07.570 --> 00:04:12.580
Yn will go down to values that
are very close to 0.

00:04:12.580 --> 00:04:16.010
And this leads us to conjecture
that, perhaps, Yn

00:04:16.010 --> 00:04:18.390
does converge to 0.

00:04:18.390 --> 00:04:21.209
This is always the first step
when dealing with convergence

00:04:21.209 --> 00:04:22.029
in probability.

00:04:22.029 --> 00:04:25.940
The first step is to make an
educated guess about what the

00:04:25.940 --> 00:04:27.580
limit might be.

00:04:27.580 --> 00:04:30.610
And then we want to verify
that this is, indeed, the

00:04:30.610 --> 00:04:32.150
correct limit.

00:04:32.150 --> 00:04:37.380
To verify that, what we do is we
fix some positive epsilon.

00:04:37.380 --> 00:04:41.640
And we look for the probability
that the distance

00:04:41.640 --> 00:04:46.090
of the random variable Yn from
the conjectured limit has a

00:04:46.090 --> 00:04:49.610
magnitude that's larger than
or equal to epsilon.

00:04:49.610 --> 00:04:52.630
And what we need to show is that
this quantity converges

00:04:52.630 --> 00:04:58.320
to 0 as n goes to infinity,
no matter what epsilon is.

00:04:58.320 --> 00:05:01.940
Now, because Yn is a
non-negative random variable,

00:05:01.940 --> 00:05:05.270
this is the same as the
probability that Yn is larger

00:05:05.270 --> 00:05:07.330
than or equal to epsilon.

00:05:07.330 --> 00:05:10.210
Now, let us distinguish
between two cases.

00:05:10.210 --> 00:05:13.870
If epsilon is bigger than
1, we're asking for the

00:05:13.870 --> 00:05:18.190
probability that Yn is larger
than or equal to a certain

00:05:18.190 --> 00:05:20.460
number epsilon that's
out there.

00:05:20.460 --> 00:05:22.610
But this probability is 0.

00:05:22.610 --> 00:05:25.690
There's no way that the minimum
of these uniforms will

00:05:25.690 --> 00:05:29.000
take a value that's larger
than some epsilon that's

00:05:29.000 --> 00:05:30.660
larger than 1.

00:05:30.660 --> 00:05:34.150
So in that case, this quantity
is equal to zero.

00:05:34.150 --> 00:05:35.380
And we are done.

00:05:35.380 --> 00:05:38.530
But we need to check that this
quantity becomes small no

00:05:38.530 --> 00:05:40.290
matter what epsilon is.

00:05:40.290 --> 00:05:44.360
So now, let us consider taking
a small epsilon that is some

00:05:44.360 --> 00:05:47.260
number that's less than
or equal to 1.

00:05:47.260 --> 00:05:51.180
For that case, let us continue
with the calculation.

00:05:51.180 --> 00:05:56.300
The minimum is going to be at
least epsilon, if, and only

00:05:56.300 --> 00:05:59.490
if, all of the random variables

00:05:59.490 --> 00:06:01.160
are at least epsilon.

00:06:04.210 --> 00:06:07.950
So this is an equivalent way
of writing this particular

00:06:07.950 --> 00:06:09.300
event here.

00:06:09.300 --> 00:06:12.470
Now, because of independence,
this is the product of the

00:06:12.470 --> 00:06:17.080
probabilities that each one of
the random variables is larger

00:06:17.080 --> 00:06:20.370
than or equal to epsilon.

00:06:20.370 --> 00:06:23.470
The probability that X1 is
larger than or equal to

00:06:23.470 --> 00:06:26.150
epsilon can be found
as follows.

00:06:26.150 --> 00:06:29.420
If we have here epsilon, the
probability of being larger

00:06:29.420 --> 00:06:31.540
than or equal to epsilon
is the probability

00:06:31.540 --> 00:06:32.990
of this event here.

00:06:32.990 --> 00:06:35.020
So it's the area of
this rectangle.

00:06:35.020 --> 00:06:38.060
The base of that rectangle
is 1 minus epsilon.

00:06:38.060 --> 00:06:42.409
And so we obtain 1 minus epsilon
for this first term.

00:06:42.409 --> 00:06:45.180
But because the Xi's are
identically distributed, all

00:06:45.180 --> 00:06:48.280
the other terms that we multiply
are also the same.

00:06:48.280 --> 00:06:52.940
And so the answer is this
expression here.

00:06:52.940 --> 00:06:55.890
Now, epsilon is a
positive number.

00:06:55.890 --> 00:06:58.920
So 1 minus epsilon is strictly
less than 1.

00:06:58.920 --> 00:07:03.010
And when we take higher powers
of a number that's less than

00:07:03.010 --> 00:07:07.500
1, we obtain something
that converges to 0

00:07:07.500 --> 00:07:09.030
as n goes to infinity.

00:07:09.030 --> 00:07:11.690
And that's what we
needed to verify.

00:07:11.690 --> 00:07:16.090
Since this is the case for any
epsilon, we conclude that the

00:07:16.090 --> 00:07:22.370
random variables Yn converge to
zero in the sense that we

00:07:22.370 --> 00:07:24.135
have defined, in probability.

00:07:27.270 --> 00:07:31.170
Generalizing from this example,
when we want to show

00:07:31.170 --> 00:07:34.850
convergence in probability, the
first step is to make a

00:07:34.850 --> 00:07:38.540
guess as to what is
the value that the

00:07:38.540 --> 00:07:40.530
sequence converges to.

00:07:40.530 --> 00:07:44.260
In this example, that value
was equal to 0.

00:07:44.260 --> 00:07:48.240
Once we have made that
conjecture, then we write down

00:07:48.240 --> 00:07:51.540
an expression for the
probability of being epsilon

00:07:51.540 --> 00:07:54.690
away from the conjectured
limit.

00:07:54.690 --> 00:07:58.280
And then we calculate that
probability either exactly, as

00:07:58.280 --> 00:07:59.475
in this example.

00:07:59.475 --> 00:08:02.500
Or we try to bound it somehow
and still show

00:08:02.500 --> 00:08:03.750
that it goes to 0.