WEBVTT

00:00:00.290 --> 00:00:03.060
The purpose of this segment is
to give you a little bit of

00:00:03.060 --> 00:00:04.510
the bigger picture.

00:00:04.510 --> 00:00:07.750
We did discuss some
inequalities, we did discuss

00:00:07.750 --> 00:00:09.820
convergence of the
sample mean--

00:00:09.820 --> 00:00:13.190
that's the weak law of large
numbers-- and we did discuss a

00:00:13.190 --> 00:00:15.330
particular notion of convergence
of random

00:00:15.330 --> 00:00:17.860
variables, convergence
in probability.

00:00:17.860 --> 00:00:21.510
How far can we take
those topics?

00:00:21.510 --> 00:00:23.790
Let's start with the issue
of inequalities.

00:00:23.790 --> 00:00:27.010
Here, one would like to obtain
bounds and approximations on

00:00:27.010 --> 00:00:30.620
tail probabilities that are
better than the Markov and

00:00:30.620 --> 00:00:33.020
Cherbyshev inequalities
that we have seen.

00:00:33.020 --> 00:00:34.740
This is indeed possible.

00:00:34.740 --> 00:00:38.300
For example, there is a
so-called Chernoff bound that

00:00:38.300 --> 00:00:40.740
takes the following form.

00:00:40.740 --> 00:00:44.630
The Chernoff bound tells us that
the probability that the

00:00:44.630 --> 00:00:51.030
sample mean is away from the
true mean by at least a, where

00:00:51.030 --> 00:00:55.710
a is a positive number, this
probability is bounded above

00:00:55.710 --> 00:01:00.290
by a function that falls
exponentially with n and where

00:01:00.290 --> 00:01:03.700
the exponent depends on the
particular number, a, that we

00:01:03.700 --> 00:01:04.860
are considering.

00:01:04.860 --> 00:01:07.110
But in any case, this
term in the exponent

00:01:07.110 --> 00:01:09.510
is a positive quantity.

00:01:09.510 --> 00:01:14.100
Notice that this is much better,
much stronger than

00:01:14.100 --> 00:01:17.900
what we obtained from the
Cherbyshev inequality because

00:01:17.900 --> 00:01:21.560
in the Cherbyshev inequality,
we only obtain an inequality

00:01:21.560 --> 00:01:24.470
for this probability that
falls off as the

00:01:24.470 --> 00:01:26.950
rate of 1 over n.

00:01:26.950 --> 00:01:29.830
So this falls much faster, and
so it tells us that this

00:01:29.830 --> 00:01:33.550
probability is indeed much
smaller than what the

00:01:33.550 --> 00:01:36.100
Cherbyshev inequality
might predict.

00:01:36.100 --> 00:01:38.860
However, this inequality
requires some additional

00:01:38.860 --> 00:01:42.259
assumptions on the random
variables involved.

00:01:42.259 --> 00:01:45.880
Another type of approximation
on this tail probability can

00:01:45.880 --> 00:01:49.120
be obtained through the central
limit theorem, which

00:01:49.120 --> 00:01:51.060
will actually be the next topic

00:01:51.060 --> 00:01:52.960
that we will be studying.

00:01:52.960 --> 00:01:57.660
Very loosely speaking, the
central limit theorem tells us

00:01:57.660 --> 00:02:03.530
that the random variable M sub
n, which is the sample mean,

00:02:03.530 --> 00:02:09.910
behaves as if it were a normal
random variable with the mean

00:02:09.910 --> 00:02:13.380
and the variance that
it should have.

00:02:13.380 --> 00:02:16.650
We know that this is the mean
and the variance of the sample

00:02:16.650 --> 00:02:19.840
mean, but the central limit
theorem tells us that in

00:02:19.840 --> 00:02:23.800
addition to that, we can also
pretend that the sample mean

00:02:23.800 --> 00:02:27.870
is normal and carry out
approximations as if this were

00:02:27.870 --> 00:02:30.020
a normal random variable.

00:02:30.020 --> 00:02:32.640
Now, this statement that
I'm making here is

00:02:32.640 --> 00:02:34.660
only a loose statement.

00:02:34.660 --> 00:02:38.550
It is not mathematically
completely accurate.

00:02:38.550 --> 00:02:41.579
We will see later a more
accurate statement of the

00:02:41.579 --> 00:02:43.640
central limit theorem.

00:02:43.640 --> 00:02:47.090
In a different direction, we can
talk about different types

00:02:47.090 --> 00:02:48.750
of convergence.

00:02:48.750 --> 00:02:52.550
We did define convergence in
probability, but that's not

00:02:52.550 --> 00:02:54.490
the only notion of convergence
that's

00:02:54.490 --> 00:02:56.350
relevant to random variables.

00:02:56.350 --> 00:02:59.160
There's an alternative notion,
which is convergence with

00:02:59.160 --> 00:03:00.770
probability one.

00:03:00.770 --> 00:03:03.240
Here is what it means.

00:03:03.240 --> 00:03:06.240
We have a single probabilistic
experiment.

00:03:06.240 --> 00:03:09.290
And within that the experiment,
we have a sequence

00:03:09.290 --> 00:03:15.220
of random variables and another
random variable, and

00:03:15.220 --> 00:03:18.160
we want to talk about this
random variable converging to

00:03:18.160 --> 00:03:20.360
that random variable.

00:03:20.360 --> 00:03:22.280
What do we mean by that?

00:03:22.280 --> 00:03:27.040
We consider a typical outcome
of the experiment, that is,

00:03:27.040 --> 00:03:29.110
some omega.

00:03:29.110 --> 00:03:33.730
Look at the values of the random
variable Yn under that

00:03:33.730 --> 00:03:38.530
particular omega, and look at
that sequence of values, the

00:03:38.530 --> 00:03:40.890
values of the different random
variables under that

00:03:40.890 --> 00:03:42.740
particular outcome.

00:03:42.740 --> 00:03:47.010
Under that particular outcome, Y
also has a certain numerical

00:03:47.010 --> 00:03:53.070
value, and we're interested in
whether this convergence takes

00:03:53.070 --> 00:03:56.760
place as n goes to infinity.

00:03:56.760 --> 00:03:59.520
Now for some outcomes, omega,
this will happen.

00:03:59.520 --> 00:04:02.080
For some, it will not happen.

00:04:02.080 --> 00:04:04.730
We will say that we have
convergence with probability

00:04:04.730 --> 00:04:13.870
one if this event has
probability equal to 1.

00:04:13.870 --> 00:04:18.060
That is, there is probability
one, that is, essential

00:04:18.060 --> 00:04:21.760
certainty, that when an outcome
of the experiment is

00:04:21.760 --> 00:04:25.380
obtained, the resulting sequence
of values of the

00:04:25.380 --> 00:04:28.730
random variables Yn will
converge to the value of the

00:04:28.730 --> 00:04:30.980
random variable Y.

00:04:30.980 --> 00:04:35.010
Now, this definition is easy to
write down, but to actually

00:04:35.010 --> 00:04:37.670
understand what it really
means and the ways it is

00:04:37.670 --> 00:04:41.770
different from convergence in
probability is not so easy.

00:04:41.770 --> 00:04:45.050
It does take some conceptual
effort, and we will not

00:04:45.050 --> 00:04:47.880
discuss it any further
at this point.

00:04:47.880 --> 00:04:50.670
Let me just say that this
is a stronger notion of

00:04:50.670 --> 00:04:51.659
convergence.

00:04:51.659 --> 00:04:54.710
If you have convergence with
probability one, you also gets

00:04:54.710 --> 00:04:57.080
convergence in probability.

00:04:57.080 --> 00:05:02.880
And it turns out that the law
of large numbers also holds

00:05:02.880 --> 00:05:06.280
under this stronger notion
of convergence.

00:05:06.280 --> 00:05:10.420
That is, we have that the sample
mean converges to the

00:05:10.420 --> 00:05:14.170
true mean with probability
one.

00:05:14.170 --> 00:05:17.970
This is the so-called strong
law of large numbers, and

00:05:17.970 --> 00:05:21.050
because this is a stronger
notion of convergence, a more

00:05:21.050 --> 00:05:26.320
demanding one, that's why this
is called the strong law.

00:05:26.320 --> 00:05:30.490
Incidentally, at this point, you
might be quite uncertain

00:05:30.490 --> 00:05:34.470
and confused as to what is
really the difference between

00:05:34.470 --> 00:05:37.040
these two notions
of convergence.

00:05:37.040 --> 00:05:40.470
The definitions do look
different, but what is the

00:05:40.470 --> 00:05:41.900
real difference?

00:05:41.900 --> 00:05:44.770
This is quite subtle,
and it does take

00:05:44.770 --> 00:05:46.270
quite a bit of thinking.

00:05:46.270 --> 00:05:50.090
It's not supposed to be
something that is obvious.

00:05:50.090 --> 00:05:53.190
So the purpose of this
discussion is only to point

00:05:53.190 --> 00:05:57.480
out these further directions
but without, at this point,

00:05:57.480 --> 00:05:59.890
going into it in any depth.

00:05:59.890 --> 00:06:03.730
Finally, there is another notion
of convergence in which

00:06:03.730 --> 00:06:06.460
we're looking at the
distributions of the random

00:06:06.460 --> 00:06:08.070
variables involved.

00:06:08.070 --> 00:06:10.790
So we may have a sequence
of random variables.

00:06:10.790 --> 00:06:13.770
Each one of them has a certain
distribution described by a

00:06:13.770 --> 00:06:17.680
CDF, and we can ask the
question, does this sequence

00:06:17.680 --> 00:06:21.510
of CDFs converge to
a limiting CDF?

00:06:21.510 --> 00:06:24.800
If that happens, then we say
that we have convergence in

00:06:24.800 --> 00:06:28.450
distribution, and this is
more or less the type of

00:06:28.450 --> 00:06:32.220
convergence that shows up when
we deal with the central limit

00:06:32.220 --> 00:06:35.040
theorem because this is really
a statement about

00:06:35.040 --> 00:06:37.560
distributions, that the
distribution of the sample

00:06:37.560 --> 00:06:41.570
mean in some sense starts to
approach the distribution of a

00:06:41.570 --> 00:06:42.820
normal random variable.