WEBVTT

00:00:00.520 --> 00:00:02.960
We have seen that
under some conditions,

00:00:02.960 --> 00:00:08.270
the binomial PMF is well
approximated by a Poisson PMF.

00:00:08.270 --> 00:00:10.890
But we have also seen
the central limit theorem

00:00:10.890 --> 00:00:15.240
that tells us that the binomial
PMF can be approximated

00:00:15.240 --> 00:00:18.120
using a normal random variable.

00:00:18.120 --> 00:00:20.990
Can we reconcile
these two facts?

00:00:20.990 --> 00:00:24.110
Let's look into the situation
in some more detail.

00:00:24.110 --> 00:00:28.950
Consider a Poisson process
that has rate equal to 1.

00:00:28.950 --> 00:00:31.020
And consider that
Poisson process

00:00:31.020 --> 00:00:34.280
running over the
unit time interval.

00:00:34.280 --> 00:00:37.340
We take the unit
interval, and we split it

00:00:37.340 --> 00:00:40.720
into many small
sub-intervals, where

00:00:40.720 --> 00:00:46.120
each sub-interval
has a length of 1/n.

00:00:46.120 --> 00:00:50.320
And let Xi be the number of
arrivals that we get during

00:00:50.320 --> 00:00:52.600
the i'th interval.

00:00:52.600 --> 00:00:56.170
Xi is a Poisson random variable.

00:00:56.170 --> 00:00:59.010
And the mean of that
random variable,

00:00:59.010 --> 00:01:01.110
or the parameter of
that random variable,

00:01:01.110 --> 00:01:05.209
is just the duration of the time
interval, since the rate is 1.

00:01:05.209 --> 00:01:09.850
So it's a Poisson random
variable, with parameter 1/n.

00:01:09.850 --> 00:01:12.980
Now, let us look at the
total number of arrivals.

00:01:12.980 --> 00:01:17.350
The total number of arrivals
is the sum of how many arrivals

00:01:17.350 --> 00:01:21.130
we had during each one
of these intervals.

00:01:21.130 --> 00:01:24.070
And we know the
distribution of S. S

00:01:24.070 --> 00:01:30.430
is a Poisson random variable,
with parameter equal to 1.

00:01:30.430 --> 00:01:35.000
Now, here what we have is
a sum of random variables

00:01:35.000 --> 00:01:39.390
that are independent and
identically distributed.

00:01:39.390 --> 00:01:41.080
They are identically
distributed,

00:01:41.080 --> 00:01:43.800
because all of these intervals
have the same length.

00:01:43.800 --> 00:01:46.750
And they're independent,
because in the Poisson process,

00:01:46.750 --> 00:01:49.250
what happens in
different intervals are

00:01:49.250 --> 00:01:52.320
independent events.

00:01:52.320 --> 00:01:55.080
So we are in a
situation where we

00:01:55.080 --> 00:01:57.590
could apply the
central limit theorem.

00:01:57.590 --> 00:02:00.520
We have a sum of
many independent,

00:02:00.520 --> 00:02:02.980
identically distributed
random variables.

00:02:02.980 --> 00:02:06.850
And by letting n go to infinity,
the central limit theorem

00:02:06.850 --> 00:02:10.723
appears to tell us that
S is going to be normal.

00:02:13.800 --> 00:02:17.440
Now, how can we reconcile
these two facts?

00:02:17.440 --> 00:02:20.930
We know that the Poisson
distribution is not the same

00:02:20.930 --> 00:02:23.960
as a normal distribution.

00:02:23.960 --> 00:02:25.565
What is the catch?

00:02:25.565 --> 00:02:29.760
Well, the catch
is the following--

00:02:29.760 --> 00:02:33.640
the central limit theorem
applies to a situation where we

00:02:33.640 --> 00:02:37.010
fix a certain
probability distribution,

00:02:37.010 --> 00:02:39.530
the distribution of the Xi's.

00:02:39.530 --> 00:02:44.270
And it tells us that as we add
more and more of these Xi's,

00:02:44.270 --> 00:02:47.480
asymptotically, we obtain
a distribution that's

00:02:47.480 --> 00:02:50.630
well approximated by a normal.

00:02:50.630 --> 00:02:56.030
On the other hand, what we have
here is actually different.

00:02:56.030 --> 00:02:59.660
The Xi's do not have
a fixed distribution.

00:02:59.660 --> 00:03:06.260
But rather, the distribution
of Xi depends on n.

00:03:08.840 --> 00:03:13.030
That is, if we change
n so that we're

00:03:13.030 --> 00:03:15.500
adding more random
variables, we're

00:03:15.500 --> 00:03:17.490
adding more random
variables that are now

00:03:17.490 --> 00:03:20.200
coming from a
different distribution.

00:03:20.200 --> 00:03:23.260
And this is not a situation
to which the central limit

00:03:23.260 --> 00:03:24.880
theorem applies.

00:03:24.880 --> 00:03:28.980
And therefore, this conclusion
here is not justified.

00:03:28.980 --> 00:03:32.040
And so there's no
contradiction between the two

00:03:32.040 --> 00:03:34.790
types of approximations.

00:03:34.790 --> 00:03:37.460
To summarize, the
situation is as follows.

00:03:37.460 --> 00:03:40.500
Consider a binomial random
variable with some parameters

00:03:40.500 --> 00:03:41.800
n and p.

00:03:41.800 --> 00:03:44.200
Now, let p be fixed.

00:03:44.200 --> 00:03:46.410
And let n go to infinity.

00:03:46.410 --> 00:03:49.030
In that case, the
binomial random variable

00:03:49.030 --> 00:03:54.740
can be thought of as the sum of
n Bernoulli random variables.

00:03:54.740 --> 00:03:57.050
And those Bernoulli
random variables

00:03:57.050 --> 00:04:01.160
have a parameter
p, which is fixed.

00:04:01.160 --> 00:04:04.750
So we're dealing with the
sum of iid random variables

00:04:04.750 --> 00:04:06.050
from a fixed distribution.

00:04:06.050 --> 00:04:08.320
And this is the situation
where the central limit

00:04:08.320 --> 00:04:09.610
theorem applies.

00:04:09.610 --> 00:04:12.120
And we have a normal
approximation.

00:04:12.120 --> 00:04:18.750
On the other hand, if
we take the product

00:04:18.750 --> 00:04:22.250
n times p, which is the
expected value of this binomial,

00:04:22.250 --> 00:04:25.540
to stay constant,
but we let n go

00:04:25.540 --> 00:04:29.990
to infinity and at the same
time let p go to 0, then

00:04:29.990 --> 00:04:33.330
in this regime, in the
limit, this random variable

00:04:33.330 --> 00:04:37.370
will be well approximated by
a Poisson random variable.

00:04:37.370 --> 00:04:40.130
So we have two different
approximations.

00:04:40.130 --> 00:04:42.430
Both of them are
valid, but they're

00:04:42.430 --> 00:04:45.674
valid in different regimes.

00:04:45.674 --> 00:04:47.340
Now, although they're
different, there's

00:04:47.340 --> 00:04:52.770
actually an interesting case in
which the two will not really

00:04:52.770 --> 00:04:53.640
differ.

00:04:53.640 --> 00:04:55.400
And this is the following.

00:04:55.400 --> 00:04:59.090
Consider a Poisson random
variable with parameter n.

00:04:59.090 --> 00:05:03.550
And we're interested in the
limit as n goes to infinity.

00:05:03.550 --> 00:05:05.380
We can think of
this random variable

00:05:05.380 --> 00:05:08.000
as the number of arrivals
during an interval

00:05:08.000 --> 00:05:13.670
of length n in a Poisson process
with arrival rate equal to 1.

00:05:13.670 --> 00:05:17.540
Now, let's take this
interval and split it

00:05:17.540 --> 00:05:23.600
into n intervals, each of
which has a length of 1.

00:05:23.600 --> 00:05:28.470
And let us call Xi the number of
arrivals in the i'th interval.

00:05:28.470 --> 00:05:31.920
Our Poisson random variable
is going to be, of course,

00:05:31.920 --> 00:05:35.040
equal to the sum of
the number of arrivals

00:05:35.040 --> 00:05:37.900
during each one
of the intervals.

00:05:37.900 --> 00:05:40.430
Each one of these
random variables

00:05:40.430 --> 00:05:45.310
is Poisson with
parameter equal to 1.

00:05:45.310 --> 00:05:50.420
And these random variables
are actually iid.

00:05:50.420 --> 00:05:52.730
Now, what's happening
in this case

00:05:52.730 --> 00:05:56.250
is that even when we
increase n, because we're

00:05:56.250 --> 00:06:00.130
using constant length intervals,
the distribution of the Xi's

00:06:00.130 --> 00:06:01.400
doesn't change.

00:06:01.400 --> 00:06:04.470
So this is a situation
in which we're

00:06:04.470 --> 00:06:09.590
going to get approximately a
normal random variable as n

00:06:09.590 --> 00:06:12.650
goes to infinity.

00:06:12.650 --> 00:06:16.520
So what we see is that a
Poisson random variable, but

00:06:16.520 --> 00:06:19.010
with a very large
parameter, starts

00:06:19.010 --> 00:06:22.180
to approach the
normal distribution.

00:06:22.180 --> 00:06:24.010
And this in particular
will tell us

00:06:24.010 --> 00:06:27.630
that these two approximations
that we have, in some regime,

00:06:27.630 --> 00:06:30.810
they would start to agree.

00:06:30.810 --> 00:06:33.770
Now, all of this discussion
here has been asymptotic.

00:06:33.770 --> 00:06:37.940
We talk about p going to
0 or n going to infinity.

00:06:37.940 --> 00:06:41.920
But in any real situation, you
will be given actual numbers.

00:06:41.920 --> 00:06:44.909
And you cannot really tell,
is this number close to 0,

00:06:44.909 --> 00:06:46.900
or is it not?

00:06:46.900 --> 00:06:52.380
Here, we need some rules of
thumb or maybe some experience.

00:06:52.380 --> 00:06:55.300
Let's look at some examples.

00:06:55.300 --> 00:07:00.510
In this case, n times
p is equal to 1.

00:07:00.510 --> 00:07:03.200
So the number of
arrivals or the values

00:07:03.200 --> 00:07:07.380
of the binomial random variable
will take values 0, 1, 2,

00:07:07.380 --> 00:07:12.860
3, but with high probability,
not a lot more than that.

00:07:12.860 --> 00:07:15.930
So the binomial random
variable is really

00:07:15.930 --> 00:07:17.630
a discrete random variable.

00:07:17.630 --> 00:07:19.940
There's no way to
approximate it with a normal.

00:07:19.940 --> 00:07:22.770
On the other hand,
p is very small.

00:07:22.770 --> 00:07:25.630
So a Poisson
approximation would be

00:07:25.630 --> 00:07:29.300
very reasonable
in this situation.

00:07:29.300 --> 00:07:33.510
On the other hand, if
p is equal to 1/3, then

00:07:33.510 --> 00:07:36.690
definitely 1/3 is
not a small number.

00:07:36.690 --> 00:07:38.920
A Poisson approximation
would not apply.

00:07:38.920 --> 00:07:40.770
But n is pretty big.

00:07:40.770 --> 00:07:45.880
So that a normal approximation
would be appropriate.

00:07:45.880 --> 00:07:51.610
And finally, in this case,
we would obtain a Poisson

00:07:51.610 --> 00:07:58.159
approximation with parameter
100, because n times p is 100.

00:07:58.159 --> 00:08:01.810
But a Poisson random
variable with parameter 100

00:08:01.810 --> 00:08:04.570
is also well
approximated by a normal.

00:08:04.570 --> 00:08:07.190
Or to think about
it differently,

00:08:07.190 --> 00:08:10.130
we start with a Bernoulli
distribution that's

00:08:10.130 --> 00:08:17.190
very skewed, [the] probability
of success is just 100.

00:08:17.190 --> 00:08:20.720
And this makes it difficult
for the central limit theorem

00:08:20.720 --> 00:08:24.320
to apply when you start with a
very asymmetric distribution.

00:08:24.320 --> 00:08:28.840
On the other hand, because
we're adding so many of them,

00:08:28.840 --> 00:08:32.049
the central limit theorem
actually does take hold.

00:08:32.049 --> 00:08:36.299
And so this is an example
where both approximations

00:08:36.299 --> 00:08:38.490
will be valid.

00:08:38.490 --> 00:08:44.460
So finally, to conclude, we have
two different approximations.

00:08:44.460 --> 00:08:46.960
They're valid in
different regimes.

00:08:46.960 --> 00:08:49.660
And in practice, you
need to do some thinking

00:08:49.660 --> 00:08:54.640
before deciding to choose one
versus the other approximation.