WEBVTT 00:00:00.000 --> 00:00:00.040 align:middle line:90% 00:00:00.040 --> 00:00:02.460 align:middle line:84% The following content is provided under a Creative 00:00:02.460 --> 00:00:03.870 align:middle line:90% Commons license. 00:00:03.870 --> 00:00:06.910 align:middle line:84% Your support will help MIT OpenCourseWare continue to 00:00:06.910 --> 00:00:10.560 align:middle line:84% offer high-quality educational resources for free. 00:00:10.560 --> 00:00:13.460 align:middle line:84% To make a donation or view additional materials from 00:00:13.460 --> 00:00:19.290 align:middle line:84% hundreds of MIT courses, visit MIT OpenCourseWare at 00:00:19.290 --> 00:00:20.540 align:middle line:90% ocw.mit.edu. 00:00:20.540 --> 00:00:22.560 align:middle line:90% 00:00:22.560 --> 00:00:25.340 align:middle line:84% PROFESSOR: We're going to finish today our discussion of 00:00:25.340 --> 00:00:27.460 align:middle line:90% limit theorems. 00:00:27.460 --> 00:00:30.340 align:middle line:84% I'm going to remind you what the central limit theorem is, 00:00:30.340 --> 00:00:33.460 align:middle line:84% which we introduced briefly last time. 00:00:33.460 --> 00:00:37.230 align:middle line:84% We're going to discuss what exactly it says and its 00:00:37.230 --> 00:00:38.780 align:middle line:90% implications. 00:00:38.780 --> 00:00:42.100 align:middle line:84% And then we're going to apply to a couple of examples, 00:00:42.100 --> 00:00:45.520 align:middle line:84% mostly on the binomial distribution. 00:00:45.520 --> 00:00:49.950 align:middle line:84% OK, so the situation is that we are dealing with a large 00:00:49.950 --> 00:00:52.420 align:middle line:84% number of independent, identically 00:00:52.420 --> 00:00:55.000 align:middle line:90% distributed random variables. 00:00:55.000 --> 00:00:58.270 align:middle line:84% And we want to look at the sum of them and say something 00:00:58.270 --> 00:01:00.510 align:middle line:84% about the distribution of the sum. 00:01:00.510 --> 00:01:03.310 align:middle line:90% 00:01:03.310 --> 00:01:06.910 align:middle line:84% We might want to say that the sum is distributed 00:01:06.910 --> 00:01:10.510 align:middle line:84% approximately as a normal random variable, although, 00:01:10.510 --> 00:01:12.750 align:middle line:84% formally, this is not quite right. 00:01:12.750 --> 00:01:16.330 align:middle line:84% As n goes to infinity, the distribution of the sum 00:01:16.330 --> 00:01:20.000 align:middle line:84% becomes very spread out, and it doesn't converge to a 00:01:20.000 --> 00:01:21.830 align:middle line:90% limiting distribution. 00:01:21.830 --> 00:01:24.930 align:middle line:84% In order to get an interesting limit, we need first to take 00:01:24.930 --> 00:01:28.150 align:middle line:90% the sum and standardize it. 00:01:28.150 --> 00:01:32.267 align:middle line:84% By standardizing it, what we mean is to subtract the mean 00:01:32.267 --> 00:01:38.060 align:middle line:84% and then divide by the standard deviation. 00:01:38.060 --> 00:01:41.320 align:middle line:84% Now, the mean is, of course, n times the expected value of 00:01:41.320 --> 00:01:43.080 align:middle line:90% each one of the X's. 00:01:43.080 --> 00:01:45.130 align:middle line:84% And the standard deviation is the 00:01:45.130 --> 00:01:46.610 align:middle line:90% square root of the variance. 00:01:46.610 --> 00:01:50.530 align:middle line:84% The variance is n times sigma squared, where sigma is the 00:01:50.530 --> 00:01:52.180 align:middle line:90% variance of the X's -- 00:01:52.180 --> 00:01:53.400 align:middle line:84% so that's the standard deviation. 00:01:53.400 --> 00:01:56.330 align:middle line:84% And after we do this, we obtain a random variable that 00:01:56.330 --> 00:02:01.100 align:middle line:84% has 0 mean -- its centered -- and the 00:02:01.100 --> 00:02:03.230 align:middle line:90% variance is equal to 1. 00:02:03.230 --> 00:02:07.240 align:middle line:84% And so the variance stays the same, no matter how large n is 00:02:07.240 --> 00:02:08.500 align:middle line:90% going to be. 00:02:08.500 --> 00:02:12.660 align:middle line:84% So the distribution of Zn keeps changing with n, but it 00:02:12.660 --> 00:02:14.080 align:middle line:90% cannot change too much. 00:02:14.080 --> 00:02:15.240 align:middle line:90% It stays in place. 00:02:15.240 --> 00:02:19.550 align:middle line:84% The mean is 0, and the width remains also roughly the same 00:02:19.550 --> 00:02:22.000 align:middle line:90% because the variance is 1. 00:02:22.000 --> 00:02:25.820 align:middle line:84% The surprising thing is that, as n grows, that distribution 00:02:25.820 --> 00:02:31.250 align:middle line:84% of Zn kind of settles in a certain asymptotic shape. 00:02:31.250 --> 00:02:33.620 align:middle line:84% And that's the shape of a standard 00:02:33.620 --> 00:02:35.290 align:middle line:90% normal random variable. 00:02:35.290 --> 00:02:37.580 align:middle line:84% So standard normal means that it has 0 00:02:37.580 --> 00:02:39.930 align:middle line:90% mean and unit variance. 00:02:39.930 --> 00:02:43.850 align:middle line:84% More precisely, what the central limit theorem tells us 00:02:43.850 --> 00:02:46.560 align:middle line:84% is a relation between the cumulative distribution 00:02:46.560 --> 00:02:49.430 align:middle line:84% function of Zn and its relation to the cumulative 00:02:49.430 --> 00:02:51.990 align:middle line:84% distribution function of the standard normal. 00:02:51.990 --> 00:02:56.620 align:middle line:84% So for any given number, c, the probability that Zn is 00:02:56.620 --> 00:03:01.140 align:middle line:84% less than or equal to c, in the limit, becomes the same as 00:03:01.140 --> 00:03:04.090 align:middle line:84% the probability that the standard normal becomes less 00:03:04.090 --> 00:03:05.760 align:middle line:90% than or equal to c. 00:03:05.760 --> 00:03:08.800 align:middle line:84% And of course, this is useful because these probabilities 00:03:08.800 --> 00:03:11.960 align:middle line:84% are available from the normal tables, whereas the 00:03:11.960 --> 00:03:15.850 align:middle line:84% distribution of Zn might be a very complicated expression if 00:03:15.850 --> 00:03:19.520 align:middle line:84% you were to calculate it exactly. 00:03:19.520 --> 00:03:22.960 align:middle line:84% So some comments about the central limit theorem. 00:03:22.960 --> 00:03:27.860 align:middle line:84% First thing is that it's quite amazing that it's universal. 00:03:27.860 --> 00:03:31.970 align:middle line:84% It doesn't matter what the distribution of the X's is. 00:03:31.970 --> 00:03:35.970 align:middle line:84% It can be any distribution whatsoever, as long as it has 00:03:35.970 --> 00:03:39.070 align:middle line:84% finite mean and finite variance. 00:03:39.070 --> 00:03:42.170 align:middle line:84% And when you go and do your approximations using the 00:03:42.170 --> 00:03:44.520 align:middle line:84% central limit theorem, the only thing that you need to 00:03:44.520 --> 00:03:47.580 align:middle line:84% know about the distribution of the X's are the 00:03:47.580 --> 00:03:49.130 align:middle line:90% mean and the variance. 00:03:49.130 --> 00:03:52.410 align:middle line:84% You need those in order to standardize Sn. 00:03:52.410 --> 00:03:55.910 align:middle line:84% I mean -- to subtract the mean and divide by the standard 00:03:55.910 --> 00:03:56.810 align:middle line:90% deviation -- 00:03:56.810 --> 00:03:59.120 align:middle line:84% you need to know the mean and the variance. 00:03:59.120 --> 00:04:02.350 align:middle line:84% But these are the only things that you need to know in order 00:04:02.350 --> 00:04:03.600 align:middle line:90% to apply it. 00:04:03.600 --> 00:04:06.060 align:middle line:90% 00:04:06.060 --> 00:04:08.730 align:middle line:84% In addition, it's a very accurate 00:04:08.730 --> 00:04:10.650 align:middle line:90% computational shortcut. 00:04:10.650 --> 00:04:14.660 align:middle line:84% So the distribution of this Zn's, in principle, you can 00:04:14.660 --> 00:04:18.130 align:middle line:84% calculate it by convolution of the distribution of the X's 00:04:18.130 --> 00:04:20.050 align:middle line:90% with itself many, many times. 00:04:20.050 --> 00:04:23.720 align:middle line:84% But this is tedious, and if you try to do it analytically, 00:04:23.720 --> 00:04:26.570 align:middle line:84% it might be a very complicated expression. 00:04:26.570 --> 00:04:29.910 align:middle line:84% Whereas by just appealing to the standard normal table for 00:04:29.910 --> 00:04:33.870 align:middle line:84% the standard normal random variable, things are done in a 00:04:33.870 --> 00:04:35.360 align:middle line:90% very quick way. 00:04:35.360 --> 00:04:39.070 align:middle line:84% So it's a nice computational shortcut if you don't want to 00:04:39.070 --> 00:04:42.210 align:middle line:84% get an exact answer to a probability problem. 00:04:42.210 --> 00:04:47.480 align:middle line:84% Now, at a more philosophical level, it justifies why we are 00:04:47.480 --> 00:04:50.930 align:middle line:84% really interested in normal random variables. 00:04:50.930 --> 00:04:55.230 align:middle line:84% Whenever you have a phenomenon which is noisy, and the noise 00:04:55.230 --> 00:05:00.420 align:middle line:84% that you observe is created by adding the lots of little 00:05:00.420 --> 00:05:03.820 align:middle line:84% pieces of randomness that are independent of each other, the 00:05:03.820 --> 00:05:06.840 align:middle line:84% overall effect that you're going to observe can be 00:05:06.840 --> 00:05:10.240 align:middle line:84% described by a normal random variable. 00:05:10.240 --> 00:05:16.810 align:middle line:84% So in a classic example that goes 100 years back or so, 00:05:16.810 --> 00:05:19.840 align:middle line:84% suppose that you have a fluid, and inside that fluid, there's 00:05:19.840 --> 00:05:23.340 align:middle line:84% a little particle of dust or whatever that's 00:05:23.340 --> 00:05:24.950 align:middle line:90% suspended in there. 00:05:24.950 --> 00:05:28.380 align:middle line:84% That little particle gets hit by molecules 00:05:28.380 --> 00:05:30.000 align:middle line:90% completely at random -- 00:05:30.000 --> 00:05:32.730 align:middle line:84% and so what you're going to see is that particle kind of 00:05:32.730 --> 00:05:36.020 align:middle line:84% moving randomly inside that liquid. 00:05:36.020 --> 00:05:40.260 align:middle line:84% Now that random motion, if you ask, after one second, how 00:05:40.260 --> 00:05:45.520 align:middle line:84% much is my particle displaced, let's say, in the x-axis along 00:05:45.520 --> 00:05:47.170 align:middle line:90% the x direction. 00:05:47.170 --> 00:05:50.960 align:middle line:84% That displacement is very, very well modeled by a normal 00:05:50.960 --> 00:05:51.960 align:middle line:90% random variable. 00:05:51.960 --> 00:05:55.710 align:middle line:84% And the reason is that the position of that particle is 00:05:55.710 --> 00:06:00.160 align:middle line:84% decided by the cumulative effect of lots of random hits 00:06:00.160 --> 00:06:04.480 align:middle line:84% by molecules that hit that particle. 00:06:04.480 --> 00:06:11.630 align:middle line:84% So that's a sort of celebrated physical model that goes under 00:06:11.630 --> 00:06:15.000 align:middle line:90% the name of Brownian motion. 00:06:15.000 --> 00:06:18.100 align:middle line:84% And it's the same model that some people use to describe 00:06:18.100 --> 00:06:20.300 align:middle line:84% the movement in the financial markets. 00:06:20.300 --> 00:06:24.660 align:middle line:84% The argument might go that the movement of prices has to do 00:06:24.660 --> 00:06:28.300 align:middle line:84% with lots of little decisions and lots of little events by 00:06:28.300 --> 00:06:31.310 align:middle line:84% many, many different actors that are 00:06:31.310 --> 00:06:32.890 align:middle line:90% involved in the market. 00:06:32.890 --> 00:06:37.440 align:middle line:84% So the distribution of stock prices might be well described 00:06:37.440 --> 00:06:39.740 align:middle line:90% by normal random variables. 00:06:39.740 --> 00:06:43.780 align:middle line:84% At least that's what people wanted to believe until 00:06:43.780 --> 00:06:45.160 align:middle line:90% somewhat recently. 00:06:45.160 --> 00:06:48.300 align:middle line:84% Now, the evidence is that, actually, these distributions 00:06:48.300 --> 00:06:52.210 align:middle line:84% are a little more heavy-tailed in the sense that extreme 00:06:52.210 --> 00:06:55.630 align:middle line:84% events are a little more likely to occur that what 00:06:55.630 --> 00:06:58.040 align:middle line:84% normal random variables would seem to indicate. 00:06:58.040 --> 00:07:03.110 align:middle line:84% But as a first model, again, it could be a plausible 00:07:03.110 --> 00:07:07.300 align:middle line:84% argument to have, at least as a starting model, one that 00:07:07.300 --> 00:07:10.200 align:middle line:84% involves normal random variables. 00:07:10.200 --> 00:07:13.020 align:middle line:84% So this is the philosophical side of things. 00:07:13.020 --> 00:07:15.820 align:middle line:84% On the more accurate, mathematical side, it's 00:07:15.820 --> 00:07:18.290 align:middle line:84% important to appreciate exactly quite kind of 00:07:18.290 --> 00:07:21.250 align:middle line:84% statement the central limit theorem is. 00:07:21.250 --> 00:07:25.460 align:middle line:84% It's a statement about the convergence of the CDF of 00:07:25.460 --> 00:07:27.940 align:middle line:84% these standardized random variables to 00:07:27.940 --> 00:07:29.840 align:middle line:90% the CDF of a normal. 00:07:29.840 --> 00:07:32.470 align:middle line:84% So it's a statement about convergence of CDFs. 00:07:32.470 --> 00:07:36.580 align:middle line:84% It's not a statement about convergence of PMFs, or 00:07:36.580 --> 00:07:39.100 align:middle line:90% convergence of PDFs. 00:07:39.100 --> 00:07:42.160 align:middle line:84% Now, if one makes additional mathematical assumptions, 00:07:42.160 --> 00:07:44.880 align:middle line:84% there are variations of the central limit theorem that 00:07:44.880 --> 00:07:47.220 align:middle line:90% talk about PDFs and PMFs. 00:07:47.220 --> 00:07:51.930 align:middle line:84% But in general, that's not necessarily the case. 00:07:51.930 --> 00:07:54.610 align:middle line:84% And I'm going to illustrate this with-- 00:07:54.610 --> 00:07:58.890 align:middle line:84% I have a plot here which is not in your slides. 00:07:58.890 --> 00:08:04.700 align:middle line:84% But just to make the point, consider two different 00:08:04.700 --> 00:08:06.710 align:middle line:90% discrete distributions. 00:08:06.710 --> 00:08:09.820 align:middle line:84% This discrete distribution takes values 1, 4, 7. 00:08:09.820 --> 00:08:13.470 align:middle line:90% 00:08:13.470 --> 00:08:16.110 align:middle line:84% This discrete distribution can take values 1, 2, 4, 6, and 7. 00:08:16.110 --> 00:08:18.720 align:middle line:90% 00:08:18.720 --> 00:08:24.270 align:middle line:84% So this one has sort of a periodicity of 3, this one, 00:08:24.270 --> 00:08:27.960 align:middle line:84% the range of values is a little more interesting. 00:08:27.960 --> 00:08:30.910 align:middle line:84% The numbers in these two distributions are cooked up so 00:08:30.910 --> 00:08:34.380 align:middle line:84% that they have the same mean and the same variance. 00:08:34.380 --> 00:08:38.970 align:middle line:84% Now, what I'm going to do is to take eight independent 00:08:38.970 --> 00:08:44.090 align:middle line:84% copies of the random variable and plot the PMF of the sum of 00:08:44.090 --> 00:08:45.980 align:middle line:90% eight random variables. 00:08:45.980 --> 00:08:51.520 align:middle line:84% Now, if I plot the PMF of the sum of 8 of these, I get the 00:08:51.520 --> 00:08:59.690 align:middle line:84% plot, which corresponds to these bullets in this diagram. 00:08:59.690 --> 00:09:03.040 align:middle line:84% If I take 8 random variables, according to this 00:09:03.040 --> 00:09:07.270 align:middle line:84% distribution, and add them up and compute their PMF, the PMF 00:09:07.270 --> 00:09:10.310 align:middle line:84% I get is the one denoted here by the X's. 00:09:10.310 --> 00:09:15.630 align:middle line:84% The two PMFs look really different, at least, when you 00:09:15.630 --> 00:09:16.890 align:middle line:90% eyeball them. 00:09:16.890 --> 00:09:23.500 align:middle line:84% On the other hand, if you were to plot the CDFs of them, then 00:09:23.500 --> 00:09:34.000 align:middle line:84% the CDFs, if you compare them with the normal CDF, which is 00:09:34.000 --> 00:09:38.390 align:middle line:84% this continuous curve, the CDF, of course, it goes up in 00:09:38.390 --> 00:09:41.870 align:middle line:84% steps because we're looking at discrete random variables. 00:09:41.870 --> 00:09:47.600 align:middle line:84% But it's very close to the normal CDF. 00:09:47.600 --> 00:09:52.000 align:middle line:84% And if we, instead of n equal to 8, we were to take 16, then 00:09:52.000 --> 00:09:54.480 align:middle line:84% the coincidence would be even better. 00:09:54.480 --> 00:09:59.850 align:middle line:84% So in terms of CDFs, when we add 8 or 16 of these, we get 00:09:59.850 --> 00:10:01.930 align:middle line:90% very close to the normal CDF. 00:10:01.930 --> 00:10:05.080 align:middle line:84% We would get essentially the same picture if I were to take 00:10:05.080 --> 00:10:06.850 align:middle line:90% 8 or 16 of these. 00:10:06.850 --> 00:10:11.730 align:middle line:84% So the CDFs sit, essentially, on top of each other, although 00:10:11.730 --> 00:10:14.400 align:middle line:84% the two PMFs look quite different. 00:10:14.400 --> 00:10:17.230 align:middle line:84% So this is to appreciate that, formally speaking, we only 00:10:17.230 --> 00:10:22.470 align:middle line:84% have a statement about CDFs, not about PMFs. 00:10:22.470 --> 00:10:26.980 align:middle line:84% Now in practice, how do you use the central limit theorem? 00:10:26.980 --> 00:10:30.550 align:middle line:84% Well, it tells us that we can calculate probabilities by 00:10:30.550 --> 00:10:32.810 align:middle line:84% treating Zn as if it were a standard 00:10:32.810 --> 00:10:34.550 align:middle line:90% normal random variable. 00:10:34.550 --> 00:10:38.280 align:middle line:84% Now Zn is a linear function of Sn. 00:10:38.280 --> 00:10:43.120 align:middle line:84% Conversely, Sn is a linear function of Zn. 00:10:43.120 --> 00:10:45.680 align:middle line:84% Linear functions of normals are normal. 00:10:45.680 --> 00:10:49.450 align:middle line:84% So if I pretend that Zn is normal, it's essentially the 00:10:49.450 --> 00:10:53.230 align:middle line:84% same as if we pretend that Sn is normal. 00:10:53.230 --> 00:10:55.560 align:middle line:84% And so we can calculate probabilities that have to do 00:10:55.560 --> 00:10:59.830 align:middle line:90% with Sn as if Sn were normal. 00:10:59.830 --> 00:11:03.850 align:middle line:84% Now, the central limit theorem does not tell us that Sn is 00:11:03.850 --> 00:11:05.120 align:middle line:90% approximately normal. 00:11:05.120 --> 00:11:08.860 align:middle line:84% The formal statement is about Zn, but, practically speaking, 00:11:08.860 --> 00:11:11.150 align:middle line:84% when you use the result, you can just 00:11:11.150 --> 00:11:14.650 align:middle line:90% pretend that Sn is normal. 00:11:14.650 --> 00:11:18.620 align:middle line:84% Finally, it's a limit theorem, so it tells us about what 00:11:18.620 --> 00:11:21.240 align:middle line:84% happens when n goes to infinity. 00:11:21.240 --> 00:11:23.880 align:middle line:84% If we are to use it in practice, of course, n is not 00:11:23.880 --> 00:11:25.120 align:middle line:90% going to be infinity. 00:11:25.120 --> 00:11:28.320 align:middle line:90% Maybe n is equal to 15. 00:11:28.320 --> 00:11:32.130 align:middle line:84% Can we use a limit theorem when n is a small number, as 00:11:32.130 --> 00:11:34.020 align:middle line:90% small as 15? 00:11:34.020 --> 00:11:36.980 align:middle line:84% Well, it turns out that it's a very good approximation. 00:11:36.980 --> 00:11:41.420 align:middle line:84% Even for quite small values of n, it gives us 00:11:41.420 --> 00:11:43.770 align:middle line:90% very accurate answers. 00:11:43.770 --> 00:11:49.710 align:middle line:84% So n over the order of 15, or 20, or so give us very good 00:11:49.710 --> 00:11:51.790 align:middle line:90% results in practice. 00:11:51.790 --> 00:11:54.820 align:middle line:84% There are no good theorems that will give us hard 00:11:54.820 --> 00:11:58.550 align:middle line:84% guarantees because the quality of the approximation does 00:11:58.550 --> 00:12:03.490 align:middle line:84% depend on the details of the distribution of the X's. 00:12:03.490 --> 00:12:07.510 align:middle line:84% If the X's have a distribution that, from the outset, looks a 00:12:07.510 --> 00:12:13.200 align:middle line:84% little bit like the normal, then for small values of n, 00:12:13.200 --> 00:12:15.700 align:middle line:84% you are going to see, essentially, a normal 00:12:15.700 --> 00:12:16.980 align:middle line:90% distribution for the sum. 00:12:16.980 --> 00:12:20.030 align:middle line:84% If the distribution of the X's is very different from the 00:12:20.030 --> 00:12:23.350 align:middle line:84% normal, it's going to take a larger value of n for the 00:12:23.350 --> 00:12:25.770 align:middle line:84% central limit theorem to take effect. 00:12:25.770 --> 00:12:29.960 align:middle line:84% So let's illustrates this with a few representative plots. 00:12:29.960 --> 00:12:32.600 align:middle line:90% 00:12:32.600 --> 00:12:36.460 align:middle line:84% So here, we're starting with a discrete uniform distribution 00:12:36.460 --> 00:12:39.580 align:middle line:90% that goes from 1 to 8. 00:12:39.580 --> 00:12:44.200 align:middle line:84% Let's add 2 of these random variables, 2 random variables 00:12:44.200 --> 00:12:47.870 align:middle line:84% with this PMF, and find the PMF of the sum. 00:12:47.870 --> 00:12:52.570 align:middle line:84% This is a convolution of 2 discrete uniforms, and I 00:12:52.570 --> 00:12:54.960 align:middle line:84% believe you have seen this exercise before. 00:12:54.960 --> 00:12:59.030 align:middle line:84% When you convolve this with itself, you get a triangle. 00:12:59.030 --> 00:13:04.400 align:middle line:84% So this is the PMF for the sum of two discrete uniforms. 00:13:04.400 --> 00:13:05.370 align:middle line:90% Now let's continue. 00:13:05.370 --> 00:13:07.980 align:middle line:84% Let's convolve this with itself. 00:13:07.980 --> 00:13:10.750 align:middle line:84% These was going to give us the PMF of a sum 00:13:10.750 --> 00:13:13.740 align:middle line:90% of 4 discrete uniforms. 00:13:13.740 --> 00:13:17.930 align:middle line:84% And we get this, which starts looking like a normal. 00:13:17.930 --> 00:13:23.450 align:middle line:84% If we go to n equal to 32, then it looks, essentially, 00:13:23.450 --> 00:13:25.270 align:middle line:90% exactly like a normal. 00:13:25.270 --> 00:13:27.850 align:middle line:84% And it's an excellent approximation. 00:13:27.850 --> 00:13:32.290 align:middle line:84% So this is the PMF of the sum of 32 discrete random 00:13:32.290 --> 00:13:36.560 align:middle line:84% variables with this uniform distribution. 00:13:36.560 --> 00:13:42.190 align:middle line:84% If we start with a PMF which is not symmetric-- 00:13:42.190 --> 00:13:44.640 align:middle line:84% this one is symmetric around the mean. 00:13:44.640 --> 00:13:47.630 align:middle line:84% But if we start with a PMF which is non-symmetric, so 00:13:47.630 --> 00:13:53.780 align:middle line:84% this is, here, is a truncated geometric PMF, then things do 00:13:53.780 --> 00:13:58.960 align:middle line:84% not work out as nicely when I add 8 of these. 00:13:58.960 --> 00:14:03.640 align:middle line:84% That is, if I convolve this with itself 8 times, I get 00:14:03.640 --> 00:14:08.600 align:middle line:84% this PMF, which maybe resembles a little bit to the 00:14:08.600 --> 00:14:09.800 align:middle line:90% normal one. 00:14:09.800 --> 00:14:13.050 align:middle line:84% But you can really tell that it's different from the normal 00:14:13.050 --> 00:14:16.640 align:middle line:84% if you focus at the details here and there. 00:14:16.640 --> 00:14:19.930 align:middle line:90% Here it sort of rises sharply. 00:14:19.930 --> 00:14:23.420 align:middle line:84% Here it tails off a bit slower. 00:14:23.420 --> 00:14:27.660 align:middle line:84% So there's an asymmetry here that's present, and which is a 00:14:27.660 --> 00:14:29.340 align:middle line:84% consequence of the asymmetry of the 00:14:29.340 --> 00:14:31.710 align:middle line:90% distribution we started with. 00:14:31.710 --> 00:14:35.320 align:middle line:84% If we go to 16, it looks a little better, but still you 00:14:35.320 --> 00:14:39.600 align:middle line:84% can see the asymmetry between this tail and that tail. 00:14:39.600 --> 00:14:43.030 align:middle line:84% If you get to 32 there's still a little bit of asymmetry, but 00:14:43.030 --> 00:14:48.520 align:middle line:84% at least now it starts looking like a normal distribution. 00:14:48.520 --> 00:14:54.270 align:middle line:84% So the moral from these plots is that it might vary, a 00:14:54.270 --> 00:14:57.360 align:middle line:84% little bit, what kind of values of n you need before 00:14:57.360 --> 00:15:00.070 align:middle line:84% you get the really good approximation. 00:15:00.070 --> 00:15:04.520 align:middle line:84% But for values of n in the range 20 to 30 or so, usually 00:15:04.520 --> 00:15:07.340 align:middle line:84% you expect to get a pretty good approximation. 00:15:07.340 --> 00:15:10.180 align:middle line:84% At least that's what the visual inspection of these 00:15:10.180 --> 00:15:13.330 align:middle line:90% graphs tells us. 00:15:13.330 --> 00:15:16.560 align:middle line:84% So now that we know that we have a good approximation in 00:15:16.560 --> 00:15:18.460 align:middle line:90% our hands, let's use it. 00:15:18.460 --> 00:15:21.890 align:middle line:84% Let's use it by revisiting an example from last time. 00:15:21.890 --> 00:15:24.480 align:middle line:90% This is the polling problem. 00:15:24.480 --> 00:15:28.360 align:middle line:84% We're interested in the fraction of population that 00:15:28.360 --> 00:15:30.220 align:middle line:90% has a certain habit been. 00:15:30.220 --> 00:15:33.680 align:middle line:90% And we try to find what f is. 00:15:33.680 --> 00:15:38.120 align:middle line:84% And the way we do it is by polling people at random and 00:15:38.120 --> 00:15:40.600 align:middle line:84% recording the answers that they give, whether they have 00:15:40.600 --> 00:15:42.340 align:middle line:90% the habit or not. 00:15:42.340 --> 00:15:45.250 align:middle line:84% So for each person, we get the Bernoulli random variable. 00:15:45.250 --> 00:15:52.050 align:middle line:84% With probability f, a person is going to respond 1, or yes, 00:15:52.050 --> 00:15:55.080 align:middle line:90% so this is with probability f. 00:15:55.080 --> 00:15:58.490 align:middle line:84% And with the remaining probability 1-f, the person 00:15:58.490 --> 00:16:00.390 align:middle line:90% responds no. 00:16:00.390 --> 00:16:04.520 align:middle line:84% We record this number, which is how many people answered 00:16:04.520 --> 00:16:06.800 align:middle line:84% yes, divided by the total number of people. 00:16:06.800 --> 00:16:10.740 align:middle line:84% That's the fraction of the population that we asked. 00:16:10.740 --> 00:16:16.980 align:middle line:84% This is the fraction inside our sample that answered yes. 00:16:16.980 --> 00:16:21.410 align:middle line:84% And as we discussed last time, you might start with some 00:16:21.410 --> 00:16:23.210 align:middle line:90% specs for the poll. 00:16:23.210 --> 00:16:25.660 align:middle line:84% And the specs have two parameters-- 00:16:25.660 --> 00:16:29.400 align:middle line:84% the accuracy that you want and the confidence that you want 00:16:29.400 --> 00:16:33.620 align:middle line:84% to have that you did really obtain the desired accuracy. 00:16:33.620 --> 00:16:40.550 align:middle line:84% So the specs here is that we want, probability 95% that our 00:16:40.550 --> 00:16:46.400 align:middle line:84% estimate is within 1 % point from the true answer. 00:16:46.400 --> 00:16:48.940 align:middle line:84% So the event of interest is this. 00:16:48.940 --> 00:16:53.640 align:middle line:84% That's the result of the poll minus distance from the true 00:16:53.640 --> 00:16:59.150 align:middle line:84% answer is less or bigger than 1 % point. 00:16:59.150 --> 00:17:02.000 align:middle line:84% And we're interested in calculating or approximating 00:17:02.000 --> 00:17:04.140 align:middle line:90% this particular probability. 00:17:04.140 --> 00:17:08.000 align:middle line:84% So we want to do it using the central limit theorem. 00:17:08.000 --> 00:17:13.050 align:middle line:84% And one way of arranging the mechanics of this calculation 00:17:13.050 --> 00:17:17.880 align:middle line:84% is to take the event of interest and massage it by 00:17:17.880 --> 00:17:21.400 align:middle line:84% subtracting and dividing things from both sides of this 00:17:21.400 --> 00:17:27.510 align:middle line:84% inequality so that you bring him to the picture the 00:17:27.510 --> 00:17:31.600 align:middle line:84% standardized random variable, the Zn, and then apply the 00:17:31.600 --> 00:17:33.900 align:middle line:90% central limit theorem. 00:17:33.900 --> 00:17:38.550 align:middle line:84% So the event of interest, let me write it in full, Mn is 00:17:38.550 --> 00:17:42.280 align:middle line:84% this quantity, so I'm putting it here, minus f, which is the 00:17:42.280 --> 00:17:44.410 align:middle line:90% same as nf divided by n. 00:17:44.410 --> 00:17:46.980 align:middle line:84% So this is the same as that event. 00:17:46.980 --> 00:17:49.840 align:middle line:84% We're going to calculate the probability of this. 00:17:49.840 --> 00:17:52.460 align:middle line:84% This is not exactly in the form in which we apply the 00:17:52.460 --> 00:17:53.430 align:middle line:90% central limit theorem. 00:17:53.430 --> 00:17:56.570 align:middle line:84% To apply the central limit theorem, we need, down here, 00:17:56.570 --> 00:17:59.660 align:middle line:90% to have sigma square root n. 00:17:59.660 --> 00:18:03.100 align:middle line:84% So how can I put sigma square root n here? 00:18:03.100 --> 00:18:07.350 align:middle line:84% I can divide both sides of this inequality by sigma. 00:18:07.350 --> 00:18:10.970 align:middle line:84% And then I can take a factor of square root n from here and 00:18:10.970 --> 00:18:13.240 align:middle line:90% send it to the other side. 00:18:13.240 --> 00:18:15.660 align:middle line:84% So this event is the same as that event. 00:18:15.660 --> 00:18:19.190 align:middle line:84% This will happen if and only if that will happen. 00:18:19.190 --> 00:18:23.670 align:middle line:84% So calculating the probability of this event here is the same 00:18:23.670 --> 00:18:27.110 align:middle line:84% as calculating the probability that this events happens. 00:18:27.110 --> 00:18:30.870 align:middle line:84% And now we are in business because the random variable 00:18:30.870 --> 00:18:36.510 align:middle line:84% that we got in here is Zn, or the absolute value of Zn, and 00:18:36.510 --> 00:18:41.480 align:middle line:84% we're talking about the probability that Zn, absolute 00:18:41.480 --> 00:18:45.660 align:middle line:84% value of Zn, is bigger than a certain number. 00:18:45.660 --> 00:18:50.310 align:middle line:84% Since Zn is to be approximated by a standard normal random 00:18:50.310 --> 00:18:54.560 align:middle line:84% variable, our approximation is going to be, instead of asking 00:18:54.560 --> 00:18:59.040 align:middle line:84% for Zn being bigger than this number, we will ask for Z, 00:18:59.040 --> 00:19:02.500 align:middle line:84% absolute value of Z, being bigger than this number. 00:19:02.500 --> 00:19:05.640 align:middle line:84% So this is the probability that we want to calculate. 00:19:05.640 --> 00:19:09.730 align:middle line:84% And now Z is a standard normal random variable. 00:19:09.730 --> 00:19:12.760 align:middle line:84% There's a small difficulty, the one that we also 00:19:12.760 --> 00:19:14.310 align:middle line:90% encountered last time. 00:19:14.310 --> 00:19:18.110 align:middle line:84% And the difficulty is that the standard deviation, sigma, of 00:19:18.110 --> 00:19:20.720 align:middle line:90% the Xi's is not known. 00:19:20.720 --> 00:19:24.560 align:middle line:90% Sigma is equal to f times-- 00:19:24.560 --> 00:19:30.090 align:middle line:84% sigma, in this example, is f times (1-f), and the only 00:19:30.090 --> 00:19:32.690 align:middle line:84% thing that we know about sigma is that it's going to be a 00:19:32.690 --> 00:19:35.010 align:middle line:90% number less than 1/2. 00:19:35.010 --> 00:19:39.810 align:middle line:90% 00:19:39.810 --> 00:19:45.180 align:middle line:84% OK, so we're going to have to use an inequality here. 00:19:45.180 --> 00:19:48.890 align:middle line:84% We're going to use a conservative value of sigma, 00:19:48.890 --> 00:19:54.120 align:middle line:84% the value of sigma equal to 1/2 and use that instead of 00:19:54.120 --> 00:19:55.760 align:middle line:90% the exact value of sigma. 00:19:55.760 --> 00:19:59.100 align:middle line:84% And this gives us an inequality going this way. 00:19:59.100 --> 00:20:03.710 align:middle line:84% Let's just make sure why the inequality goes this way. 00:20:03.710 --> 00:20:06.683 align:middle line:84% We got, on our axis, two numbers. 00:20:06.683 --> 00:20:12.390 align:middle line:90% 00:20:12.390 --> 00:20:21.650 align:middle line:84% One number is 0.01 square root n divided by sigma. 00:20:21.650 --> 00:20:27.870 align:middle line:84% And the other number is 0.02 square root of n. 00:20:27.870 --> 00:20:30.840 align:middle line:84% And my claim is that the numbers are related to each 00:20:30.840 --> 00:20:32.930 align:middle line:90% other in this particular way. 00:20:32.930 --> 00:20:33.500 align:middle line:90% Why is this? 00:20:33.500 --> 00:20:35.410 align:middle line:90% Sigma is less than 2. 00:20:35.410 --> 00:20:39.580 align:middle line:90% So 1/sigma is bigger than 2. 00:20:39.580 --> 00:20:44.020 align:middle line:84% So since 1/sigma is bigger than 2 this means that this 00:20:44.020 --> 00:20:47.740 align:middle line:84% numbers sits to the right of that number. 00:20:47.740 --> 00:20:51.950 align:middle line:84% So here we have the probability that Z is bigger 00:20:51.950 --> 00:20:54.820 align:middle line:90% than this number. 00:20:54.820 --> 00:20:59.060 align:middle line:84% The probability of falling out there is less than the 00:20:59.060 --> 00:21:03.060 align:middle line:84% probability of falling in this interval. 00:21:03.060 --> 00:21:06.170 align:middle line:84% So that's what that last inequality is saying-- 00:21:06.170 --> 00:21:09.330 align:middle line:84% this probability is smaller than that probability. 00:21:09.330 --> 00:21:12.010 align:middle line:84% This is the probability that we're interested in, but since 00:21:12.010 --> 00:21:16.490 align:middle line:84% we don't know sigma, we take the conservative value, and we 00:21:16.490 --> 00:21:21.610 align:middle line:84% use an upper bound in terms of the probability of this 00:21:21.610 --> 00:21:23.730 align:middle line:90% interval here. 00:21:23.730 --> 00:21:26.920 align:middle line:90% And now we are in business. 00:21:26.920 --> 00:21:30.980 align:middle line:84% We can start using our normal tables to calculate 00:21:30.980 --> 00:21:33.140 align:middle line:90% probabilities of interest. 00:21:33.140 --> 00:21:40.300 align:middle line:84% So for example, let's say that's we take n to be 10,000. 00:21:40.300 --> 00:21:42.370 align:middle line:84% How is the calculation going to go? 00:21:42.370 --> 00:21:45.860 align:middle line:84% We want to calculate the probability that the absolute 00:21:45.860 --> 00:21:52.920 align:middle line:84% value of Z is bigger than 0.2 times 1000, which is the 00:21:52.920 --> 00:21:56.530 align:middle line:84% probability that the absolute value of Z is larger than or 00:21:56.530 --> 00:21:58.490 align:middle line:90% equal to 2. 00:21:58.490 --> 00:22:00.500 align:middle line:84% And here let's do some mechanics, 00:22:00.500 --> 00:22:03.300 align:middle line:90% just to stay in shape. 00:22:03.300 --> 00:22:05.860 align:middle line:84% The probability that you're larger than or equal to 2 in 00:22:05.860 --> 00:22:09.290 align:middle line:84% absolute value, since the normal is symmetric around the 00:22:09.290 --> 00:22:13.590 align:middle line:84% mean, this is going to be twice the probability that Z 00:22:13.590 --> 00:22:16.560 align:middle line:90% is larger than or equal to 2. 00:22:16.560 --> 00:22:22.330 align:middle line:84% Can we use the cumulative distribution function of Z to 00:22:22.330 --> 00:22:23.300 align:middle line:90% calculate this? 00:22:23.300 --> 00:22:26.100 align:middle line:84% Well, almost the cumulative gives us probabilities of 00:22:26.100 --> 00:22:28.910 align:middle line:84% being less than something, not bigger than something. 00:22:28.910 --> 00:22:33.480 align:middle line:84% So we need one more step and write this as 1 minus the 00:22:33.480 --> 00:22:38.420 align:middle line:84% probability that Z is less than or equal to 2. 00:22:38.420 --> 00:22:41.620 align:middle line:84% And this probability, now, you can read off 00:22:41.620 --> 00:22:43.770 align:middle line:90% from the normal tables. 00:22:43.770 --> 00:22:46.460 align:middle line:84% And the normal tables will tell you that this 00:22:46.460 --> 00:22:52.840 align:middle line:90% probability is 0.9772. 00:22:52.840 --> 00:22:54.520 align:middle line:90% And you do get an answer. 00:22:54.520 --> 00:23:02.530 align:middle line:90% And the answer is 0.0456. 00:23:02.530 --> 00:23:05.220 align:middle line:90% OK, so we tried 10,000. 00:23:05.220 --> 00:23:10.990 align:middle line:84% And we find that our probably of error is 4.5%, so we're 00:23:10.990 --> 00:23:15.710 align:middle line:84% doing better than the spec that we had. 00:23:15.710 --> 00:23:19.490 align:middle line:84% So this tells us that maybe we have some leeway. 00:23:19.490 --> 00:23:24.070 align:middle line:84% Maybe we can use a smaller sample size and still stay 00:23:24.070 --> 00:23:26.030 align:middle line:90% without our specs. 00:23:26.030 --> 00:23:29.630 align:middle line:84% Let's try to find how much we can push the envelope. 00:23:29.630 --> 00:23:34.716 align:middle line:84% How much smaller can we take n? 00:23:34.716 --> 00:23:37.890 align:middle line:84% To answer that question, we need to do this kind of 00:23:37.890 --> 00:23:40.790 align:middle line:84% calculation, essentially, going backwards. 00:23:40.790 --> 00:23:46.420 align:middle line:84% We're going to fix this number to be 0.05 and work backwards 00:23:46.420 --> 00:23:49.130 align:middle line:90% here to find-- 00:23:49.130 --> 00:23:50.770 align:middle line:90% did I do a mistake here? 00:23:50.770 --> 00:23:51.770 align:middle line:90% 10,000. 00:23:51.770 --> 00:23:53.700 align:middle line:90% So I'm missing a 0 here. 00:23:53.700 --> 00:23:57.440 align:middle line:90% 00:23:57.440 --> 00:24:07.540 align:middle line:84% Ah, but I'm taking the square root, so it's 100. 00:24:07.540 --> 00:24:11.080 align:middle line:84% Where did the 0.02 come in from? 00:24:11.080 --> 00:24:12.020 align:middle line:90% Ah, from here. 00:24:12.020 --> 00:24:15.870 align:middle line:90% OK, all right. 00:24:15.870 --> 00:24:19.330 align:middle line:84% 0.02 times 100, that gives us 2. 00:24:19.330 --> 00:24:22.130 align:middle line:90% OK, all right. 00:24:22.130 --> 00:24:24.240 align:middle line:90% Very good, OK. 00:24:24.240 --> 00:24:27.570 align:middle line:84% So we'll have to do this calculation now backwards, 00:24:27.570 --> 00:24:33.510 align:middle line:84% figure out if this is 0.05, what kind of number we're 00:24:33.510 --> 00:24:41.380 align:middle line:84% going to need here and then here, and from this we will be 00:24:41.380 --> 00:24:45.240 align:middle line:84% able to tell what value of n do we need. 00:24:45.240 --> 00:24:53.670 align:middle line:84% OK, so we want to find n such that the probability that Z is 00:24:53.670 --> 00:25:04.870 align:middle line:84% bigger than 0.02 square root n is 0.05. 00:25:04.870 --> 00:25:09.320 align:middle line:84% OK, so Z is a standard normal random variable. 00:25:09.320 --> 00:25:16.810 align:middle line:84% And we want the probability that we are 00:25:16.810 --> 00:25:18.640 align:middle line:90% outside this range. 00:25:18.640 --> 00:25:21.940 align:middle line:84% We want the probability of those two tails together. 00:25:21.940 --> 00:25:24.960 align:middle line:90% 00:25:24.960 --> 00:25:26.920 align:middle line:84% Those two tails together should have 00:25:26.920 --> 00:25:29.990 align:middle line:90% probability of 0.05. 00:25:29.990 --> 00:25:33.280 align:middle line:84% This means that this tail, by itself, should have 00:25:33.280 --> 00:25:36.900 align:middle line:90% probability 0.025. 00:25:36.900 --> 00:25:45.960 align:middle line:84% And this means that this probability should be 0.975. 00:25:45.960 --> 00:25:52.350 align:middle line:84% Now, if this probability is to be 0.975, what 00:25:52.350 --> 00:25:54.970 align:middle line:90% should that number be? 00:25:54.970 --> 00:25:59.980 align:middle line:84% You go to the normal tables, and you find which is the 00:25:59.980 --> 00:26:03.190 align:middle line:84% entry that corresponds to that number. 00:26:03.190 --> 00:26:07.020 align:middle line:84% I actually brought a normal table with me. 00:26:07.020 --> 00:26:12.740 align:middle line:90% And 0.975 is down here. 00:26:12.740 --> 00:26:15.420 align:middle line:84% And it tells you that to the number that 00:26:15.420 --> 00:26:19.820 align:middle line:90% corresponds to it is 1.96. 00:26:19.820 --> 00:26:24.890 align:middle line:84% So this tells us that this number 00:26:24.890 --> 00:26:31.790 align:middle line:90% should be equal to 1.96. 00:26:31.790 --> 00:26:36.380 align:middle line:84% And now, from here, you do the calculations. 00:26:36.380 --> 00:26:47.510 align:middle line:90% And you find that n is 9604. 00:26:47.510 --> 00:26:53.200 align:middle line:84% So with a sample of 10,000, we got probability of error 4.5%. 00:26:53.200 --> 00:26:57.910 align:middle line:84% With a slightly smaller sample size of 9,600, we can get the 00:26:57.910 --> 00:27:01.880 align:middle line:84% probability of a mistake to be 0.05, which 00:27:01.880 --> 00:27:04.070 align:middle line:90% was exactly our spec. 00:27:04.070 --> 00:27:07.450 align:middle line:84% So these are essentially the two ways that you're going to 00:27:07.450 --> 00:27:09.830 align:middle line:84% be using the central limit theorem. 00:27:09.830 --> 00:27:12.690 align:middle line:84% Either you're given n and you try to calculate 00:27:12.690 --> 00:27:13.610 align:middle line:90% probabilities. 00:27:13.610 --> 00:27:15.590 align:middle line:84% Or you're given the probabilities, and you want to 00:27:15.590 --> 00:27:18.210 align:middle line:84% work backwards to find n itself. 00:27:18.210 --> 00:27:20.990 align:middle line:90% 00:27:20.990 --> 00:27:27.710 align:middle line:84% So in this example, the random variable that we dealt with 00:27:27.710 --> 00:27:30.450 align:middle line:84% was, of course, a binomial random variable. 00:27:30.450 --> 00:27:38.590 align:middle line:84% The Xi's were Bernoulli, so the sum of 00:27:38.590 --> 00:27:40.950 align:middle line:90% the Xi's were binomial. 00:27:40.950 --> 00:27:44.100 align:middle line:84% So the central limit theorem certainly applies to the 00:27:44.100 --> 00:27:45.950 align:middle line:90% binomial distribution. 00:27:45.950 --> 00:27:49.440 align:middle line:84% To be more precise, of course, it applies to the standardized 00:27:49.440 --> 00:27:52.730 align:middle line:84% version of the binomial random variable. 00:27:52.730 --> 00:27:55.140 align:middle line:84% So here's what we did, essentially, in 00:27:55.140 --> 00:27:57.300 align:middle line:90% the previous example. 00:27:57.300 --> 00:28:00.690 align:middle line:84% We fixed the number p, which is the probability of success 00:28:00.690 --> 00:28:02.010 align:middle line:90% in our experiments. 00:28:02.010 --> 00:28:06.550 align:middle line:84% p corresponds to f in the previous example. 00:28:06.550 --> 00:28:10.570 align:middle line:84% Let every Xi a Bernoulli random variable and are 00:28:10.570 --> 00:28:13.790 align:middle line:84% standing assumption is that these random variables are 00:28:13.790 --> 00:28:15.040 align:middle line:90% independent. 00:28:15.040 --> 00:28:17.580 align:middle line:90% 00:28:17.580 --> 00:28:20.730 align:middle line:84% When we add them, we get a random variable that has a 00:28:20.730 --> 00:28:22.030 align:middle line:90% binomial distribution. 00:28:22.030 --> 00:28:25.220 align:middle line:84% We know the mean and the variance of the binomial, so 00:28:25.220 --> 00:28:29.130 align:middle line:84% we take Sn, we subtract the mean, which is this, divide by 00:28:29.130 --> 00:28:30.470 align:middle line:90% the standard deviation. 00:28:30.470 --> 00:28:32.790 align:middle line:84% The central limit theorem tells us that the cumulative 00:28:32.790 --> 00:28:36.130 align:middle line:84% distribution function of this random variable is a standard 00:28:36.130 --> 00:28:39.860 align:middle line:84% normal random variable in the limit. 00:28:39.860 --> 00:28:43.730 align:middle line:84% So let's do one more example of a calculation. 00:28:43.730 --> 00:28:47.160 align:middle line:90% Let's take n to be-- 00:28:47.160 --> 00:28:50.110 align:middle line:84% let's choose some specific numbers to work with. 00:28:50.110 --> 00:28:52.950 align:middle line:90% 00:28:52.950 --> 00:28:58.300 align:middle line:84% So in this example, first thing to do is to find the 00:28:58.300 --> 00:29:02.390 align:middle line:84% expected value of Sn, which is n times p. 00:29:02.390 --> 00:29:04.150 align:middle line:90% It's 18. 00:29:04.150 --> 00:29:08.100 align:middle line:84% Then we need to write down the standard deviation. 00:29:08.100 --> 00:29:12.430 align:middle line:90% 00:29:12.430 --> 00:29:16.530 align:middle line:84% The variance of Sn is the sum of the variances. 00:29:16.530 --> 00:29:19.940 align:middle line:90% It's np times (1-p). 00:29:19.940 --> 00:29:25.920 align:middle line:84% And in this particular example, p times (1-p) is 1/4, 00:29:25.920 --> 00:29:28.320 align:middle line:90% n is 36, so this is 9. 00:29:28.320 --> 00:29:33.120 align:middle line:84% And that tells us that the standard deviation of this n 00:29:33.120 --> 00:29:34.370 align:middle line:90% is equal to 3. 00:29:34.370 --> 00:29:37.170 align:middle line:90% 00:29:37.170 --> 00:29:40.650 align:middle line:84% So what we're going to do is to take the event of interest, 00:29:40.650 --> 00:29:46.400 align:middle line:84% which is Sn less than 21, and rewrite it in a way that 00:29:46.400 --> 00:29:48.910 align:middle line:84% involves the standardized random variable. 00:29:48.910 --> 00:29:51.700 align:middle line:84% So to do that, we need to subtract the mean. 00:29:51.700 --> 00:29:55.680 align:middle line:84% So we write this as Sn-3 should be less 00:29:55.680 --> 00:29:58.460 align:middle line:90% than or equal to 21-3. 00:29:58.460 --> 00:30:00.360 align:middle line:90% This is the same event. 00:30:00.360 --> 00:30:02.890 align:middle line:84% And then divide by the standard deviation, which is 00:30:02.890 --> 00:30:06.450 align:middle line:90% 3, and we end up with this. 00:30:06.450 --> 00:30:08.300 align:middle line:90% So the event itself of-- 00:30:08.300 --> 00:30:09.550 align:middle line:90% AUDIENCE: [INAUDIBLE]. 00:30:09.550 --> 00:30:13.700 align:middle line:90% 00:30:13.700 --> 00:30:24.150 align:middle line:84% Should subtract, 18, yes, which gives me a much nicer 00:30:24.150 --> 00:30:26.640 align:middle line:90% number out here, which is 1. 00:30:26.640 --> 00:30:31.650 align:middle line:84% So the event of interest, that Sn is less than 21, is the 00:30:31.650 --> 00:30:37.330 align:middle line:84% same as the event that a standard normal random 00:30:37.330 --> 00:30:41.580 align:middle line:84% variable is less than or equal to 1. 00:30:41.580 --> 00:30:44.690 align:middle line:84% And once more, you can look this up at the normal tables. 00:30:44.690 --> 00:30:50.690 align:middle line:84% And you find that the answer that you get is 0.43. 00:30:50.690 --> 00:30:53.390 align:middle line:84% Now it's interesting to compare this answer that we 00:30:53.390 --> 00:30:57.230 align:middle line:84% got through the central limit theorem with the exact answer. 00:30:57.230 --> 00:31:01.920 align:middle line:84% The exact answer involves the exact binomial distribution. 00:31:01.920 --> 00:31:08.780 align:middle line:84% What we have here is the binomial probability that, Sn 00:31:08.780 --> 00:31:10.970 align:middle line:90% is equal to k. 00:31:10.970 --> 00:31:15.230 align:middle line:84% Sn being equal to k is given by this formula. 00:31:15.230 --> 00:31:22.610 align:middle line:84% And we add, over all values for k going from 0 up to 21, 00:31:22.610 --> 00:31:28.670 align:middle line:84% we write a two lines code to calculate this sum, and we get 00:31:28.670 --> 00:31:32.530 align:middle line:84% the exact answer, which is 0.8785. 00:31:32.530 --> 00:31:35.760 align:middle line:84% So there's a pretty good agreements between the two, 00:31:35.760 --> 00:31:38.600 align:middle line:84% although you wouldn't call that's 00:31:38.600 --> 00:31:40.395 align:middle line:90% necessarily excellent agreement. 00:31:40.395 --> 00:31:45.080 align:middle line:90% 00:31:45.080 --> 00:31:47.060 align:middle line:84% Can we do a little better than that? 00:31:47.060 --> 00:31:51.570 align:middle line:90% 00:31:51.570 --> 00:31:53.750 align:middle line:90% OK. 00:31:53.750 --> 00:31:56.510 align:middle line:90% It turns out that we can. 00:31:56.510 --> 00:31:58.625 align:middle line:90% And here's the idea. 00:31:58.625 --> 00:32:02.300 align:middle line:90% 00:32:02.300 --> 00:32:07.750 align:middle line:84% So our random variable Sn has a mean of 18. 00:32:07.750 --> 00:32:09.540 align:middle line:84% It has a binomial distribution. 00:32:09.540 --> 00:32:14.050 align:middle line:84% It's described by a PMF that has a shape roughly like this 00:32:14.050 --> 00:32:16.690 align:middle line:90% and which keeps going on. 00:32:16.690 --> 00:32:20.960 align:middle line:84% Using the central limit theorem is basically 00:32:20.960 --> 00:32:26.650 align:middle line:84% pretending that Sn is normal with the 00:32:26.650 --> 00:32:28.650 align:middle line:90% right mean and variance. 00:32:28.650 --> 00:32:35.200 align:middle line:84% So pretending that Zn has 0 mean unit variance, we 00:32:35.200 --> 00:32:38.850 align:middle line:84% approximate it with Z, that has 0 mean unit variance. 00:32:38.850 --> 00:32:42.190 align:middle line:84% If you were to pretend that Sn is normal, you would 00:32:42.190 --> 00:32:45.407 align:middle line:84% approximate it with a normal that has the correct mean and 00:32:45.407 --> 00:32:46.250 align:middle line:90% correct variance. 00:32:46.250 --> 00:32:49.390 align:middle line:84% So it would still be centered at 18. 00:32:49.390 --> 00:32:53.800 align:middle line:84% And it would have the same variance as the binomial PMF. 00:32:53.800 --> 00:32:57.350 align:middle line:84% So using the central limit theorem essentially means that 00:32:57.350 --> 00:33:00.420 align:middle line:84% we keep the mean and the variance what they are but we 00:33:00.420 --> 00:33:03.960 align:middle line:84% pretend that our distribution is normal. 00:33:03.960 --> 00:33:06.780 align:middle line:84% We want to calculate the probability that Sn is less 00:33:06.780 --> 00:33:09.590 align:middle line:90% than or equal to 21. 00:33:09.590 --> 00:33:14.310 align:middle line:84% I pretend that my random variable is normal, so I draw 00:33:14.310 --> 00:33:18.680 align:middle line:84% a line here and I calculate the area under the normal 00:33:18.680 --> 00:33:22.000 align:middle line:90% curve going up to 21. 00:33:22.000 --> 00:33:23.500 align:middle line:84% That's essentially what we did. 00:33:23.500 --> 00:33:26.260 align:middle line:90% 00:33:26.260 --> 00:33:29.730 align:middle line:84% Now, a smart person comes around and says, Sn is a 00:33:29.730 --> 00:33:31.360 align:middle line:90% discrete random variable. 00:33:31.360 --> 00:33:34.750 align:middle line:84% So the event that Sn is less than or equal to 21 is the 00:33:34.750 --> 00:33:38.480 align:middle line:84% same as Sn being strictly less than 22 because nothing in 00:33:38.480 --> 00:33:41.240 align:middle line:90% between can happen. 00:33:41.240 --> 00:33:43.700 align:middle line:84% So I'm going to use the central limit theorem 00:33:43.700 --> 00:33:48.290 align:middle line:84% approximation by pretending again that Sn is normal and 00:33:48.290 --> 00:33:51.650 align:middle line:84% finding the probability of this event while pretending 00:33:51.650 --> 00:33:53.720 align:middle line:90% that Sn is normal. 00:33:53.720 --> 00:33:57.870 align:middle line:84% So what this person would do would be to draw a line here, 00:33:57.870 --> 00:34:02.780 align:middle line:84% at 22, and calculate the area under the normal curve 00:34:02.780 --> 00:34:05.490 align:middle line:90% all the way to 22. 00:34:05.490 --> 00:34:06.700 align:middle line:90% Who is right? 00:34:06.700 --> 00:34:08.820 align:middle line:90% Which one is better? 00:34:08.820 --> 00:34:15.639 align:middle line:84% Well neither, but we can do better than both if we sort of 00:34:15.639 --> 00:34:17.949 align:middle line:90% split the difference. 00:34:17.949 --> 00:34:21.969 align:middle line:84% So another way of writing the same event for Sn is to write 00:34:21.969 --> 00:34:25.940 align:middle line:90% it as Sn being less than 21.5. 00:34:25.940 --> 00:34:29.570 align:middle line:84% In terms of the discrete random variable Sn, all three 00:34:29.570 --> 00:34:32.239 align:middle line:84% of these are exactly the same event. 00:34:32.239 --> 00:34:35.090 align:middle line:84% But when you do the continuous approximation, they give you 00:34:35.090 --> 00:34:36.250 align:middle line:90% different probabilities. 00:34:36.250 --> 00:34:39.760 align:middle line:84% It's a matter of whether you integrate the area under the 00:34:39.760 --> 00:34:46.159 align:middle line:84% normal curve up to here, up to the midway point, or up to 22. 00:34:46.159 --> 00:34:50.659 align:middle line:84% It turns out that integrating up to the midpoint is what 00:34:50.659 --> 00:34:54.469 align:middle line:84% gives us the better numerical results. 00:34:54.469 --> 00:34:59.170 align:middle line:84% So we take here 21 and 1/2, and we integrate the area 00:34:59.170 --> 00:35:01.170 align:middle line:84% under the normal curve up to here. 00:35:01.170 --> 00:35:14.100 align:middle line:90% 00:35:14.100 --> 00:35:18.560 align:middle line:84% So let's do this calculation and see what we get. 00:35:18.560 --> 00:35:21.330 align:middle line:90% What would we change here? 00:35:21.330 --> 00:35:27.730 align:middle line:84% Instead of 21, we would now write 21 and 1/2. 00:35:27.730 --> 00:35:32.810 align:middle line:84% This 18 becomes, no, that 18 stays what it is. 00:35:32.810 --> 00:35:36.890 align:middle line:84% But this 21 becomes 21 and 1/2. 00:35:36.890 --> 00:35:44.790 align:middle line:84% And so this one becomes 1 + 0.5 by 3. 00:35:44.790 --> 00:35:48.210 align:middle line:90% This is 117. 00:35:48.210 --> 00:35:51.980 align:middle line:84% So we now look up into the normal tables and ask for the 00:35:51.980 --> 00:36:00.000 align:middle line:84% probability that Z is less than 1.17. 00:36:00.000 --> 00:36:06.070 align:middle line:84% So this here gets approximated by the probability that the 00:36:06.070 --> 00:36:09.240 align:middle line:84% standard normal is less than 1.17. 00:36:09.240 --> 00:36:15.960 align:middle line:84% And the normal tables will tell us this is 0.879. 00:36:15.960 --> 00:36:23.550 align:middle line:84% Going back to the previous slide, what we got this time 00:36:23.550 --> 00:36:30.310 align:middle line:84% with this improved approximation is 0.879. 00:36:30.310 --> 00:36:33.730 align:middle line:84% This is a really good approximation 00:36:33.730 --> 00:36:35.730 align:middle line:90% of the correct number. 00:36:35.730 --> 00:36:39.160 align:middle line:84% This is what we got using the 21. 00:36:39.160 --> 00:36:42.360 align:middle line:84% This is what we get using the 21 and 1/2. 00:36:42.360 --> 00:36:45.940 align:middle line:84% And it's an approximation that's sort of right on-- a 00:36:45.940 --> 00:36:48.350 align:middle line:90% very good one. 00:36:48.350 --> 00:36:54.120 align:middle line:84% The moral from this numerical example is that doing this 1 00:36:54.120 --> 00:37:00.933 align:middle line:84% and 1/2 correction does give us better approximations. 00:37:00.933 --> 00:37:06.070 align:middle line:90% 00:37:06.070 --> 00:37:12.010 align:middle line:84% In fact, we can use this 1/2 idea to even calculate 00:37:12.010 --> 00:37:14.340 align:middle line:90% individual probabilities. 00:37:14.340 --> 00:37:17.130 align:middle line:84% So suppose you want to approximate the probability 00:37:17.130 --> 00:37:21.010 align:middle line:90% that Sn equal to 19. 00:37:21.010 --> 00:37:25.620 align:middle line:84% If you were to pretend that Sn is normal and calculate this 00:37:25.620 --> 00:37:28.470 align:middle line:84% probability, the probability that the normal random 00:37:28.470 --> 00:37:31.670 align:middle line:90% variable is equal to 19 is 0. 00:37:31.670 --> 00:37:34.150 align:middle line:84% So you don't get an interesting answer. 00:37:34.150 --> 00:37:37.610 align:middle line:84% You get a more interesting answer by writing this event, 00:37:37.610 --> 00:37:41.460 align:middle line:84% 19 as being the same as the event of falling between 18 00:37:41.460 --> 00:37:45.910 align:middle line:84% and 1/2 and 19 and 1/2 and using the normal approximation 00:37:45.910 --> 00:37:48.230 align:middle line:90% to calculate this probability. 00:37:48.230 --> 00:37:51.890 align:middle line:84% In terms of our previous picture, this corresponds to 00:37:51.890 --> 00:37:53.140 align:middle line:90% the following. 00:37:53.140 --> 00:37:59.400 align:middle line:90% 00:37:59.400 --> 00:38:04.650 align:middle line:84% We are interested in the probability that 00:38:04.650 --> 00:38:07.130 align:middle line:90% Sn is equal to 19. 00:38:07.130 --> 00:38:11.230 align:middle line:84% So we're interested in the height of this bar. 00:38:11.230 --> 00:38:15.720 align:middle line:84% We're going to consider the area under the normal curve 00:38:15.720 --> 00:38:21.500 align:middle line:84% going from here to here, and use this area as an 00:38:21.500 --> 00:38:25.110 align:middle line:84% approximation for the height of that particular bar. 00:38:25.110 --> 00:38:30.670 align:middle line:84% So what we're basically doing is, we take the probability 00:38:30.670 --> 00:38:33.830 align:middle line:84% under the normal curve that's assigned over a continuum of 00:38:33.830 --> 00:38:38.280 align:middle line:84% values and attributed it to different discrete values. 00:38:38.280 --> 00:38:43.510 align:middle line:84% Whatever is above the midpoint gets attributed to 19. 00:38:43.510 --> 00:38:45.640 align:middle line:84% Whatever is below that midpoint gets 00:38:45.640 --> 00:38:47.250 align:middle line:90% attributed to 18. 00:38:47.250 --> 00:38:54.280 align:middle line:84% So this is green area is our approximation of the value of 00:38:54.280 --> 00:38:56.500 align:middle line:90% the PMF at 19. 00:38:56.500 --> 00:39:00.740 align:middle line:84% So similarly, if you wanted to approximate the value of the 00:39:00.740 --> 00:39:04.440 align:middle line:84% PMF at this point, you would take this interval and 00:39:04.440 --> 00:39:06.580 align:middle line:84% integrate the area under the normal 00:39:06.580 --> 00:39:09.350 align:middle line:90% curve over that interval. 00:39:09.350 --> 00:39:13.410 align:middle line:84% It turns out that this gives a very good approximation of the 00:39:13.410 --> 00:39:15.660 align:middle line:90% PMF of the binomial. 00:39:15.660 --> 00:39:22.580 align:middle line:84% And actually, this was the context in which the central 00:39:22.580 --> 00:39:26.310 align:middle line:84% limit theorem was proved in the first place, when this 00:39:26.310 --> 00:39:27.990 align:middle line:90% business started. 00:39:27.990 --> 00:39:33.060 align:middle line:84% So this business goes back a few hundred years. 00:39:33.060 --> 00:39:35.700 align:middle line:84% And the central limit theorem was first approved by 00:39:35.700 --> 00:39:39.420 align:middle line:84% considering the PMF of a binomial random variable when 00:39:39.420 --> 00:39:41.840 align:middle line:90% p is equal to 1/2. 00:39:41.840 --> 00:39:45.590 align:middle line:84% People did the algebra, and they found out that the exact 00:39:45.590 --> 00:39:49.700 align:middle line:84% expression for the PMF is quite well approximated by 00:39:49.700 --> 00:39:51.980 align:middle line:84% that expression hat you would get from a normal 00:39:51.980 --> 00:39:53.380 align:middle line:90% distribution. 00:39:53.380 --> 00:39:57.510 align:middle line:84% Then the proof was extended to binomials for more general 00:39:57.510 --> 00:39:59.690 align:middle line:90% values of p. 00:39:59.690 --> 00:40:04.220 align:middle line:84% So here we talk about this as a refinement of the general 00:40:04.220 --> 00:40:07.480 align:middle line:84% central limit theorem, but, historically, that refinement 00:40:07.480 --> 00:40:09.830 align:middle line:84% was where the whole business got started 00:40:09.830 --> 00:40:11.820 align:middle line:90% in the first place. 00:40:11.820 --> 00:40:18.700 align:middle line:84% All right, so let's go through the mechanics of approximating 00:40:18.700 --> 00:40:21.970 align:middle line:84% the probability that Sn is equal to 19-- 00:40:21.970 --> 00:40:23.810 align:middle line:90% exactly 19. 00:40:23.810 --> 00:40:27.340 align:middle line:84% As we said, we're going to write this event as an event 00:40:27.340 --> 00:40:31.040 align:middle line:84% that covers an interval of unit length from 18 and 1/2 to 00:40:31.040 --> 00:40:31.970 align:middle line:90% 19 and 1/2. 00:40:31.970 --> 00:40:33.730 align:middle line:90% This is the event of interest. 00:40:33.730 --> 00:40:37.070 align:middle line:84% First step is to massage the event of interest so that it 00:40:37.070 --> 00:40:40.010 align:middle line:84% involves our Zn random variable. 00:40:40.010 --> 00:40:43.290 align:middle line:90% So subtract 18 from all sides. 00:40:43.290 --> 00:40:46.860 align:middle line:84% Divide by the standard deviation of 3 from all sides. 00:40:46.860 --> 00:40:50.850 align:middle line:84% That's the equivalent representation of the event. 00:40:50.850 --> 00:40:54.200 align:middle line:84% This is our standardized random variable Zn. 00:40:54.200 --> 00:40:56.950 align:middle line:90% These are just these numbers. 00:40:56.950 --> 00:41:00.530 align:middle line:84% And to do an approximation, we want to find the probability 00:41:00.530 --> 00:41:04.380 align:middle line:84% of this event, but Zn is approximately normal, so we 00:41:04.380 --> 00:41:08.030 align:middle line:84% plug in here the Z, which is the standard normal. 00:41:08.030 --> 00:41:10.150 align:middle line:84% So we want to find the probability that the standard 00:41:10.150 --> 00:41:12.890 align:middle line:84% normal falls inside this interval. 00:41:12.890 --> 00:41:15.630 align:middle line:84% You find these using CDFs because this is the 00:41:15.630 --> 00:41:18.760 align:middle line:84% probability that you're less than this but 00:41:18.760 --> 00:41:22.370 align:middle line:90% not less than that. 00:41:22.370 --> 00:41:25.370 align:middle line:84% So it's a difference between two cumulative probabilities. 00:41:25.370 --> 00:41:27.400 align:middle line:84% Then, you look up your normal tables. 00:41:27.400 --> 00:41:30.560 align:middle line:84% You find two numbers for these quantities, and, finally, you 00:41:30.560 --> 00:41:35.140 align:middle line:84% get a numerical answer for an individual entry of the PMF of 00:41:35.140 --> 00:41:36.480 align:middle line:90% the binomial. 00:41:36.480 --> 00:41:39.350 align:middle line:84% This is a pretty good approximation, it turns out. 00:41:39.350 --> 00:41:42.910 align:middle line:84% If you were to do the calculations using the exact 00:41:42.910 --> 00:41:47.130 align:middle line:84% formula, you would get something 00:41:47.130 --> 00:41:49.360 align:middle line:90% which is pretty close-- 00:41:49.360 --> 00:41:52.800 align:middle line:90% an error in the third digit-- 00:41:52.800 --> 00:41:56.980 align:middle line:90% this is pretty good. 00:41:56.980 --> 00:41:59.650 align:middle line:84% So I guess what we did here with our discussion of the 00:41:59.650 --> 00:42:04.560 align:middle line:84% binomial slightly contradicts what I said before-- 00:42:04.560 --> 00:42:07.330 align:middle line:84% that the central limit theorem is a statement about 00:42:07.330 --> 00:42:09.240 align:middle line:84% cumulative distribution functions. 00:42:09.240 --> 00:42:13.240 align:middle line:84% In general, it doesn't tell you what to do to approximate 00:42:13.240 --> 00:42:15.270 align:middle line:90% PMFs themselves. 00:42:15.270 --> 00:42:17.440 align:middle line:84% And that's indeed the case in general. 00:42:17.440 --> 00:42:20.220 align:middle line:84% One the other hand, for the special case of a binomial 00:42:20.220 --> 00:42:23.610 align:middle line:84% distribution, the central limit theorem approximation, 00:42:23.610 --> 00:42:28.200 align:middle line:84% with this 1/2 correction, is a very good approximation even 00:42:28.200 --> 00:42:29.560 align:middle line:90% for the individual PMF. 00:42:29.560 --> 00:42:33.290 align:middle line:90% 00:42:33.290 --> 00:42:40.210 align:middle line:84% All right, so we spent quite a bit of time on mechanics. 00:42:40.210 --> 00:42:46.050 align:middle line:84% So let's spend the last few minutes today thinking a bit 00:42:46.050 --> 00:42:47.930 align:middle line:90% and look at a small puzzle. 00:42:47.930 --> 00:42:51.390 align:middle line:90% 00:42:51.390 --> 00:42:54.240 align:middle line:84% So the puzzle is the following. 00:42:54.240 --> 00:43:02.460 align:middle line:84% Consider Poisson process that runs over a unit interval. 00:43:02.460 --> 00:43:07.770 align:middle line:84% And where the arrival rate is equal to 1. 00:43:07.770 --> 00:43:09.790 align:middle line:90% So this is the unit interval. 00:43:09.790 --> 00:43:12.720 align:middle line:84% And let X be the number of arrivals. 00:43:12.720 --> 00:43:15.430 align:middle line:90% 00:43:15.430 --> 00:43:19.930 align:middle line:84% And this is Poisson, with mean 1. 00:43:19.930 --> 00:43:25.000 align:middle line:90% 00:43:25.000 --> 00:43:28.160 align:middle line:84% Now, let me take this interval and divide it 00:43:28.160 --> 00:43:30.650 align:middle line:90% into n little pieces. 00:43:30.650 --> 00:43:34.270 align:middle line:90% So each piece has length 1/n. 00:43:34.270 --> 00:43:41.225 align:middle line:84% And let Xi be the number of arrivals during 00:43:41.225 --> 00:43:43.490 align:middle line:90% the Ith little interval. 00:43:43.490 --> 00:43:48.000 align:middle line:90% 00:43:48.000 --> 00:43:51.630 align:middle line:84% OK, what do we know about the random variables Xi? 00:43:51.630 --> 00:43:55.260 align:middle line:84% Is they are themselves Poisson. 00:43:55.260 --> 00:43:58.490 align:middle line:84% It's a number of arrivals during a small interval. 00:43:58.490 --> 00:44:02.340 align:middle line:84% We also know that when n is big, so the length of the 00:44:02.340 --> 00:44:08.190 align:middle line:84% interval is small, these Xi's are approximately Bernoulli, 00:44:08.190 --> 00:44:11.730 align:middle line:90% with mean 1/n. 00:44:11.730 --> 00:44:13.970 align:middle line:84% Guess it doesn't matter whether we model them as 00:44:13.970 --> 00:44:15.720 align:middle line:90% Bernoulli or not. 00:44:15.720 --> 00:44:19.660 align:middle line:84% What matters is that the Xi's are independent. 00:44:19.660 --> 00:44:20.970 align:middle line:90% Why are they independent? 00:44:20.970 --> 00:44:24.410 align:middle line:84% Because, in a Poisson process, these joint intervals are 00:44:24.410 --> 00:44:26.770 align:middle line:90% independent of each other. 00:44:26.770 --> 00:44:28.955 align:middle line:90% So the Xi's are independent. 00:44:28.955 --> 00:44:31.840 align:middle line:90% 00:44:31.840 --> 00:44:35.570 align:middle line:84% And they also have the same distribution. 00:44:35.570 --> 00:44:40.360 align:middle line:84% And we have that X, the total number of arrivals, is the sum 00:44:40.360 --> 00:44:41.610 align:middle line:90% over the Xn's. 00:44:41.610 --> 00:44:44.470 align:middle line:90% 00:44:44.470 --> 00:44:49.510 align:middle line:84% So the central limit theorem tells us that, approximately, 00:44:49.510 --> 00:44:53.670 align:middle line:84% the sum of independent, identically distributed random 00:44:53.670 --> 00:44:57.720 align:middle line:84% variables, when we have lots of these random variables, 00:44:57.720 --> 00:45:01.530 align:middle line:84% behaves like a normal random variable. 00:45:01.530 --> 00:45:07.475 align:middle line:84% So by using this decomposition of X into a sum of i.i.d 00:45:07.475 --> 00:45:11.540 align:middle line:84% random variables, and by using values of n that are bigger 00:45:11.540 --> 00:45:16.540 align:middle line:84% and bigger, by taking the limit, it should follow that X 00:45:16.540 --> 00:45:19.510 align:middle line:90% has a normal distribution. 00:45:19.510 --> 00:45:22.120 align:middle line:84% On the other hand, we know that X has a Poisson 00:45:22.120 --> 00:45:23.370 align:middle line:90% distribution. 00:45:23.370 --> 00:45:25.270 align:middle line:90% 00:45:25.270 --> 00:45:32.640 align:middle line:84% So something must be wrong in this argument here. 00:45:32.640 --> 00:45:34.900 align:middle line:84% Can we really use the central limit 00:45:34.900 --> 00:45:38.330 align:middle line:90% theorem in this situation? 00:45:38.330 --> 00:45:41.300 align:middle line:84% So what do we need for the central limit theorem? 00:45:41.300 --> 00:45:44.160 align:middle line:84% We need to have independent, identically 00:45:44.160 --> 00:45:46.700 align:middle line:90% distributed random variables. 00:45:46.700 --> 00:45:49.060 align:middle line:90% We have it here. 00:45:49.060 --> 00:45:53.410 align:middle line:84% We want them to have a finite mean and finite variance. 00:45:53.410 --> 00:45:57.610 align:middle line:84% We also have it here, means variances are finite. 00:45:57.610 --> 00:46:02.050 align:middle line:84% What is another assumption that was never made explicit, 00:46:02.050 --> 00:46:04.080 align:middle line:90% but essentially was there? 00:46:04.080 --> 00:46:07.680 align:middle line:90% 00:46:07.680 --> 00:46:13.260 align:middle line:84% Or in other words, what is the flaw in this argument that 00:46:13.260 --> 00:46:15.520 align:middle line:84% uses the central limit theorem here? 00:46:15.520 --> 00:46:16.770 align:middle line:90% Any thoughts? 00:46:16.770 --> 00:46:24.110 align:middle line:90% 00:46:24.110 --> 00:46:29.640 align:middle line:84% So in the central limit theorem, we said, consider-- 00:46:29.640 --> 00:46:34.820 align:middle line:84% fix a probability distribution, and let the Xi's 00:46:34.820 --> 00:46:38.280 align:middle line:84% be distributed according to that probability distribution, 00:46:38.280 --> 00:46:42.935 align:middle line:84% and add a larger and larger number or Xi's. 00:46:42.935 --> 00:46:47.410 align:middle line:84% But the underlying, unstated assumption is that we fix the 00:46:47.410 --> 00:46:49.490 align:middle line:90% distribution of the Xi's. 00:46:49.490 --> 00:46:52.810 align:middle line:84% As we let n increase, the statistics of 00:46:52.810 --> 00:46:55.930 align:middle line:90% each Xi do not change. 00:46:55.930 --> 00:46:59.010 align:middle line:84% Whereas here, I'm playing a trick on you. 00:46:59.010 --> 00:47:03.700 align:middle line:84% As I'm taking more and more random variables, I'm actually 00:47:03.700 --> 00:47:07.850 align:middle line:84% changing what those random variables are. 00:47:07.850 --> 00:47:12.960 align:middle line:84% When I take a larger n, the Xi's are random variables with 00:47:12.960 --> 00:47:15.720 align:middle line:84% a different mean and different variance. 00:47:15.720 --> 00:47:19.800 align:middle line:84% So I'm adding more of these, but at the same time, in this 00:47:19.800 --> 00:47:23.420 align:middle line:84% example, I'm changing their distributions. 00:47:23.420 --> 00:47:26.380 align:middle line:84% That's something that doesn't fit the setting of the central 00:47:26.380 --> 00:47:27.000 align:middle line:90% limit theorem. 00:47:27.000 --> 00:47:29.910 align:middle line:84% In the central limit theorem, you first fix the distribution 00:47:29.910 --> 00:47:31.200 align:middle line:90% of the X's. 00:47:31.200 --> 00:47:35.290 align:middle line:84% You keep it fixed, and then you consider adding more and 00:47:35.290 --> 00:47:38.950 align:middle line:84% more according to that particular fixed distribution. 00:47:38.950 --> 00:47:40.020 align:middle line:90% So that's the catch. 00:47:40.020 --> 00:47:42.240 align:middle line:84% That's why the central limit theorem does not 00:47:42.240 --> 00:47:43.970 align:middle line:90% apply to this situation. 00:47:43.970 --> 00:47:46.230 align:middle line:84% And we're lucky that it doesn't apply because, 00:47:46.230 --> 00:47:50.220 align:middle line:84% otherwise, we would have a huge contradiction destroying 00:47:50.220 --> 00:47:52.770 align:middle line:90% probability theory. 00:47:52.770 --> 00:48:02.240 align:middle line:84% OK, but now that's still leaves us with a 00:48:02.240 --> 00:48:05.040 align:middle line:90% little bit of a dilemma. 00:48:05.040 --> 00:48:08.510 align:middle line:84% Suppose that, here, essentially we're adding 00:48:08.510 --> 00:48:12.815 align:middle line:84% independent Bernoulli random variables. 00:48:12.815 --> 00:48:22.650 align:middle line:90% 00:48:22.650 --> 00:48:25.300 align:middle line:84% So the issue is that the central limit theorem has to 00:48:25.300 --> 00:48:28.920 align:middle line:84% do with asymptotics as n goes to infinity. 00:48:28.920 --> 00:48:34.260 align:middle line:84% And if we consider a binomial, and somebody gives us specific 00:48:34.260 --> 00:48:38.870 align:middle line:84% numbers about the parameters of that binomial, it might not 00:48:38.870 --> 00:48:40.830 align:middle line:84% necessarily be obvious what kind of 00:48:40.830 --> 00:48:42.790 align:middle line:90% approximation do we use. 00:48:42.790 --> 00:48:45.660 align:middle line:84% In particular, we do have two different approximations for 00:48:45.660 --> 00:48:47.100 align:middle line:90% the binomial. 00:48:47.100 --> 00:48:51.610 align:middle line:84% If we fix p, then the binomial is the sum of Bernoulli's that 00:48:51.610 --> 00:48:54.930 align:middle line:84% come from a fixed distribution, we consider more 00:48:54.930 --> 00:48:56.450 align:middle line:90% and more of these. 00:48:56.450 --> 00:48:58.990 align:middle line:84% When we add them, the central limit theorem tells us that we 00:48:58.990 --> 00:49:01.190 align:middle line:90% get the normal distribution. 00:49:01.190 --> 00:49:04.430 align:middle line:84% There's another sort of limit, which has the flavor of this 00:49:04.430 --> 00:49:10.770 align:middle line:84% example, in which we still deal with a binomial, sum of n 00:49:10.770 --> 00:49:11.170 align:middle line:90% Bernoulli's. 00:49:11.170 --> 00:49:14.310 align:middle line:84% We let that sum, the number of the 00:49:14.310 --> 00:49:16.090 align:middle line:90% Bernoulli's go to infinity. 00:49:16.090 --> 00:49:18.890 align:middle line:84% But each Bernoulli has a probability of success that 00:49:18.890 --> 00:49:23.830 align:middle line:84% goes to 0, and we do this in a way so that np, the expected 00:49:23.830 --> 00:49:27.090 align:middle line:84% number of successes, stays finite. 00:49:27.090 --> 00:49:30.660 align:middle line:84% This is the situation that we dealt with when we first 00:49:30.660 --> 00:49:32.960 align:middle line:90% defined our Poisson process. 00:49:32.960 --> 00:49:37.540 align:middle line:84% We have a very, very large number so lots, of time slots, 00:49:37.540 --> 00:49:40.920 align:middle line:84% but during each time slot, there's a tiny probability of 00:49:40.920 --> 00:49:42.950 align:middle line:90% obtaining an arrival. 00:49:42.950 --> 00:49:48.460 align:middle line:84% Under that setting, in discrete time, we have a 00:49:48.460 --> 00:49:51.670 align:middle line:84% binomial distribution, or Bernoulli process, but when we 00:49:51.670 --> 00:49:54.530 align:middle line:84% take the limit, we obtain the Poisson process and the 00:49:54.530 --> 00:49:56.470 align:middle line:90% Poisson approximation. 00:49:56.470 --> 00:49:58.510 align:middle line:90% So these are two equally valid 00:49:58.510 --> 00:50:00.550 align:middle line:90% approximations of the binomial. 00:50:00.550 --> 00:50:03.300 align:middle line:84% But they're valid in different asymptotic regimes. 00:50:03.300 --> 00:50:06.180 align:middle line:84% In one regime, we fixed p, let n go to infinity. 00:50:06.180 --> 00:50:09.360 align:middle line:84% In the other regime, we let both n and p change 00:50:09.360 --> 00:50:11.540 align:middle line:90% simultaneously. 00:50:11.540 --> 00:50:14.240 align:middle line:84% Now, in real life, you're never dealing with the 00:50:14.240 --> 00:50:15.290 align:middle line:90% limiting situations. 00:50:15.290 --> 00:50:17.870 align:middle line:84% You're dealing with actual numbers. 00:50:17.870 --> 00:50:21.820 align:middle line:84% So if somebody tells you that the numbers are like this, 00:50:21.820 --> 00:50:25.160 align:middle line:84% then you should probably say that this is the situation 00:50:25.160 --> 00:50:27.380 align:middle line:84% that fits the Poisson description-- 00:50:27.380 --> 00:50:30.180 align:middle line:84% large number of slots with each slot having a tiny 00:50:30.180 --> 00:50:32.460 align:middle line:90% probability of success. 00:50:32.460 --> 00:50:36.890 align:middle line:84% On the other hand, if p is something like this, and n is 00:50:36.890 --> 00:50:40.460 align:middle line:84% 500, then you expect to get the distribution for the 00:50:40.460 --> 00:50:41.680 align:middle line:90% number of successes. 00:50:41.680 --> 00:50:45.740 align:middle line:84% It's going to have a mean of 50 and to have a fair amount 00:50:45.740 --> 00:50:47.280 align:middle line:90% of spread around there. 00:50:47.280 --> 00:50:50.150 align:middle line:84% It turns out that the normal approximation would be better 00:50:50.150 --> 00:50:51.500 align:middle line:90% in this context. 00:50:51.500 --> 00:50:57.120 align:middle line:84% As a rule of thumb, if n times p is bigger than 10 or 20, you 00:50:57.120 --> 00:50:59.320 align:middle line:84% can start using the normal approximation. 00:50:59.320 --> 00:51:04.310 align:middle line:84% If n times p is a small number, then you prefer to use 00:51:04.310 --> 00:51:06.090 align:middle line:90% the Poisson approximation. 00:51:06.090 --> 00:51:08.840 align:middle line:84% But there's no hard theorems or rules about 00:51:08.840 --> 00:51:11.650 align:middle line:90% how to go about this. 00:51:11.650 --> 00:51:15.440 align:middle line:84% OK, so from next time we're going to switch base again. 00:51:15.440 --> 00:51:17.830 align:middle line:84% And we're going to put together everything we learned 00:51:17.830 --> 00:51:20.620 align:middle line:84% in this class to start solving inference problems. 00:51:20.620 --> 00:51:22.050 align:middle line:90%