WEBVTT 00:00:00.000 --> 00:00:00.040 align:middle line:90% 00:00:00.040 --> 00:00:02.460 align:middle line:84% The following content is provided under a Creative 00:00:02.460 --> 00:00:03.870 align:middle line:90% Commons license. 00:00:03.870 --> 00:00:06.910 align:middle line:84% Your support will help MIT OpenCourseWare continue to 00:00:06.910 --> 00:00:10.560 align:middle line:84% offer high quality educational resources for free. 00:00:10.560 --> 00:00:13.460 align:middle line:84% To make a donation or view additional materials from 00:00:13.460 --> 00:00:17.390 align:middle line:84% hundreds of MIT courses, visit MIT OpenCourseWare at 00:00:17.390 --> 00:00:18.640 align:middle line:90% ocw.mit.edu. 00:00:18.640 --> 00:00:22.790 align:middle line:90% 00:00:22.790 --> 00:00:24.630 align:middle line:84% PROFESSOR: So last time we started talking 00:00:24.630 --> 00:00:26.420 align:middle line:90% about random processes. 00:00:26.420 --> 00:00:30.710 align:middle line:84% A random process is a random experiment that 00:00:30.710 --> 00:00:32.570 align:middle line:90% evolves over time. 00:00:32.570 --> 00:00:35.110 align:middle line:84% And conceptually, it's important to realize that it's 00:00:35.110 --> 00:00:37.760 align:middle line:84% a single probabilistic experiment 00:00:37.760 --> 00:00:39.250 align:middle line:90% that has many stages. 00:00:39.250 --> 00:00:42.440 align:middle line:84% Actually, it has an infinite number of stages. 00:00:42.440 --> 00:00:47.060 align:middle line:84% And we discussed the simplest random process there is, the 00:00:47.060 --> 00:00:50.270 align:middle line:84% Bernoulli process, which is nothing but the sequence of 00:00:50.270 --> 00:00:51.570 align:middle line:90% Bernoulli trials-- 00:00:51.570 --> 00:00:54.580 align:middle line:84% an infinite sequence of Bernoulli trials. 00:00:54.580 --> 00:00:58.250 align:middle line:84% For example, flipping a coin over and over. 00:00:58.250 --> 00:01:01.640 align:middle line:84% Once we understand what's going on with that process, 00:01:01.640 --> 00:01:05.519 align:middle line:84% then what we want is to move into a continuous time version 00:01:05.519 --> 00:01:06.750 align:middle line:90% of the Bernoulli process. 00:01:06.750 --> 00:01:09.420 align:middle line:84% And this is what we will call the Poisson process. 00:01:09.420 --> 00:01:11.970 align:middle line:84% And for the Poisson process, we're going to do exactly the 00:01:11.970 --> 00:01:14.300 align:middle line:84% same things that we did for the Bernoulli process. 00:01:14.300 --> 00:01:18.210 align:middle line:84% That is, talk about the number of arrivals during a given 00:01:18.210 --> 00:01:21.400 align:middle line:84% time period, and talk also about the time between 00:01:21.400 --> 00:01:24.160 align:middle line:84% consecutive arrivals, and for the distribution of 00:01:24.160 --> 00:01:27.660 align:middle line:90% inter-arrival times. 00:01:27.660 --> 00:01:30.660 align:middle line:84% So let's start with a quick review of what we 00:01:30.660 --> 00:01:32.680 align:middle line:90% discussed last time. 00:01:32.680 --> 00:01:35.500 align:middle line:90% First, a note about language. 00:01:35.500 --> 00:01:38.100 align:middle line:84% If you think of coin tosses, we then talk 00:01:38.100 --> 00:01:40.660 align:middle line:90% about heads and tails. 00:01:40.660 --> 00:01:43.430 align:middle line:84% If you think of these as a sequence of trials, you can 00:01:43.430 --> 00:01:47.145 align:middle line:84% talk about successes and failures. 00:01:47.145 --> 00:01:50.312 align:middle line:84% The language that we will be using will be more the 00:01:50.312 --> 00:01:51.800 align:middle line:90% language of arrivals. 00:01:51.800 --> 00:01:56.020 align:middle line:84% That is, if in a given slot you have a success, you say 00:01:56.020 --> 00:01:57.930 align:middle line:90% that something arrived. 00:01:57.930 --> 00:02:00.470 align:middle line:84% If you have a failure, nothing arrived. 00:02:00.470 --> 00:02:03.090 align:middle line:84% And that language is a little more convenient and more 00:02:03.090 --> 00:02:06.820 align:middle line:84% natural, especially when we talk about continuous time-- 00:02:06.820 --> 00:02:10.250 align:middle line:84% to talk about arrivals instead of successes. 00:02:10.250 --> 00:02:12.640 align:middle line:84% But in any case, for the Bernoulli process let's keep, 00:02:12.640 --> 00:02:14.990 align:middle line:84% for a little bit, the language of successes. 00:02:14.990 --> 00:02:18.530 align:middle line:84% Whereas working in discrete time, we have time slots. 00:02:18.530 --> 00:02:20.870 align:middle line:84% During each time slot, we have an 00:02:20.870 --> 00:02:22.810 align:middle line:90% independent Bernoulli trial. 00:02:22.810 --> 00:02:25.750 align:middle line:84% There is probability p of having a success. 00:02:25.750 --> 00:02:29.400 align:middle line:84% Different slots are independent of each other. 00:02:29.400 --> 00:02:33.190 align:middle line:84% And this probability p is the same for any given time slot. 00:02:33.190 --> 00:02:36.540 align:middle line:84% So for this process we will discuss the one random 00:02:36.540 --> 00:02:39.240 align:middle line:84% variable of interest, which is the following. 00:02:39.240 --> 00:02:43.060 align:middle line:84% If we have n time slots, or n trials, how many 00:02:43.060 --> 00:02:44.600 align:middle line:90% arrivals will there be? 00:02:44.600 --> 00:02:46.990 align:middle line:84% Or how many successes will there be? 00:02:46.990 --> 00:02:50.910 align:middle line:84% Well, this is just given by the binomial PMF. 00:02:50.910 --> 00:02:54.780 align:middle line:84% Number of successes in n trials is a random variable 00:02:54.780 --> 00:02:58.420 align:middle line:84% that has a binomial PMF, and we know what this is. 00:02:58.420 --> 00:03:01.150 align:middle line:84% Then we talked about inter-arrival times. 00:03:01.150 --> 00:03:04.490 align:middle line:84% The time until the first arrival happens has a 00:03:04.490 --> 00:03:07.890 align:middle line:90% geometric distribution. 00:03:07.890 --> 00:03:11.770 align:middle line:84% And we have seen that from some time ago. 00:03:11.770 --> 00:03:14.890 align:middle line:84% Now if you start thinking about the time until k 00:03:14.890 --> 00:03:20.040 align:middle line:84% arrivals happen, and we denote that by Yk, this is the time 00:03:20.040 --> 00:03:22.190 align:middle line:84% until the first arrival happens. 00:03:22.190 --> 00:03:24.860 align:middle line:84% And then after the first arrival happens, you have to 00:03:24.860 --> 00:03:27.000 align:middle line:84% wait some time until the second arrival 00:03:27.000 --> 00:03:28.730 align:middle line:90% happens, and so on. 00:03:28.730 --> 00:03:32.650 align:middle line:84% And then the time from the (k -1)th arrival, until 00:03:32.650 --> 00:03:34.820 align:middle line:90% arrival number k. 00:03:34.820 --> 00:03:37.680 align:middle line:84% The important thing to realize here is that because the 00:03:37.680 --> 00:03:41.830 align:middle line:84% process has a memorylessness property, once the first 00:03:41.830 --> 00:03:45.330 align:middle line:84% arrival comes, it's as if we're starting from scratch 00:03:45.330 --> 00:03:48.030 align:middle line:84% and we will be flipping our coins until the 00:03:48.030 --> 00:03:49.560 align:middle line:90% next arrival comes. 00:03:49.560 --> 00:03:52.350 align:middle line:84% So the time it will take until the next arrival comes will 00:03:52.350 --> 00:03:54.810 align:middle line:84% also be a geometric random variable. 00:03:54.810 --> 00:03:57.790 align:middle line:84% And because different slots are independent, whatever 00:03:57.790 --> 00:04:00.580 align:middle line:84% happens after the first arrival is independent from 00:04:00.580 --> 00:04:02.640 align:middle line:90% whatever happened before. 00:04:02.640 --> 00:04:06.250 align:middle line:84% So T1 and T2 will be independent random variables. 00:04:06.250 --> 00:04:08.940 align:middle line:84% And similarly, all the way up to Tk. 00:04:08.940 --> 00:04:13.460 align:middle line:84% So the time until the k-th arrival is a sum of 00:04:13.460 --> 00:04:17.640 align:middle line:84% independent geometric random variables, with the same 00:04:17.640 --> 00:04:19.459 align:middle line:90% parameter p. 00:04:19.459 --> 00:04:23.010 align:middle line:84% And we saw last time that we can find the probability 00:04:23.010 --> 00:04:25.850 align:middle line:90% distribution of Yk. 00:04:25.850 --> 00:04:30.880 align:middle line:84% The probability that Yk takes a value of t is equal to-- 00:04:30.880 --> 00:04:36.110 align:middle line:84% there's this combinatorial factor here, and then you get 00:04:36.110 --> 00:04:41.590 align:middle line:84% p to the k, (1-p) to the (t-k), and this formula is 00:04:41.590 --> 00:04:48.020 align:middle line:84% true for t equal to k, k+1, and so on. 00:04:48.020 --> 00:04:49.950 align:middle line:84% And this distribution has a name. 00:04:49.950 --> 00:04:51.660 align:middle line:90% It's called the Pascal PMF. 00:04:51.660 --> 00:04:54.500 align:middle line:90% 00:04:54.500 --> 00:04:57.080 align:middle line:84% So this is all there is to know about 00:04:57.080 --> 00:04:59.130 align:middle line:90% the Bernoulli process. 00:04:59.130 --> 00:05:02.880 align:middle line:84% One important comment is to realize what exactly this 00:05:02.880 --> 00:05:05.540 align:middle line:84% memorylessness property is saying. 00:05:05.540 --> 00:05:07.450 align:middle line:84% So I discussed it a little bit last time. 00:05:07.450 --> 00:05:10.490 align:middle line:90% Let me reiterate it. 00:05:10.490 --> 00:05:13.460 align:middle line:84% So we have a Bernoulli process, which is a sequence 00:05:13.460 --> 00:05:15.230 align:middle line:90% of Bernoulli trials. 00:05:15.230 --> 00:05:17.990 align:middle line:84% And these are (0,1) random variables that 00:05:17.990 --> 00:05:20.320 align:middle line:90% keep going on forever. 00:05:20.320 --> 00:05:27.400 align:middle line:84% So someone is watching this movie of Bernoulli trials B_t. 00:05:27.400 --> 00:05:31.540 align:middle line:84% And at some point, they say they think, or something 00:05:31.540 --> 00:05:33.990 align:middle line:84% interesting has happened, why don't you come 00:05:33.990 --> 00:05:36.300 align:middle line:90% in and start watching? 00:05:36.300 --> 00:05:39.760 align:middle line:84% So at some time t, they tell you to come 00:05:39.760 --> 00:05:41.300 align:middle line:90% in and start watching. 00:05:41.300 --> 00:05:44.780 align:middle line:84% So what you will see once you come in will 00:05:44.780 --> 00:05:48.320 align:middle line:90% be this future trials. 00:05:48.320 --> 00:05:52.610 align:middle line:84% So actually what you will see is a random process, whose 00:05:52.610 --> 00:05:57.170 align:middle line:84% first random variable is going to be the first one that you 00:05:57.170 --> 00:05:59.340 align:middle line:90% see, B_(t +1). 00:05:59.340 --> 00:06:03.040 align:middle line:84% The second one is going to be this, and so on. 00:06:03.040 --> 00:06:06.850 align:middle line:84% So this is the process that's seen by the person who's asked 00:06:06.850 --> 00:06:10.360 align:middle line:84% to come in and start watching at that time. 00:06:10.360 --> 00:06:15.220 align:middle line:84% And the claim is that this process is itself a Bernoulli 00:06:15.220 --> 00:06:20.380 align:middle line:84% process, provided that the person who calls you into the 00:06:20.380 --> 00:06:23.740 align:middle line:84% room does not look into the future. 00:06:23.740 --> 00:06:27.100 align:middle line:84% The person who calls you into the room decides to call you 00:06:27.100 --> 00:06:31.180 align:middle line:84% in only on the basis of what they have seen so far. 00:06:31.180 --> 00:06:33.860 align:middle line:84% So for example, who calls you into the room might have a 00:06:33.860 --> 00:06:39.600 align:middle line:84% rule that says, as soon as I see a sequence of 3 heads, I 00:06:39.600 --> 00:06:43.730 align:middle line:84% ask the other person to come in. 00:06:43.730 --> 00:06:46.390 align:middle line:84% So if they use that particular rule, it means that when 00:06:46.390 --> 00:06:49.900 align:middle line:84% you're called in, the previous 3 were heads. 00:06:49.900 --> 00:06:53.190 align:middle line:84% But this doesn't give you any information about the future. 00:06:53.190 --> 00:06:55.260 align:middle line:84% And so the future ones will be just 00:06:55.260 --> 00:06:57.160 align:middle line:90% independent Bernoulli trials. 00:06:57.160 --> 00:07:00.370 align:middle line:84% If on the other hand, the person who calls you in has 00:07:00.370 --> 00:07:04.180 align:middle line:84% seen the movie before and they use a rule, such as, for 00:07:04.180 --> 00:07:09.710 align:middle line:84% example, I call you in just before 3 heads show up for the 00:07:09.710 --> 00:07:10.820 align:middle line:90% first time. 00:07:10.820 --> 00:07:13.850 align:middle line:84% So the person calls you in based on knowledge that these 00:07:13.850 --> 00:07:15.430 align:middle line:90% two would be three heads. 00:07:15.430 --> 00:07:17.490 align:middle line:90% If they have such foresight-- 00:07:17.490 --> 00:07:19.820 align:middle line:84% if they can look into the future-- 00:07:19.820 --> 00:07:25.050 align:middle line:84% then X1, X2, X3, they're certain to be three heads, so 00:07:25.050 --> 00:07:27.190 align:middle line:84% they do not correspond to random 00:07:27.190 --> 00:07:29.680 align:middle line:90% independent Bernoulli trials. 00:07:29.680 --> 00:07:33.660 align:middle line:84% So to rephrase this, the process is memoryless. 00:07:33.660 --> 00:07:38.460 align:middle line:84% It does not matter what has happened in the past. 00:07:38.460 --> 00:07:42.090 align:middle line:84% And that's true even if you are called into the room and 00:07:42.090 --> 00:07:45.700 align:middle line:84% start watching at a random time, as long as that random 00:07:45.700 --> 00:07:50.950 align:middle line:84% time is determined in a causal way on the basis of what has 00:07:50.950 --> 00:07:52.340 align:middle line:90% happened so far. 00:07:52.340 --> 00:07:55.660 align:middle line:84% So you are called into the room in a causal manner, just 00:07:55.660 --> 00:07:57.630 align:middle line:84% based on what's happened so far. 00:07:57.630 --> 00:08:00.030 align:middle line:84% What you're going to see starting from that time will 00:08:00.030 --> 00:08:03.240 align:middle line:84% still be a sequence of independent Bernoulli trials. 00:08:03.240 --> 00:08:06.560 align:middle line:84% And this is the argument that we used here, essentially, to 00:08:06.560 --> 00:08:09.190 align:middle line:84% argue that this T2 is an independent random 00:08:09.190 --> 00:08:11.030 align:middle line:90% variable from T1. 00:08:11.030 --> 00:08:14.530 align:middle line:84% So a person is watching the movie, sees the first success. 00:08:14.530 --> 00:08:17.260 align:middle line:90% 00:08:17.260 --> 00:08:19.810 align:middle line:84% And on the basis of what they have seen-- 00:08:19.810 --> 00:08:21.620 align:middle line:84% they have just seen the first success-- 00:08:21.620 --> 00:08:23.590 align:middle line:90% they ask you to come in. 00:08:23.590 --> 00:08:24.350 align:middle line:90% You come in. 00:08:24.350 --> 00:08:27.260 align:middle line:84% What you're going to see is a sequence of Bernoulli trials. 00:08:27.260 --> 00:08:32.260 align:middle line:84% And you wait this long until the next success comes in. 00:08:32.260 --> 00:08:35.390 align:middle line:84% What you see is a Bernoulli process, as if the process was 00:08:35.390 --> 00:08:37.299 align:middle line:90% just starting right now. 00:08:37.299 --> 00:08:40.830 align:middle line:84% And that convinces us that this should be a geometric 00:08:40.830 --> 00:08:43.200 align:middle line:84% random variable of the same kind as this one, as 00:08:43.200 --> 00:08:47.080 align:middle line:84% independent from what happened before. 00:08:47.080 --> 00:08:47.370 align:middle line:90% All right. 00:08:47.370 --> 00:08:49.610 align:middle line:84% So this is pretty much all there is to know about the 00:08:49.610 --> 00:08:50.650 align:middle line:90% Bernoulli process. 00:08:50.650 --> 00:08:52.860 align:middle line:84% Plus the two things that we did at the end of the last 00:08:52.860 --> 00:08:55.640 align:middle line:84% lecture where we merge two independent Bernoulli 00:08:55.640 --> 00:08:58.070 align:middle line:84% processes, we get a Bernoulli process. 00:08:58.070 --> 00:09:01.240 align:middle line:84% If we have a Bernoulli process and we split it by flipping a 00:09:01.240 --> 00:09:05.450 align:middle line:84% coin and sending things one way or the other, then we get 00:09:05.450 --> 00:09:07.790 align:middle line:84% two separate Bernoulli processes. 00:09:07.790 --> 00:09:10.590 align:middle line:84% And we see that all of these carry over to 00:09:10.590 --> 00:09:11.690 align:middle line:90% the continuous time. 00:09:11.690 --> 00:09:14.700 align:middle line:84% And our task for today is basically to work these 00:09:14.700 --> 00:09:18.000 align:middle line:90% continuous time variations. 00:09:18.000 --> 00:09:21.440 align:middle line:84% So the Poisson process is a continuous time version of the 00:09:21.440 --> 00:09:23.480 align:middle line:90% Bernoulli process. 00:09:23.480 --> 00:09:25.250 align:middle line:84% Here's the motivation for considering 00:09:25.250 --> 00:09:26.930 align:middle line:90% it a Bernoulli process. 00:09:26.930 --> 00:09:29.850 align:middle line:84% So you have that person whose job is to sit outside 00:09:29.850 --> 00:09:32.120 align:middle line:90% the door of a bank. 00:09:32.120 --> 00:09:38.280 align:middle line:84% And they have this long sheet, and for every one second slot, 00:09:38.280 --> 00:09:42.560 align:middle line:84% they mark an X if a person came in, or they mark 00:09:42.560 --> 00:09:45.760 align:middle line:84% something else if no one came in during that slot. 00:09:45.760 --> 00:09:48.500 align:middle line:84% Now the bank manager is a really scientifically trained 00:09:48.500 --> 00:09:50.530 align:middle line:84% person and wants very accurate results. 00:09:50.530 --> 00:09:53.380 align:middle line:84% So they tell you, don't use one second slots, use 00:09:53.380 --> 00:09:54.400 align:middle line:90% milliseconds slots. 00:09:54.400 --> 00:09:57.160 align:middle line:84% So you have all those slots and you keep filling if 00:09:57.160 --> 00:09:59.950 align:middle line:84% someone arrived or not during that slot. 00:09:59.950 --> 00:10:01.870 align:middle line:84% Well then you come up with an idea. 00:10:01.870 --> 00:10:06.760 align:middle line:84% Why use millisecond slots and keep putting crosses or zero's 00:10:06.760 --> 00:10:08.150 align:middle line:90% into each slot? 00:10:08.150 --> 00:10:12.380 align:middle line:84% It's much simpler if I just record the exact times when 00:10:12.380 --> 00:10:14.010 align:middle line:90% people came in. 00:10:14.010 --> 00:10:16.380 align:middle line:90% So time is continuous. 00:10:16.380 --> 00:10:20.340 align:middle line:84% I don't keep doing something at every time slot. 00:10:20.340 --> 00:10:24.440 align:middle line:84% But instead of the time axis, I mark the times at which 00:10:24.440 --> 00:10:26.370 align:middle line:90% customers arrive. 00:10:26.370 --> 00:10:28.620 align:middle line:84% So there's no real need for slots. 00:10:28.620 --> 00:10:32.370 align:middle line:84% The only information that you want is when did we have 00:10:32.370 --> 00:10:34.130 align:middle line:90% arrivals of people. 00:10:34.130 --> 00:10:37.830 align:middle line:84% And we want to now model a process of this kind happening 00:10:37.830 --> 00:10:41.880 align:middle line:84% in continuous time, that has the same flavor, however, as 00:10:41.880 --> 00:10:44.210 align:middle line:90% the Bernoulli process. 00:10:44.210 --> 00:10:48.170 align:middle line:84% So that's the model we want to develop. 00:10:48.170 --> 00:10:48.550 align:middle line:90% OK. 00:10:48.550 --> 00:10:52.880 align:middle line:84% So what are the properties that we're going to have? 00:10:52.880 --> 00:10:57.190 align:middle line:84% First, we're going to assume that intervals over the same 00:10:57.190 --> 00:11:01.340 align:middle line:84% length behave probabilistically in an 00:11:01.340 --> 00:11:04.500 align:middle line:90% identical fashion. 00:11:04.500 --> 00:11:06.740 align:middle line:90% So what does that mean? 00:11:06.740 --> 00:11:09.640 align:middle line:84% Think of an interval of some given length. 00:11:09.640 --> 00:11:12.450 align:middle line:84% During the interval of that length, there's going to be a 00:11:12.450 --> 00:11:14.750 align:middle line:90% random number of arrivals. 00:11:14.750 --> 00:11:17.140 align:middle line:84% And that random number of arrivals is going to have a 00:11:17.140 --> 00:11:19.050 align:middle line:90% probability distribution. 00:11:19.050 --> 00:11:20.630 align:middle line:84% So that probability distribution-- 00:11:20.630 --> 00:11:24.820 align:middle line:84% let's denote it by this notation. 00:11:24.820 --> 00:11:29.410 align:middle line:90% We fix t, we fix the duration. 00:11:29.410 --> 00:11:31.650 align:middle line:90% So this is fixed. 00:11:31.650 --> 00:11:34.180 align:middle line:84% And we look at the different k's. 00:11:34.180 --> 00:11:37.190 align:middle line:84% The probability of having 0 arrivals, the probability of 1 00:11:37.190 --> 00:11:40.100 align:middle line:84% arrival, the probability of 2 arrivals, and so on. 00:11:40.100 --> 00:11:42.630 align:middle line:84% So this thing is essentially a PMF. 00:11:42.630 --> 00:11:46.920 align:middle line:84% So it should have the property that the sum over all k's of 00:11:46.920 --> 00:11:49.890 align:middle line:84% this P_(k, tau) should be equal to 1. 00:11:49.890 --> 00:11:52.620 align:middle line:90% 00:11:52.620 --> 00:11:57.490 align:middle line:84% Now, hidden inside this notation is an assumption of 00:11:57.490 --> 00:11:59.550 align:middle line:90% time homogeneity. 00:11:59.550 --> 00:12:03.520 align:middle line:84% That is, this probability distribution for the number of 00:12:03.520 --> 00:12:09.020 align:middle line:84% arrivals only depends on the length of the interval, but 00:12:09.020 --> 00:12:13.230 align:middle line:84% not the exact location of the interval on the time axis. 00:12:13.230 --> 00:12:18.810 align:middle line:84% That is, if I take an interval of length tau, and I ask about 00:12:18.810 --> 00:12:21.200 align:middle line:84% the number of arrivals in this interval. 00:12:21.200 --> 00:12:24.990 align:middle line:84% And I take another interval of length tau, and I ask about 00:12:24.990 --> 00:12:27.790 align:middle line:84% the number of arrivals during that interval. 00:12:27.790 --> 00:12:31.040 align:middle line:84% Number of arrivals here, and number of arrivals there have 00:12:31.040 --> 00:12:34.400 align:middle line:84% the same probability distribution, which is 00:12:34.400 --> 00:12:36.990 align:middle line:90% denoted this way. 00:12:36.990 --> 00:12:41.650 align:middle line:84% So the statistical behavior of arrivals here is the same as 00:12:41.650 --> 00:12:44.310 align:middle line:84% the statistical behavioral of arrivals there. 00:12:44.310 --> 00:12:46.820 align:middle line:84% What's the relation with the Bernoulli process? 00:12:46.820 --> 00:12:48.510 align:middle line:84% It's very much like the assumption-- 00:12:48.510 --> 00:12:49.710 align:middle line:90% the Bernoulli process-- 00:12:49.710 --> 00:12:52.530 align:middle line:84% that in different slots, we have the same 00:12:52.530 --> 00:12:54.450 align:middle line:90% probability of success. 00:12:54.450 --> 00:12:56.770 align:middle line:84% Every slot looks probabilistically 00:12:56.770 --> 00:12:58.400 align:middle line:90% as any other slot. 00:12:58.400 --> 00:13:02.760 align:middle line:84% So similarly here, any interval of length tau looks 00:13:02.760 --> 00:13:06.790 align:middle line:84% probabilistically as any other interval of length tau. 00:13:06.790 --> 00:13:09.710 align:middle line:84% And the number of arrivals during that interval is a 00:13:09.710 --> 00:13:12.350 align:middle line:84% random variable described by these probabilities. 00:13:12.350 --> 00:13:15.570 align:middle line:84% Number of arrivals here is a random variable described by 00:13:15.570 --> 00:13:18.270 align:middle line:90% these same probabilities. 00:13:18.270 --> 00:13:19.900 align:middle line:84% So that's our first assumption. 00:13:19.900 --> 00:13:21.030 align:middle line:90% Then what else? 00:13:21.030 --> 00:13:23.340 align:middle line:84% In the Bernoulli process we had the assumption that 00:13:23.340 --> 00:13:27.710 align:middle line:84% different time slots were independent of each other. 00:13:27.710 --> 00:13:32.970 align:middle line:84% Here we do not have time slots, but we can still think 00:13:32.970 --> 00:13:37.490 align:middle line:84% in a similar way and impose the following assumption, that 00:13:37.490 --> 00:13:40.970 align:middle line:84% these joint time intervals are statistically independent. 00:13:40.970 --> 00:13:42.640 align:middle line:90% What does that mean? 00:13:42.640 --> 00:13:45.530 align:middle line:84% Does a random number of arrivals during this interval, 00:13:45.530 --> 00:13:48.270 align:middle line:84% and the random number of arrivals during this interval, 00:13:48.270 --> 00:13:49.830 align:middle line:90% and the random number of 00:13:49.830 --> 00:13:51.730 align:middle line:90% arrivals during this interval-- 00:13:51.730 --> 00:13:55.110 align:middle line:84% so these are three different random variables-- 00:13:55.110 --> 00:13:58.900 align:middle line:84% these three random variables are independent of each other. 00:13:58.900 --> 00:14:02.170 align:middle line:84% How many arrivals we got here is independent from how many 00:14:02.170 --> 00:14:04.040 align:middle line:90% arrivals we got there. 00:14:04.040 --> 00:14:07.410 align:middle line:84% So this is similar to saying that different time slots were 00:14:07.410 --> 00:14:08.110 align:middle line:90% independent. 00:14:08.110 --> 00:14:10.020 align:middle line:84% That's what we did in discrete time. 00:14:10.020 --> 00:14:13.190 align:middle line:84% The continuous time analog is this independence assumption. 00:14:13.190 --> 00:14:16.400 align:middle line:84% So for example, in particular, number of arrivals here is 00:14:16.400 --> 00:14:19.440 align:middle line:84% independent from the number of arrivals there. 00:14:19.440 --> 00:14:22.980 align:middle line:84% So these are two basic assumptions about the process. 00:14:22.980 --> 00:14:25.620 align:middle line:90% 00:14:25.620 --> 00:14:30.270 align:middle line:84% Now in order to write down a formula, eventually, about 00:14:30.270 --> 00:14:33.050 align:middle line:84% this probability distribution-- 00:14:33.050 --> 00:14:36.860 align:middle line:84% which is our next objective, we would like to say something 00:14:36.860 --> 00:14:38.790 align:middle line:84% specific about this distribution 00:14:38.790 --> 00:14:40.310 align:middle line:90% of number of arrivals-- 00:14:40.310 --> 00:14:43.980 align:middle line:84% we need to add a little more structure into the problem. 00:14:43.980 --> 00:14:47.380 align:middle line:84% And we're going to make the following assumption. 00:14:47.380 --> 00:14:51.140 align:middle line:84% If we look at the time interval of length delta-- 00:14:51.140 --> 00:14:54.090 align:middle line:84% and delta now is supposed to be a small number, so a 00:14:54.090 --> 00:14:55.900 align:middle line:90% picture like this-- 00:14:55.900 --> 00:15:00.790 align:middle line:84% during a very small time interval, there is a 00:15:00.790 --> 00:15:06.140 align:middle line:84% probability that we get exactly one arrival, which is 00:15:06.140 --> 00:15:07.750 align:middle line:90% lambda times delta. 00:15:07.750 --> 00:15:10.630 align:middle line:84% Delta is the length of the interval and lambda is a 00:15:10.630 --> 00:15:14.510 align:middle line:84% proportionality factor, which is sort of the intensity of 00:15:14.510 --> 00:15:16.190 align:middle line:90% the arrival process. 00:15:16.190 --> 00:15:21.000 align:middle line:84% Bigger lambda means that a little interval is more likely 00:15:21.000 --> 00:15:24.140 align:middle line:90% to get an arrival. 00:15:24.140 --> 00:15:25.595 align:middle line:90% So there's a probability lambda 00:15:25.595 --> 00:15:27.570 align:middle line:90% times delta of 1 arrival. 00:15:27.570 --> 00:15:31.740 align:middle line:84% The remaining probability goes to 0 arrivals. 00:15:31.740 --> 00:15:35.560 align:middle line:84% And when delta is small, the probability of 2 arrivals can 00:15:35.560 --> 00:15:39.660 align:middle line:90% be approximated by 0. 00:15:39.660 --> 00:15:42.460 align:middle line:84% So this is a description of what happens during 00:15:42.460 --> 00:15:45.240 align:middle line:90% a small, tiny slot. 00:15:45.240 --> 00:15:48.200 align:middle line:84% Now this is something that's supposed to be true in some 00:15:48.200 --> 00:15:51.530 align:middle line:84% limiting sense, when delta is very small. 00:15:51.530 --> 00:15:56.070 align:middle line:84% So the exact version of this statement would be that this 00:15:56.070 --> 00:16:02.050 align:middle line:84% is an equality, plus order of delta squared terms. 00:16:02.050 --> 00:16:04.060 align:middle line:84% So this is an approximate equality. 00:16:04.060 --> 00:16:07.850 align:middle line:84% And what approximation means is that in the limit of small 00:16:07.850 --> 00:16:11.970 align:middle line:90% deltas, the dominant terms-- 00:16:11.970 --> 00:16:15.900 align:middle line:84% the constant and the first order term are given by this. 00:16:15.900 --> 00:16:19.760 align:middle line:84% Now when delta is very small, second order terms in delta do 00:16:19.760 --> 00:16:21.130 align:middle line:90% not matter. 00:16:21.130 --> 00:16:24.380 align:middle line:84% They are small compared to first order terms. 00:16:24.380 --> 00:16:26.190 align:middle line:90% So we ignore this. 00:16:26.190 --> 00:16:30.280 align:middle line:84% So you can either think in terms of an exact relation, 00:16:30.280 --> 00:16:34.640 align:middle line:84% which is the probabilities are given by this, plus delta 00:16:34.640 --> 00:16:35.990 align:middle line:90% squared terms. 00:16:35.990 --> 00:16:38.730 align:middle line:84% Or if you want to be a little more loose, you just write 00:16:38.730 --> 00:16:41.000 align:middle line:84% here, as an approximate equality. 00:16:41.000 --> 00:16:44.080 align:middle line:84% And the understanding is that this equality holds-- 00:16:44.080 --> 00:16:50.850 align:middle line:84% approximately becomes more and more correct as 00:16:50.850 --> 00:16:53.410 align:middle line:90% delta goes to 0. 00:16:53.410 --> 00:16:57.250 align:middle line:84% So another version of that statement would be that if you 00:16:57.250 --> 00:17:03.280 align:middle line:84% take the limit as delta goes to 0, of p, the probability of 00:17:03.280 --> 00:17:06.829 align:middle line:84% having 1 arrival in an interval of length delta, 00:17:06.829 --> 00:17:10.300 align:middle line:84% divided by delta, this is equal to lambda. 00:17:10.300 --> 00:17:16.010 align:middle line:84% So that would be one version of an exact statement of what 00:17:16.010 --> 00:17:19.250 align:middle line:90% we are assuming here. 00:17:19.250 --> 00:17:22.750 align:middle line:84% So this lambda, we call it the arrival rate, or the intensity 00:17:22.750 --> 00:17:23.930 align:middle line:90% of the process. 00:17:23.930 --> 00:17:27.349 align:middle line:84% And clearly, if you double lambda, then a little interval 00:17:27.349 --> 00:17:29.340 align:middle line:90% is likely -- 00:17:29.340 --> 00:17:31.630 align:middle line:90% you expect to get -- 00:17:31.630 --> 00:17:34.200 align:middle line:84% the probability of obtaining an arrival during that 00:17:34.200 --> 00:17:35.740 align:middle line:90% interval has doubled. 00:17:35.740 --> 00:17:40.400 align:middle line:84% So in some sense we have twice as intense arrival process. 00:17:40.400 --> 00:17:46.470 align:middle line:84% If you look at the number of arrivals during delta 00:17:46.470 --> 00:17:54.490 align:middle line:84% interval, what is the expected value of that random variable? 00:17:54.490 --> 00:18:00.100 align:middle line:84% Well with probability lambda delta we get 1 arrival. 00:18:00.100 --> 00:18:01.240 align:middle line:90% And with the remaining 00:18:01.240 --> 00:18:03.610 align:middle line:90% probability, we get 0 arrivals. 00:18:03.610 --> 00:18:06.680 align:middle line:84% So it's just lambda times delta. 00:18:06.680 --> 00:18:10.640 align:middle line:84% So expected number of arrivals during a little interval is 00:18:10.640 --> 00:18:12.170 align:middle line:90% lambda times delta. 00:18:12.170 --> 00:18:15.460 align:middle line:84% So expected number of arrivals is proportional to lambda, and 00:18:15.460 --> 00:18:19.050 align:middle line:84% that's again why we call lambda the arrival rate. 00:18:19.050 --> 00:18:22.820 align:middle line:84% If you send delta to the denominator in this equality, 00:18:22.820 --> 00:18:26.380 align:middle line:84% it tells you that lambda is the expected number of 00:18:26.380 --> 00:18:30.000 align:middle line:90% arrivals per unit time. 00:18:30.000 --> 00:18:37.010 align:middle line:84% So the arrival rate is expected number of arrivals 00:18:37.010 --> 00:18:38.750 align:middle line:90% per unit time. 00:18:38.750 --> 00:18:42.580 align:middle line:84% And again, that justifies why we call lambda the intensity 00:18:42.580 --> 00:18:43.830 align:middle line:90% of this process. 00:18:43.830 --> 00:18:46.316 align:middle line:90% 00:18:46.316 --> 00:18:46.760 align:middle line:90% All right. 00:18:46.760 --> 00:18:49.680 align:middle line:90% So where are we now? 00:18:49.680 --> 00:18:53.740 align:middle line:84% For the Bernoulli process, the number of arrivals during a 00:18:53.740 --> 00:19:00.210 align:middle line:84% given interval of length n had the PMF that we knew it was 00:19:00.210 --> 00:19:01.545 align:middle line:90% the binomial PMF. 00:19:01.545 --> 00:19:04.190 align:middle line:90% 00:19:04.190 --> 00:19:07.530 align:middle line:84% What is the formula for the corresponding PMF for the 00:19:07.530 --> 00:19:09.190 align:middle line:90% continuous time process? 00:19:09.190 --> 00:19:12.570 align:middle line:84% Somehow we would like to use our assumptions and come up 00:19:12.570 --> 00:19:16.100 align:middle line:84% with the formula for this quantity. 00:19:16.100 --> 00:19:19.110 align:middle line:84% So this tells us about the distribution of number of 00:19:19.110 --> 00:19:23.750 align:middle line:84% arrivals during an interval of some general length. 00:19:23.750 --> 00:19:27.690 align:middle line:84% We have made assumptions about the number of arrivals during 00:19:27.690 --> 00:19:30.410 align:middle line:90% an interval of small length. 00:19:30.410 --> 00:19:34.350 align:middle line:84% An interval of big length is composed of many intervals of 00:19:34.350 --> 00:19:37.830 align:middle line:84% small length, so maybe this is the way to go. 00:19:37.830 --> 00:19:43.120 align:middle line:84% Take a big interval, and split it into many intervals of 00:19:43.120 --> 00:19:44.970 align:middle line:90% small length. 00:19:44.970 --> 00:19:48.410 align:middle line:90% So we have here our time axis. 00:19:48.410 --> 00:19:51.480 align:middle line:84% And we have an interval of length tau. 00:19:51.480 --> 00:19:55.240 align:middle line:84% And I'm going to split it into lots of little intervals of 00:19:55.240 --> 00:19:56.580 align:middle line:90% length delta. 00:19:56.580 --> 00:19:59.000 align:middle line:84% So how many intervals are we going to have? 00:19:59.000 --> 00:20:03.060 align:middle line:84% The number of intervals is going to be the total time, 00:20:03.060 --> 00:20:04.620 align:middle line:90% divided by delta. 00:20:04.620 --> 00:20:07.520 align:middle line:90% 00:20:07.520 --> 00:20:12.960 align:middle line:84% Now what happens during each one of these little intervals? 00:20:12.960 --> 00:20:22.380 align:middle line:84% As long as the intervals are small, what you have is that 00:20:22.380 --> 00:20:24.240 align:middle line:84% during an interval, you're going to have 00:20:24.240 --> 00:20:27.220 align:middle line:90% either 0 or 1 arrival. 00:20:27.220 --> 00:20:29.940 align:middle line:84% The probability of more than 1 arrival during a little 00:20:29.940 --> 00:20:31.950 align:middle line:90% interval is negligible. 00:20:31.950 --> 00:20:35.380 align:middle line:84% So with this picture, you have essentially a Bernoulli 00:20:35.380 --> 00:20:39.970 align:middle line:84% process that consists of so many trials. 00:20:39.970 --> 00:20:43.160 align:middle line:84% And during each one of those trials, we have a probability 00:20:43.160 --> 00:20:46.730 align:middle line:84% of success, which is lambda times delta. 00:20:46.730 --> 00:20:51.845 align:middle line:90% 00:20:51.845 --> 00:20:54.330 align:middle line:84% Different little intervals here are 00:20:54.330 --> 00:20:56.140 align:middle line:90% independent of each other. 00:20:56.140 --> 00:20:58.670 align:middle line:84% That's one of our assumptions, that these joint time 00:20:58.670 --> 00:21:00.380 align:middle line:90% intervals are independent. 00:21:00.380 --> 00:21:05.590 align:middle line:84% So approximately, what we have is a Bernoulli process. 00:21:05.590 --> 00:21:06.980 align:middle line:90% We have independence. 00:21:06.980 --> 00:21:09.250 align:middle line:84% We have the number of slots of interest. 00:21:09.250 --> 00:21:11.450 align:middle line:84% And during each one of the slots we have a certain 00:21:11.450 --> 00:21:13.530 align:middle line:90% probability of success. 00:21:13.530 --> 00:21:17.300 align:middle line:84% So if we think of this as another good approximation of 00:21:17.300 --> 00:21:18.870 align:middle line:90% the Poisson process-- 00:21:18.870 --> 00:21:21.090 align:middle line:84% with the approximation becoming more and more 00:21:21.090 --> 00:21:23.595 align:middle line:90% accurate as delta goes to 0 -- 00:21:23.595 --> 00:21:28.150 align:middle line:84% what we should do would be to take the formula for the PMF 00:21:28.150 --> 00:21:32.320 align:middle line:84% of number of arrivals in a Bernoulli process, and then 00:21:32.320 --> 00:21:37.230 align:middle line:84% take the limit as delta goes to 0. 00:21:37.230 --> 00:21:45.260 align:middle line:84% So in the Bernoulli process, the probability of k arrivals 00:21:45.260 --> 00:21:52.730 align:middle line:84% is n choose k, and then you have p to the k. 00:21:52.730 --> 00:21:57.610 align:middle line:84% Now in our case, we have here lambda times delta, delta is 00:21:57.610 --> 00:21:59.340 align:middle line:90% tau over n. 00:21:59.340 --> 00:22:02.190 align:middle line:90% 00:22:02.190 --> 00:22:08.410 align:middle line:84% Delta is tau over n, so p is lambda times tau divided by n. 00:22:08.410 --> 00:22:11.010 align:middle line:90% So here's our p -- 00:22:11.010 --> 00:22:13.690 align:middle line:90% Lambda tau over n -- 00:22:13.690 --> 00:22:22.760 align:middle line:84% to the power k, and then times one minus this-- this is our 00:22:22.760 --> 00:22:24.540 align:middle line:90% one minus p-- 00:22:24.540 --> 00:22:25.790 align:middle line:90% to the power n-k. 00:22:25.790 --> 00:22:30.010 align:middle line:90% 00:22:30.010 --> 00:22:35.730 align:middle line:84% So this is the exact formula for the Bernoulli process. 00:22:35.730 --> 00:22:39.830 align:middle line:84% For the Poisson process, what we do is we take that formula 00:22:39.830 --> 00:22:43.360 align:middle line:90% and we let delta go to 0. 00:22:43.360 --> 00:22:48.150 align:middle line:84% As delta goes to 0, n goes to infinity. 00:22:48.150 --> 00:22:51.280 align:middle line:84% So that's the limit that we're taking. 00:22:51.280 --> 00:22:55.580 align:middle line:84% On the other hand, this expression lambda times tau-- 00:22:55.580 --> 00:22:59.730 align:middle line:90% 00:22:59.730 --> 00:23:03.740 align:middle line:84% lambda times tau, what is it going to be? 00:23:03.740 --> 00:23:06.860 align:middle line:84% Lambda times tau is equal to n times p. 00:23:06.860 --> 00:23:09.900 align:middle line:90% 00:23:09.900 --> 00:23:11.990 align:middle line:84% n times p, is that what I want? 00:23:11.990 --> 00:23:21.300 align:middle line:90% 00:23:21.300 --> 00:23:22.550 align:middle line:90% No, let's see. 00:23:22.550 --> 00:23:26.600 align:middle line:90% 00:23:26.600 --> 00:23:28.110 align:middle line:90% Lambda tau is np. 00:23:28.110 --> 00:23:29.990 align:middle line:90% Yeah. 00:23:29.990 --> 00:23:32.060 align:middle line:90% So lambda tau is np. 00:23:32.060 --> 00:23:53.370 align:middle line:90% 00:23:53.370 --> 00:23:54.030 align:middle line:90% All right. 00:23:54.030 --> 00:23:59.320 align:middle line:84% So we have this relation, lambda tau equals np. 00:23:59.320 --> 00:24:03.070 align:middle line:84% These two numbers being equal kind of makes sense. np is the 00:24:03.070 --> 00:24:05.890 align:middle line:84% expected number of successes you're going to get in the 00:24:05.890 --> 00:24:07.750 align:middle line:90% Bernoulli process. 00:24:07.750 --> 00:24:08.780 align:middle line:90% Lambda tau-- 00:24:08.780 --> 00:24:11.800 align:middle line:84% since lambda is the arrival rate and you have a total time 00:24:11.800 --> 00:24:15.710 align:middle line:84% of tau, lambda tau you can think of it as the number of 00:24:15.710 --> 00:24:19.750 align:middle line:84% expected arrivals in the Bernoulli process. 00:24:19.750 --> 00:24:22.000 align:middle line:84% We're doing a Bernoulli approximation 00:24:22.000 --> 00:24:23.250 align:middle line:90% to the Poisson process. 00:24:23.250 --> 00:24:26.150 align:middle line:84% We take the formula for the Bernoulli, and now take the 00:24:26.150 --> 00:24:30.060 align:middle line:90% limit as n goes to infinity. 00:24:30.060 --> 00:24:35.330 align:middle line:84% Now lambda tau over n is equal to p, so it's clear what this 00:24:35.330 --> 00:24:37.040 align:middle line:90% term is going to give us. 00:24:37.040 --> 00:24:39.695 align:middle line:90% This is just p to the power k. 00:24:39.695 --> 00:24:48.820 align:middle line:90% 00:24:48.820 --> 00:24:53.230 align:middle line:84% It will actually take a little more work than that. 00:24:53.230 --> 00:24:58.210 align:middle line:84% Now I'm not going to do the algebra, but I'm just telling 00:24:58.210 --> 00:25:03.390 align:middle line:84% you that one can take the limit in this formula here, as 00:25:03.390 --> 00:25:05.000 align:middle line:90% n goes to infinity. 00:25:05.000 --> 00:25:08.860 align:middle line:84% And that will give you another formula, the final formula for 00:25:08.860 --> 00:25:10.320 align:middle line:90% the Poisson PMF. 00:25:10.320 --> 00:25:13.400 align:middle line:84% One thing to notice is that here you have something like 1 00:25:13.400 --> 00:25:17.460 align:middle line:84% minus a constant over n, to the power n. 00:25:17.460 --> 00:25:21.630 align:middle line:84% And you may recall from calculus a formula of this 00:25:21.630 --> 00:25:26.540 align:middle line:84% kind, that this converges to e to the minus c. 00:25:26.540 --> 00:25:29.560 align:middle line:84% If you remember that formula from calculus, then you will 00:25:29.560 --> 00:25:32.750 align:middle line:84% expect that here, in the limit, you are going to get 00:25:32.750 --> 00:25:36.520 align:middle line:84% something like an e to the minus lambda tau. 00:25:36.520 --> 00:25:39.180 align:middle line:84% So indeed, we will get such a term. 00:25:39.180 --> 00:25:42.230 align:middle line:84% There is some work that needs to be done to find the limit 00:25:42.230 --> 00:25:45.880 align:middle line:84% of this expression, times that expression. 00:25:45.880 --> 00:25:48.690 align:middle line:84% The algebra is not hard, it's in the text. 00:25:48.690 --> 00:25:51.340 align:middle line:84% Let's not spend more time doing this. 00:25:51.340 --> 00:25:53.820 align:middle line:84% But let me just give you the formula of what 00:25:53.820 --> 00:25:55.620 align:middle line:90% comes at the end. 00:25:55.620 --> 00:25:59.720 align:middle line:84% And the formula that comes at the end is of this form. 00:25:59.720 --> 00:26:03.710 align:middle line:84% So what matters here is not so much the specific algebra that 00:26:03.710 --> 00:26:07.690 align:middle line:84% you will do to go from this formula to that one. 00:26:07.690 --> 00:26:09.370 align:middle line:90% It's kind of straightforward. 00:26:09.370 --> 00:26:14.535 align:middle line:84% What's important is the idea that the Poisson process, by 00:26:14.535 --> 00:26:19.710 align:middle line:84% definition, can be approximated by a Bernoulli 00:26:19.710 --> 00:26:25.040 align:middle line:84% process in which we have a very large number of slots-- 00:26:25.040 --> 00:26:27.970 align:middle line:90% n goes to infinity. 00:26:27.970 --> 00:26:32.600 align:middle line:84% Whereas we have a very small probability of success during 00:26:32.600 --> 00:26:34.420 align:middle line:90% each time slot. 00:26:34.420 --> 00:26:38.640 align:middle line:84% So a large number of slots, but tiny probability of 00:26:38.640 --> 00:26:40.480 align:middle line:90% success during each slot. 00:26:40.480 --> 00:26:42.370 align:middle line:84% And we take the limit as the slots 00:26:42.370 --> 00:26:44.680 align:middle line:90% become smaller and smaller. 00:26:44.680 --> 00:26:47.170 align:middle line:84% So with this approximation we end up with 00:26:47.170 --> 00:26:49.030 align:middle line:90% this particular formula. 00:26:49.030 --> 00:26:51.890 align:middle line:84% And this is the so-called Poisson PMF. 00:26:51.890 --> 00:26:53.930 align:middle line:90% Now this function P here -- 00:26:53.930 --> 00:26:55.190 align:middle line:90% has two arguments. 00:26:55.190 --> 00:26:58.320 align:middle line:84% The important thing to realize is that when you think of this 00:26:58.320 --> 00:27:02.900 align:middle line:90% as a PMF, you fix t to tau. 00:27:02.900 --> 00:27:06.010 align:middle line:84% And for a fixed tau, now this is a PMF. 00:27:06.010 --> 00:27:11.260 align:middle line:84% As I said before, the sum over k has to be equal to 1. 00:27:11.260 --> 00:27:15.510 align:middle line:84% So for a given tau, these probabilities add up to 1. 00:27:15.510 --> 00:27:20.590 align:middle line:84% The formula is moderately messy, but not too messy. 00:27:20.590 --> 00:27:24.570 align:middle line:84% One can work with it without too much pain. 00:27:24.570 --> 00:27:28.460 align:middle line:84% And what's the mean and variance of this PMF? 00:27:28.460 --> 00:27:31.560 align:middle line:84% Well what's the expected number of arrivals? 00:27:31.560 --> 00:27:35.680 align:middle line:84% If you think of this Bernoulli analogy, we know that the 00:27:35.680 --> 00:27:37.940 align:middle line:84% expected number of arrivals in the Bernoulli 00:27:37.940 --> 00:27:41.250 align:middle line:90% process is n times p. 00:27:41.250 --> 00:27:44.500 align:middle line:84% In the approximation that we're using in these 00:27:44.500 --> 00:27:48.170 align:middle line:84% procedure, n times p is the same as lambda tau. 00:27:48.170 --> 00:27:52.490 align:middle line:84% And that's why we get lambda tau to be the expected number 00:27:52.490 --> 00:27:53.480 align:middle line:90% of arrivals. 00:27:53.480 --> 00:27:56.140 align:middle line:84% Here I'm using t instead of tau. 00:27:56.140 --> 00:28:01.670 align:middle line:84% The expected number of arrivals is lambda t. 00:28:01.670 --> 00:28:05.450 align:middle line:84% So if you double the time, you expect to get 00:28:05.450 --> 00:28:07.290 align:middle line:90% twice as many arrivals. 00:28:07.290 --> 00:28:10.960 align:middle line:84% If you double the arrival rate, you expect to get twice 00:28:10.960 --> 00:28:12.760 align:middle line:90% as many arrivals. 00:28:12.760 --> 00:28:15.290 align:middle line:84% How about the formula for the variance? 00:28:15.290 --> 00:28:18.930 align:middle line:84% The variance of the Bernoulli process is np, 00:28:18.930 --> 00:28:22.720 align:middle line:90% times one minus p. 00:28:22.720 --> 00:28:25.800 align:middle line:84% What does this go to in the limit? 00:28:25.800 --> 00:28:31.170 align:middle line:84% In the limit that we're taking, as delta goes to zero, 00:28:31.170 --> 00:28:33.550 align:middle line:90% then p also goes to zero. 00:28:33.550 --> 00:28:37.270 align:middle line:84% The probability of success in any given slot goes to zero. 00:28:37.270 --> 00:28:39.700 align:middle line:84% So this term becomes insignificant. 00:28:39.700 --> 00:28:45.980 align:middle line:84% So this becomes n times p, which is again lambda t, or 00:28:45.980 --> 00:28:47.710 align:middle line:90% lambda tau. 00:28:47.710 --> 00:28:50.960 align:middle line:84% So the variance, instead of having this more complicated 00:28:50.960 --> 00:28:54.290 align:middle line:84% formula of the variance is the Bernoulli process, here it 00:28:54.290 --> 00:28:56.840 align:middle line:84% gets simplified and it's lambda t. 00:28:56.840 --> 00:29:00.580 align:middle line:84% So interestingly, the variance in the Poisson process is 00:29:00.580 --> 00:29:03.200 align:middle line:84% exactly the same as the expected value. 00:29:03.200 --> 00:29:06.360 align:middle line:84% So you can look at this as just some interesting 00:29:06.360 --> 00:29:07.780 align:middle line:90% coincidence. 00:29:07.780 --> 00:29:10.260 align:middle line:84% So now we're going to take this formula and 00:29:10.260 --> 00:29:11.370 align:middle line:90% see how to use it. 00:29:11.370 --> 00:29:14.060 align:middle line:84% First we're going to do a completely trivial, 00:29:14.060 --> 00:29:16.560 align:middle line:90% straightforward example. 00:29:16.560 --> 00:29:24.630 align:middle line:84% So 15 years ago when that example was made, email was 00:29:24.630 --> 00:29:27.230 align:middle line:84% coming at a rate of five messages per hour. 00:29:27.230 --> 00:29:30.660 align:middle line:84% I wish that was the case today. 00:29:30.660 --> 00:29:38.450 align:middle line:84% And now emails that are coming in, let's say during the day-- 00:29:38.450 --> 00:29:41.750 align:middle line:84% the arrival rates of emails are probably different in 00:29:41.750 --> 00:29:42.960 align:middle line:90% different times of the day. 00:29:42.960 --> 00:29:46.840 align:middle line:84% But if you fix a time slot, let's say 1:00 to 2:00 in the 00:29:46.840 --> 00:29:49.370 align:middle line:84% afternoon, there's probably a constant rate. 00:29:49.370 --> 00:29:53.050 align:middle line:84% And email arrivals are reasonably well modeled by a 00:29:53.050 --> 00:29:54.790 align:middle line:90% Poisson process. 00:29:54.790 --> 00:29:58.220 align:middle line:84% Speaking of modeling, it's not just email arrivals. 00:29:58.220 --> 00:30:02.290 align:middle line:84% Whenever arrivals happen in a completely random way, without 00:30:02.290 --> 00:30:05.370 align:middle line:84% any additional structure, the Poisson process is a good 00:30:05.370 --> 00:30:07.010 align:middle line:90% model of these arrivals. 00:30:07.010 --> 00:30:10.200 align:middle line:84% So the times at which car accidents will happen, that's 00:30:10.200 --> 00:30:11.450 align:middle line:90% a Poisson processes. 00:30:11.450 --> 00:30:15.530 align:middle line:90% 00:30:15.530 --> 00:30:19.550 align:middle line:84% If you have a very, very weak light source that's shooting 00:30:19.550 --> 00:30:24.290 align:middle line:84% out photons, just one at a time, the times at which these 00:30:24.290 --> 00:30:27.240 align:middle line:84% photons will go out is well modeled again 00:30:27.240 --> 00:30:28.670 align:middle line:90% by a Poisson process. 00:30:28.670 --> 00:30:30.540 align:middle line:90% So it's completely random. 00:30:30.540 --> 00:30:35.230 align:middle line:84% Or if you have a radioactive material where one atom at a 00:30:35.230 --> 00:30:43.720 align:middle line:90% time changes at random times. 00:30:43.720 --> 00:30:45.920 align:middle line:84% So it's a very slow radioactive decay. 00:30:45.920 --> 00:30:48.900 align:middle line:84% The time at which these alpha particles, or whatever we get 00:30:48.900 --> 00:30:51.580 align:middle line:84% emitted, again is going to be described 00:30:51.580 --> 00:30:53.200 align:middle line:90% by a Poisson process. 00:30:53.200 --> 00:30:58.220 align:middle line:84% So if you have arrivals, or emissions, that happen at 00:30:58.220 --> 00:31:02.660 align:middle line:84% completely random times, and once in a while you get an 00:31:02.660 --> 00:31:07.500 align:middle line:84% arrival or an event, then the Poisson process is a very good 00:31:07.500 --> 00:31:10.070 align:middle line:90% model for these events. 00:31:10.070 --> 00:31:12.200 align:middle line:90% So back to emails. 00:31:12.200 --> 00:31:16.350 align:middle line:84% Get them at a rate of five messages per day, per hour. 00:31:16.350 --> 00:31:19.420 align:middle line:84% In 30 minutes this is half an hour. 00:31:19.420 --> 00:31:23.770 align:middle line:84% So what we have is that lambda t, total 00:31:23.770 --> 00:31:26.520 align:middle line:90% number of arrivals is-- 00:31:26.520 --> 00:31:29.020 align:middle line:84% the expected number of arrivals is-- 00:31:29.020 --> 00:31:33.810 align:middle line:84% lambda is five, t is one-half, if we talk about hours. 00:31:33.810 --> 00:31:36.480 align:middle line:90% So lambda t is two to the 0.5. 00:31:36.480 --> 00:31:41.220 align:middle line:84% The probability of no new messages is the probability of 00:31:41.220 --> 00:31:48.560 align:middle line:84% zero, in time interval of length t, which, in our case, 00:31:48.560 --> 00:31:51.790 align:middle line:90% is one-half. 00:31:51.790 --> 00:31:55.510 align:middle line:84% And then we look back into the formula from the previous 00:31:55.510 --> 00:31:59.550 align:middle line:84% slide, and the probability of zero arrivals is lambda t to 00:31:59.550 --> 00:32:03.770 align:middle line:84% the power zero, divided by zero factorial, and then an e 00:32:03.770 --> 00:32:05.450 align:middle line:90% to the lambda t. 00:32:05.450 --> 00:32:07.840 align:middle line:84% And you plug in the numbers that we have. 00:32:07.840 --> 00:32:10.380 align:middle line:84% Lambda t to the zero power is one. 00:32:10.380 --> 00:32:12.040 align:middle line:90% Zero factorial is one. 00:32:12.040 --> 00:32:15.500 align:middle line:84% So we're left with e to the minus 2.5. 00:32:15.500 --> 00:32:18.860 align:middle line:90% And that number is 0.08. 00:32:18.860 --> 00:32:22.090 align:middle line:84% Similarly, you can ask for the probability that you get 00:32:22.090 --> 00:32:24.850 align:middle line:84% exactly one message in half an hour. 00:32:24.850 --> 00:32:27.420 align:middle line:84% And that would be-- the probability of one message in 00:32:27.420 --> 00:32:28.680 align:middle line:90% one-half an hour-- 00:32:28.680 --> 00:32:32.590 align:middle line:84% is going to be lambda t to the first power, divided by 1 00:32:32.590 --> 00:32:38.230 align:middle line:84% factorial, e to the minus lambda t, which-- 00:32:38.230 --> 00:32:41.900 align:middle line:84% as we now get the extra lambda t factor-- is going to be 2.5, 00:32:41.900 --> 00:32:43.650 align:middle line:90% e to the minus 2.5. 00:32:43.650 --> 00:32:46.930 align:middle line:84% And the numerical answer is 0.20. 00:32:46.930 --> 00:32:50.450 align:middle line:84% So this is how you use the PMF formula for the Poisson 00:32:50.450 --> 00:32:55.540 align:middle line:84% distribution that we had in the previous slide. 00:32:55.540 --> 00:32:55.890 align:middle line:90% All right. 00:32:55.890 --> 00:33:00.010 align:middle line:84% So this was all about the distribution of 00:33:00.010 --> 00:33:01.780 align:middle line:90% the number of arrivals. 00:33:01.780 --> 00:33:03.350 align:middle line:90% What else did we do last time? 00:33:03.350 --> 00:33:08.250 align:middle line:84% Last time we also talked about the time it takes until the 00:33:08.250 --> 00:33:09.500 align:middle line:90% k-th arrival. 00:33:09.500 --> 00:33:12.390 align:middle line:90% 00:33:12.390 --> 00:33:12.790 align:middle line:90% OK. 00:33:12.790 --> 00:33:16.020 align:middle line:84% So let's try to figure out something about this 00:33:16.020 --> 00:33:18.260 align:middle line:90% particular distribution. 00:33:18.260 --> 00:33:21.180 align:middle line:84% We can derive the distribution of the time of the k-th 00:33:21.180 --> 00:33:24.730 align:middle line:84% arrival by using the exact same argument 00:33:24.730 --> 00:33:27.360 align:middle line:90% as we did last time. 00:33:27.360 --> 00:33:31.650 align:middle line:84% So now the time of the k-th arrival is a 00:33:31.650 --> 00:33:33.830 align:middle line:90% continuous random variable. 00:33:33.830 --> 00:33:36.160 align:middle line:90% So it has a PDF. 00:33:36.160 --> 00:33:38.430 align:middle line:84% Since we are in continuous time, arrivals can 00:33:38.430 --> 00:33:39.900 align:middle line:90% happen at any time. 00:33:39.900 --> 00:33:42.310 align:middle line:84% So Yk is a continuous random variable. 00:33:42.310 --> 00:33:45.200 align:middle line:90% 00:33:45.200 --> 00:33:48.160 align:middle line:84% But now let's think of a time interval of 00:33:48.160 --> 00:33:49.410 align:middle line:90% length little delta. 00:33:49.410 --> 00:33:52.370 align:middle line:90% 00:33:52.370 --> 00:33:58.620 align:middle line:84% And use our usual interpretation of PDFs. 00:33:58.620 --> 00:34:03.180 align:middle line:84% The PDF of a random variable evaluated at a certain time 00:34:03.180 --> 00:34:08.010 align:middle line:84% times delta, this is the probability that the Yk falls 00:34:08.010 --> 00:34:09.514 align:middle line:90% in this little interval. 00:34:09.514 --> 00:34:13.460 align:middle line:90% 00:34:13.460 --> 00:34:16.639 align:middle line:84% So as I've said before, this is the best way of thinking 00:34:16.639 --> 00:34:18.420 align:middle line:90% about PDFs. 00:34:18.420 --> 00:34:22.179 align:middle line:84% PDFs give you probabilities of little intervals. 00:34:22.179 --> 00:34:25.540 align:middle line:84% So now let's try to calculate this probability. 00:34:25.540 --> 00:34:29.880 align:middle line:84% For the k-th arrival to happen inside this little interval, 00:34:29.880 --> 00:34:31.550 align:middle line:90% we need two things. 00:34:31.550 --> 00:34:35.790 align:middle line:84% We need an arrival to happen in this interval, and we need 00:34:35.790 --> 00:34:41.530 align:middle line:84% k minus one arrivals to happen during that interval. 00:34:41.530 --> 00:34:41.880 align:middle line:90% OK. 00:34:41.880 --> 00:34:45.469 align:middle line:84% You'll tell me, but it's possible that we might have 00:34:45.469 --> 00:34:50.130 align:middle line:84% the k minus one arrival happen here, and the k-th arrival to 00:34:50.130 --> 00:34:51.219 align:middle line:90% happen here. 00:34:51.219 --> 00:34:53.050 align:middle line:90% In principle, that's possible. 00:34:53.050 --> 00:34:56.139 align:middle line:84% But in the limit, when we take delta very small, the 00:34:56.139 --> 00:34:59.850 align:middle line:84% probability of having two arrivals in the same little 00:34:59.850 --> 00:35:01.830 align:middle line:90% slot is negligible. 00:35:01.830 --> 00:35:06.870 align:middle line:84% So assuming that no two arrivals can happen in the 00:35:06.870 --> 00:35:10.940 align:middle line:84% same mini slot, then for the k-th one to happen here, we 00:35:10.940 --> 00:35:15.710 align:middle line:84% must have k minus one during this interval. 00:35:15.710 --> 00:35:20.210 align:middle line:84% Now because we have assumed that these joint intervals are 00:35:20.210 --> 00:35:23.900 align:middle line:84% independent of each other, this breaks down into the 00:35:23.900 --> 00:35:33.070 align:middle line:84% probability that we have exactly k minus one arrivals, 00:35:33.070 --> 00:35:37.600 align:middle line:84% during the interval from zero to t, times the probability of 00:35:37.600 --> 00:35:41.410 align:middle line:84% exactly one arrival during that little interval, which is 00:35:41.410 --> 00:35:43.420 align:middle line:90% lambda delta. 00:35:43.420 --> 00:35:51.010 align:middle line:84% We do have a formula for this from the previous slide, which 00:35:51.010 --> 00:35:59.340 align:middle line:84% is lambda t, to the k minus 1, over k minus one factorial, 00:35:59.340 --> 00:36:07.190 align:middle line:90% times e to minus lambda t. 00:36:07.190 --> 00:36:09.070 align:middle line:90% And then lambda times delta. 00:36:09.070 --> 00:36:14.910 align:middle line:90% 00:36:14.910 --> 00:36:16.160 align:middle line:90% Did I miss something? 00:36:16.160 --> 00:36:24.680 align:middle line:90% 00:36:24.680 --> 00:36:26.310 align:middle line:90% Yeah, OK. 00:36:26.310 --> 00:36:26.970 align:middle line:90% All right. 00:36:26.970 --> 00:36:30.220 align:middle line:84% And now you cancel this delta with that delta. 00:36:30.220 --> 00:36:36.820 align:middle line:84% And that gives us a formula for the PDF of the time until 00:36:36.820 --> 00:36:39.170 align:middle line:90% the k-th arrival. 00:36:39.170 --> 00:36:43.290 align:middle line:84% This PDF, of course, depends on the number k. 00:36:43.290 --> 00:36:46.850 align:middle line:84% The first arrival is going to happen somewhere in 00:36:46.850 --> 00:36:48.040 align:middle line:90% this range of time. 00:36:48.040 --> 00:36:50.140 align:middle line:84% So this is the PDF that it has. 00:36:50.140 --> 00:36:53.170 align:middle line:84% The second arrival, of course, is going to happen later. 00:36:53.170 --> 00:36:54.860 align:middle line:90% And the PDF is this. 00:36:54.860 --> 00:36:57.880 align:middle line:84% So it's more likely to happen around these times. 00:36:57.880 --> 00:37:01.410 align:middle line:84% The third arrival has this PDF, so it's more likely to 00:37:01.410 --> 00:37:03.690 align:middle line:90% happen around those times. 00:37:03.690 --> 00:37:08.020 align:middle line:84% And if you were to take k equal to 100, 00:37:08.020 --> 00:37:10.470 align:middle line:90% you might get a PDF-- 00:37:10.470 --> 00:37:13.260 align:middle line:84% it's extremely unlikely that the k-th arrival happens in 00:37:13.260 --> 00:37:18.060 align:middle line:84% the beginning, and it might happen somewhere down there, 00:37:18.060 --> 00:37:20.010 align:middle line:90% far into the future. 00:37:20.010 --> 00:37:22.230 align:middle line:84% So depending on which particular arrival we're 00:37:22.230 --> 00:37:25.510 align:middle line:84% talking about, it has a different probability 00:37:25.510 --> 00:37:26.350 align:middle line:90% distribution. 00:37:26.350 --> 00:37:30.340 align:middle line:84% The time of the 100th arrival, of course, is expected to be a 00:37:30.340 --> 00:37:34.100 align:middle line:84% lot larger than the time of the first arrival. 00:37:34.100 --> 00:37:38.550 align:middle line:84% Incidentally, the time of the first arrival has a PDF whose 00:37:38.550 --> 00:37:40.160 align:middle line:90% form is quite simple. 00:37:40.160 --> 00:37:43.850 align:middle line:84% If you let k equal to one here, this term disappears. 00:37:43.850 --> 00:37:46.310 align:middle line:90% That term becomes a one. 00:37:46.310 --> 00:37:49.880 align:middle line:84% You're left with just lambda, e to the minus lambda. 00:37:49.880 --> 00:37:53.210 align:middle line:84% And you recognize it, it's the exponential distribution. 00:37:53.210 --> 00:37:57.210 align:middle line:84% So the time until the first arrival in a Poisson process 00:37:57.210 --> 00:38:00.160 align:middle line:84% is an exponential distribution. 00:38:00.160 --> 00:38:02.150 align:middle line:84% What was the time of the first arrival in 00:38:02.150 --> 00:38:03.970 align:middle line:90% the Bernoulli process? 00:38:03.970 --> 00:38:07.060 align:middle line:84% It was a geometric distribution. 00:38:07.060 --> 00:38:11.170 align:middle line:84% Well, not coincidentally, these two look quite a bit 00:38:11.170 --> 00:38:13.030 align:middle line:90% like the other. 00:38:13.030 --> 00:38:17.980 align:middle line:84% A geometric distribution has this kind of shape. 00:38:17.980 --> 00:38:21.900 align:middle line:84% The exponential distribution has that kind of shape. 00:38:21.900 --> 00:38:25.560 align:middle line:84% The geometric is just a discrete version of the 00:38:25.560 --> 00:38:27.090 align:middle line:90% exponential. 00:38:27.090 --> 00:38:29.860 align:middle line:84% In the Bernoulli case, we are in discrete time. 00:38:29.860 --> 00:38:32.540 align:middle line:84% We have a PMF for the time of the first 00:38:32.540 --> 00:38:35.080 align:middle line:90% arrival, which is geometric. 00:38:35.080 --> 00:38:38.540 align:middle line:84% In the Poisson case, what we get is the limit of the 00:38:38.540 --> 00:38:41.560 align:middle line:84% geometric as you let those lines become closer and 00:38:41.560 --> 00:38:46.480 align:middle line:84% closer, which gives you the exponential distribution. 00:38:46.480 --> 00:38:50.430 align:middle line:84% Now the Poisson process shares all the memorylessness 00:38:50.430 --> 00:38:52.870 align:middle line:84% properties of the Bernoulli process. 00:38:52.870 --> 00:38:56.750 align:middle line:84% And the way one can argue is just in terms of this picture. 00:38:56.750 --> 00:39:00.250 align:middle line:84% Since the Poisson process is the limit of Bernoulli 00:39:00.250 --> 00:39:03.570 align:middle line:84% processes, whatever qualitative processes you have 00:39:03.570 --> 00:39:07.340 align:middle line:84% in the Bernoulli process remain valid 00:39:07.340 --> 00:39:08.360 align:middle line:90% for the Poisson process. 00:39:08.360 --> 00:39:11.470 align:middle line:84% In particular we have this memorylessness property. 00:39:11.470 --> 00:39:15.120 align:middle line:84% You let the Poisson process run for some time, and then 00:39:15.120 --> 00:39:16.600 align:middle line:90% you start watching it. 00:39:16.600 --> 00:39:18.520 align:middle line:84% What ever happened in the past has no 00:39:18.520 --> 00:39:20.220 align:middle line:90% bearing about the future. 00:39:20.220 --> 00:39:23.150 align:middle line:84% Starting from right now, what's going to happen in the 00:39:23.150 --> 00:39:27.330 align:middle line:84% future is described again by a Poisson process, in the sense 00:39:27.330 --> 00:39:30.530 align:middle line:84% that during every little slot of length delta, there's going 00:39:30.530 --> 00:39:33.790 align:middle line:84% to be a probability of lambda delta of having an arrival. 00:39:33.790 --> 00:39:36.590 align:middle line:84% And that probably lambda delta is the same-- is 00:39:36.590 --> 00:39:38.070 align:middle line:90% always lambda delta-- 00:39:38.070 --> 00:39:41.270 align:middle line:84% no matter what happened in the past of the process. 00:39:41.270 --> 00:39:47.040 align:middle line:84% And in particular, we could use this argument to say that 00:39:47.040 --> 00:39:50.460 align:middle line:84% the time until the k-th arrival is the time that it 00:39:50.460 --> 00:39:53.720 align:middle line:84% takes for the first arrival to happen. 00:39:53.720 --> 00:39:56.380 align:middle line:84% OK, let me do it for k equal to two. 00:39:56.380 --> 00:39:59.630 align:middle line:84% And then after the first arrival happens, you wait a 00:39:59.630 --> 00:40:02.600 align:middle line:84% certain amount of time until the second arrival happens. 00:40:02.600 --> 00:40:06.410 align:middle line:84% Now once the first arrival happened, that's in the past. 00:40:06.410 --> 00:40:07.400 align:middle line:90% You start watching. 00:40:07.400 --> 00:40:10.690 align:middle line:84% From now on you have mini slots of length delta, each 00:40:10.690 --> 00:40:13.230 align:middle line:84% one having a probability of success lambda delta. 00:40:13.230 --> 00:40:16.230 align:middle line:84% It's as if we started the Poisson process from scratch. 00:40:16.230 --> 00:40:19.310 align:middle line:84% So starting from that time, the time until the next 00:40:19.310 --> 00:40:22.840 align:middle line:84% arrival is going to be again an exponential distribution, 00:40:22.840 --> 00:40:26.010 align:middle line:84% which doesn't care about what happened in the past, how long 00:40:26.010 --> 00:40:28.000 align:middle line:84% it took you for the first arrival. 00:40:28.000 --> 00:40:33.410 align:middle line:84% So these two random variables are going to be independent 00:40:33.410 --> 00:40:38.140 align:middle line:84% and exponential, with the same parameter lambda. 00:40:38.140 --> 00:40:42.570 align:middle line:84% So among other things, what we have done here is we have 00:40:42.570 --> 00:40:48.130 align:middle line:84% essentially derived the PDF of the sum of k independent 00:40:48.130 --> 00:40:49.320 align:middle line:90% exponentials. 00:40:49.320 --> 00:40:53.990 align:middle line:84% The time of the k-th arrival is the sum of k 00:40:53.990 --> 00:40:56.230 align:middle line:90% inter-arrival times. 00:40:56.230 --> 00:40:59.380 align:middle line:84% The inter-arrival times are all independent of each other 00:40:59.380 --> 00:41:01.450 align:middle line:90% because of memorylessness. 00:41:01.450 --> 00:41:04.245 align:middle line:84% And they all have the same exponential distribution. 00:41:04.245 --> 00:41:07.130 align:middle line:90% 00:41:07.130 --> 00:41:08.980 align:middle line:84% And by the way, this gives you a way to 00:41:08.980 --> 00:41:11.080 align:middle line:90% simulate the Poisson process. 00:41:11.080 --> 00:41:14.070 align:middle line:84% If you wanted to simulate it on your computer, you would 00:41:14.070 --> 00:41:20.140 align:middle line:84% have one option to break time into tiny, tiny slots. 00:41:20.140 --> 00:41:24.030 align:middle line:84% And for every tiny slot, use your random number generator 00:41:24.030 --> 00:41:27.520 align:middle line:84% to decide whether there was an arrival or not. 00:41:27.520 --> 00:41:29.810 align:middle line:84% To get it very accurate, you would have to 00:41:29.810 --> 00:41:32.090 align:middle line:90% use tiny, tiny slots. 00:41:32.090 --> 00:41:35.280 align:middle line:84% So that would be a lot of computation. 00:41:35.280 --> 00:41:38.530 align:middle line:84% The more clever way of simulating the Poisson process 00:41:38.530 --> 00:41:42.320 align:middle line:84% is you use your random number generator to generate a sample 00:41:42.320 --> 00:41:45.280 align:middle line:84% from an exponential distribution and call that 00:41:45.280 --> 00:41:47.240 align:middle line:90% your first arrival time. 00:41:47.240 --> 00:41:50.050 align:middle line:84% Then go back to the random number generator, generate 00:41:50.050 --> 00:41:53.040 align:middle line:84% another independent sample, again from the same 00:41:53.040 --> 00:41:54.780 align:middle line:90% exponential distribution. 00:41:54.780 --> 00:41:58.490 align:middle line:84% That's the time between the first and the second arrival, 00:41:58.490 --> 00:42:01.390 align:middle line:90% and you keep going that way. 00:42:01.390 --> 00:42:03.260 align:middle line:84% So as a sort of a quick summary, 00:42:03.260 --> 00:42:04.910 align:middle line:90% this is the big picture. 00:42:04.910 --> 00:42:08.630 align:middle line:84% This table doesn't tell you anything new. 00:42:08.630 --> 00:42:12.230 align:middle line:84% But it's good to have it as a reference, and to look at it, 00:42:12.230 --> 00:42:14.740 align:middle line:84% and to make sure you understand what all the 00:42:14.740 --> 00:42:16.300 align:middle line:90% different boxes are. 00:42:16.300 --> 00:42:18.930 align:middle line:84% Basically the Bernoulli process runs in discrete time. 00:42:18.930 --> 00:42:20.960 align:middle line:84% The Poisson process runs in continuous time. 00:42:20.960 --> 00:42:25.140 align:middle line:84% There's an analogy of arrival rates, p per trial, or 00:42:25.140 --> 00:42:27.270 align:middle line:90% intensity per unit time. 00:42:27.270 --> 00:42:32.190 align:middle line:84% We did derive, or sketched the derivation for the PMF of the 00:42:32.190 --> 00:42:33.610 align:middle line:90% number of arrivals. 00:42:33.610 --> 00:42:37.810 align:middle line:84% And the Poisson distribution, which is the distribution that 00:42:37.810 --> 00:42:40.450 align:middle line:90% we get, this Pk of t. 00:42:40.450 --> 00:42:44.220 align:middle line:84% Pk and t is the limit of the binomial when we take the 00:42:44.220 --> 00:42:49.710 align:middle line:84% limit in this particular way, as delta goes to zero, and n 00:42:49.710 --> 00:42:51.220 align:middle line:90% goes to infinity. 00:42:51.220 --> 00:42:54.270 align:middle line:84% The geometric becomes an exponential in the limit. 00:42:54.270 --> 00:42:56.960 align:middle line:84% And the distribution of the time of the k-th arrival-- 00:42:56.960 --> 00:42:59.600 align:middle line:84% we had a closed form formula last time for 00:42:59.600 --> 00:43:01.050 align:middle line:90% the Bernoulli process. 00:43:01.050 --> 00:43:03.930 align:middle line:84% We got the closed form formula this time 00:43:03.930 --> 00:43:05.230 align:middle line:90% for the Poisson process. 00:43:05.230 --> 00:43:08.940 align:middle line:84% And we actually used exactly the same argument to get these 00:43:08.940 --> 00:43:12.320 align:middle line:90% two closed form formulas. 00:43:12.320 --> 00:43:12.650 align:middle line:90% All right. 00:43:12.650 --> 00:43:18.280 align:middle line:84% So now let's talk about adding or merging Poisson processes. 00:43:18.280 --> 00:43:21.060 align:middle line:84% And there's two statements that we can make here. 00:43:21.060 --> 00:43:25.970 align:middle line:84% One has to do with adding Poisson random variables, just 00:43:25.970 --> 00:43:26.820 align:middle line:90% random variables. 00:43:26.820 --> 00:43:28.290 align:middle line:90% There's another statement about 00:43:28.290 --> 00:43:30.770 align:middle line:90% adding Poisson processes. 00:43:30.770 --> 00:43:34.540 align:middle line:84% And the second is a bigger statement than the first. 00:43:34.540 --> 00:43:36.140 align:middle line:90% But this is a warm up. 00:43:36.140 --> 00:43:39.140 align:middle line:84% Let's work with the first statement. 00:43:39.140 --> 00:43:42.460 align:middle line:84% So the claim is that the sum of independent Poisson random 00:43:42.460 --> 00:43:45.340 align:middle line:90% variables is Poisson. 00:43:45.340 --> 00:43:45.990 align:middle line:90% OK. 00:43:45.990 --> 00:43:50.490 align:middle line:84% So suppose that we have a Poisson process with rate-- 00:43:50.490 --> 00:43:51.760 align:middle line:90% just for simplicity-- 00:43:51.760 --> 00:43:53.170 align:middle line:90% lambda one. 00:43:53.170 --> 00:43:56.240 align:middle line:84% And I take the interval from zero to two. 00:43:56.240 --> 00:44:00.620 align:middle line:84% And that take then the interval from two until five. 00:44:00.620 --> 00:44:03.720 align:middle line:84% The number of arrivals during this interval-- 00:44:03.720 --> 00:44:06.730 align:middle line:84% let's call it n from zero to two-- 00:44:06.730 --> 00:44:13.920 align:middle line:84% is going to be a Poisson random variable, with 00:44:13.920 --> 00:44:18.240 align:middle line:90% parameter, or with mean, two. 00:44:18.240 --> 00:44:24.340 align:middle line:84% The number of arrivals during this interval is n from time 00:44:24.340 --> 00:44:26.340 align:middle line:90% two until five. 00:44:26.340 --> 00:44:31.120 align:middle line:84% This is again a Poisson random variable with mean equal to 00:44:31.120 --> 00:44:34.690 align:middle line:84% three, because the arrival rate is 1 and the duration of 00:44:34.690 --> 00:44:36.990 align:middle line:90% the interval is three. 00:44:36.990 --> 00:44:41.320 align:middle line:84% These two random variables are independent. 00:44:41.320 --> 00:44:43.760 align:middle line:84% They obey the Poisson distribution 00:44:43.760 --> 00:44:45.640 align:middle line:90% that we derived before. 00:44:45.640 --> 00:44:50.930 align:middle line:84% If you add them, what you get is the number of arrivals 00:44:50.930 --> 00:44:53.850 align:middle line:84% during the interval from zero to five. 00:44:53.850 --> 00:44:56.290 align:middle line:84% Now what kind of distribution does this 00:44:56.290 --> 00:44:57.910 align:middle line:90% random variable have? 00:44:57.910 --> 00:45:00.760 align:middle line:84% Well this is the number of arrivals over an interval of a 00:45:00.760 --> 00:45:03.600 align:middle line:84% certain length in a Poisson process. 00:45:03.600 --> 00:45:08.580 align:middle line:84% Therefore, this is also Poisson with mean five. 00:45:08.580 --> 00:45:16.520 align:middle line:90% 00:45:16.520 --> 00:45:19.040 align:middle line:84% Because for the Poisson process we know that this 00:45:19.040 --> 00:45:23.300 align:middle line:84% number of arrivals is Poisson, this is Poisson, but also the 00:45:23.300 --> 00:45:26.610 align:middle line:84% number of overall arrivals is also Poisson. 00:45:26.610 --> 00:45:30.040 align:middle line:84% This establishes that the sum of a Poisson plus a Poisson 00:45:30.040 --> 00:45:32.200 align:middle line:84% random variable gives us another 00:45:32.200 --> 00:45:33.630 align:middle line:90% Poisson random variable. 00:45:33.630 --> 00:45:37.110 align:middle line:84% So adding Poisson random variables gives us a Poisson 00:45:37.110 --> 00:45:38.720 align:middle line:90% random variable. 00:45:38.720 --> 00:45:42.660 align:middle line:84% But now I'm going to make a more general statement that 00:45:42.660 --> 00:45:44.940 align:middle line:84% it's not just number of arrivals during 00:45:44.940 --> 00:45:46.415 align:middle line:90% a fixed time interval-- 00:45:46.415 --> 00:45:50.420 align:middle line:90% 00:45:50.420 --> 00:45:53.240 align:middle line:84% it's not just numbers of arrivals for given time 00:45:53.240 --> 00:45:54.260 align:middle line:90% intervals-- 00:45:54.260 --> 00:45:57.770 align:middle line:84% but rather if you take two different Poisson processes 00:45:57.770 --> 00:46:02.330 align:middle line:84% and add them up, the process itself is Poisson in the sense 00:46:02.330 --> 00:46:05.930 align:middle line:84% that this process is going to satisfy all the assumptions of 00:46:05.930 --> 00:46:07.510 align:middle line:90% a Poisson process. 00:46:07.510 --> 00:46:11.060 align:middle line:84% So the story is that you have a red bulb that flashes at 00:46:11.060 --> 00:46:13.350 align:middle line:84% random times at the rate of lambda one. 00:46:13.350 --> 00:46:14.980 align:middle line:90% It's a Poisson process. 00:46:14.980 --> 00:46:19.080 align:middle line:84% You have an independent process where a green bulb 00:46:19.080 --> 00:46:21.230 align:middle line:90% flashes at random times. 00:46:21.230 --> 00:46:24.800 align:middle line:84% And you happen to be color blind, so you just see when 00:46:24.800 --> 00:46:26.630 align:middle line:90% something is flashing. 00:46:26.630 --> 00:46:29.920 align:middle line:84% So these two are assumed to be independent Poisson processes. 00:46:29.920 --> 00:46:34.968 align:middle line:84% What can we say about the process that you observe? 00:46:34.968 --> 00:46:40.250 align:middle line:84% So in the processes that you observe, if you take a typical 00:46:40.250 --> 00:46:45.170 align:middle line:84% time interval of length little delta, what can happen during 00:46:45.170 --> 00:46:48.380 align:middle line:90% that little time interval? 00:46:48.380 --> 00:46:55.280 align:middle line:84% The red process may have something flashing. 00:46:55.280 --> 00:46:56.815 align:middle line:90% So red flashes. 00:46:56.815 --> 00:46:59.850 align:middle line:90% 00:46:59.850 --> 00:47:01.580 align:middle line:90% Or the red does not. 00:47:01.580 --> 00:47:06.610 align:middle line:90% 00:47:06.610 --> 00:47:12.170 align:middle line:84% And for the other bulb, the green bulb, there's two 00:47:12.170 --> 00:47:13.020 align:middle line:90% possibilities. 00:47:13.020 --> 00:47:17.910 align:middle line:90% The green one flashes. 00:47:17.910 --> 00:47:20.900 align:middle line:84% And the other possibility is that the green does not. 00:47:20.900 --> 00:47:24.990 align:middle line:90% 00:47:24.990 --> 00:47:25.330 align:middle line:90% OK. 00:47:25.330 --> 00:47:29.070 align:middle line:84% So there's four possibilities about what can happen during a 00:47:29.070 --> 00:47:31.170 align:middle line:90% little slot. 00:47:31.170 --> 00:47:36.080 align:middle line:84% The probability that the red one flashes and the green one 00:47:36.080 --> 00:47:39.750 align:middle line:84% flashes, what is this probability? 00:47:39.750 --> 00:47:43.510 align:middle line:84% It's lambda one delta that the first one flashes, and lambda 00:47:43.510 --> 00:47:47.290 align:middle line:84% two delta that the second one does. 00:47:47.290 --> 00:47:50.280 align:middle line:84% I'm multiplying probabilities here because I'm making the 00:47:50.280 --> 00:47:52.645 align:middle line:84% assumption that the two processes are independent. 00:47:52.645 --> 00:47:55.330 align:middle line:90% 00:47:55.330 --> 00:47:57.330 align:middle line:90% OK. 00:47:57.330 --> 00:48:00.130 align:middle line:84% Now the probability that the red one flashes 00:48:00.130 --> 00:48:01.440 align:middle line:90% is lambda one delta. 00:48:01.440 --> 00:48:08.210 align:middle line:84% But the green one doesn't is one, minus lambda two delta. 00:48:08.210 --> 00:48:12.840 align:middle line:84% Here the probability would be that the red one does not, 00:48:12.840 --> 00:48:16.400 align:middle line:84% times the probability that the green one does. 00:48:16.400 --> 00:48:20.750 align:middle line:84% And then here we have the probability that none of them 00:48:20.750 --> 00:48:26.790 align:middle line:84% flash, which is whatever is left. 00:48:26.790 --> 00:48:29.600 align:middle line:84% But it's one minus lambda one delta, times one 00:48:29.600 --> 00:48:33.160 align:middle line:90% minus lambda two delta. 00:48:33.160 --> 00:48:36.920 align:middle line:84% Now we're thinking about delta as small. 00:48:36.920 --> 00:48:43.260 align:middle line:84% So think of the case where delta goes to zero, but in a 00:48:43.260 --> 00:48:49.840 align:middle line:84% way that we keep the first order terms. 00:48:49.840 --> 00:48:54.070 align:middle line:84% We keep the delta terms, but we throw away the delta 00:48:54.070 --> 00:48:55.020 align:middle line:90% squared terms. 00:48:55.020 --> 00:48:58.170 align:middle line:84% Delta squared terms are much smaller than the delta terms 00:48:58.170 --> 00:49:00.260 align:middle line:90% when delta becomes small. 00:49:00.260 --> 00:49:01.920 align:middle line:90% If we do that-- 00:49:01.920 --> 00:49:05.650 align:middle line:84% if we only keep the order of delta terms-- 00:49:05.650 --> 00:49:07.940 align:middle line:84% this term effectively disappears. 00:49:07.940 --> 00:49:09.110 align:middle line:90% This is delta squared. 00:49:09.110 --> 00:49:11.550 align:middle line:90% So we make it zero. 00:49:11.550 --> 00:49:14.550 align:middle line:84% So the probability of having simultaneously a red and a 00:49:14.550 --> 00:49:17.940 align:middle line:84% green flash during a little interval is negligible. 00:49:17.940 --> 00:49:20.150 align:middle line:90% What do we get here? 00:49:20.150 --> 00:49:23.200 align:middle line:84% Lambda delta times one survives, but 00:49:23.200 --> 00:49:24.910 align:middle line:90% this times that doesn't. 00:49:24.910 --> 00:49:28.820 align:middle line:90% So we can throw that away. 00:49:28.820 --> 00:49:32.190 align:middle line:84% So the approximation that we get is lambda one delta. 00:49:32.190 --> 00:49:34.010 align:middle line:84% Similarly here, this goes away. 00:49:34.010 --> 00:49:36.420 align:middle line:84% We're left with a lambda two delta. 00:49:36.420 --> 00:49:42.140 align:middle line:84% And this is whatever remains, whatever is left. 00:49:42.140 --> 00:49:45.000 align:middle line:90% So what do we have? 00:49:45.000 --> 00:49:51.400 align:middle line:84% That there is a probability of seeing a flash, either a red 00:49:51.400 --> 00:49:54.360 align:middle line:84% or a green, which is lambda one delta, 00:49:54.360 --> 00:49:57.020 align:middle line:90% plus lambda two delta. 00:49:57.020 --> 00:50:03.590 align:middle line:84% So if we take a little interval of length delta here, 00:50:03.590 --> 00:50:11.780 align:middle line:84% it's going to see an arrival with probability approximately 00:50:11.780 --> 00:50:15.100 align:middle line:84% lambda one, plus lambda two, delta. 00:50:15.100 --> 00:50:20.940 align:middle line:84% So every slot in this merged process has an arrival 00:50:20.940 --> 00:50:25.780 align:middle line:84% probability with a rate which is the sum of the rates of 00:50:25.780 --> 00:50:27.600 align:middle line:90% these two processes. 00:50:27.600 --> 00:50:29.640 align:middle line:84% So this is one part of the definition 00:50:29.640 --> 00:50:31.680 align:middle line:90% of the Poisson process. 00:50:31.680 --> 00:50:34.890 align:middle line:84% There's a few more things that one would need to verify. 00:50:34.890 --> 00:50:37.980 align:middle line:84% Namely, that intervals of the same length have the same 00:50:37.980 --> 00:50:41.000 align:middle line:84% probability distribution and that different slots are 00:50:41.000 --> 00:50:42.710 align:middle line:90% independent of each other. 00:50:42.710 --> 00:50:50.780 align:middle line:84% This can be argued by starting from here because different 00:50:50.780 --> 00:50:53.620 align:middle line:84% intervals in this process are independent from each other. 00:50:53.620 --> 00:50:56.900 align:middle line:84% Different intervals here are independent from each other. 00:50:56.900 --> 00:50:59.900 align:middle line:84% It's not hard to argue that different intervals in the 00:50:59.900 --> 00:51:03.580 align:middle line:84% merged process will also be independent of each other. 00:51:03.580 --> 00:51:06.480 align:middle line:84% So the conclusion that comes at the end is that this 00:51:06.480 --> 00:51:10.130 align:middle line:84% process is a Poisson process, with a total rate which is 00:51:10.130 --> 00:51:13.210 align:middle line:84% equal to the sum of the rate of the two processes. 00:51:13.210 --> 00:51:17.010 align:middle line:84% And now if I tell you that an arrival happened in the merged 00:51:17.010 --> 00:51:20.530 align:middle line:84% process at a certain time, how likely is it that 00:51:20.530 --> 00:51:23.470 align:middle line:90% it came from here? 00:51:23.470 --> 00:51:24.950 align:middle line:90% How likely is it? 00:51:24.950 --> 00:51:26.980 align:middle line:90% We go to this picture. 00:51:26.980 --> 00:51:30.140 align:middle line:84% Given that an arrival occurred-- 00:51:30.140 --> 00:51:36.050 align:middle line:84% which is the event that this or that happened-- 00:51:36.050 --> 00:51:39.330 align:middle line:84% what is the probability that it came from the first 00:51:39.330 --> 00:51:42.060 align:middle line:90% process, the red one? 00:51:42.060 --> 00:51:45.190 align:middle line:84% Well it's the probability of this divided by the 00:51:45.190 --> 00:51:48.030 align:middle line:84% probability of this, times that. 00:51:48.030 --> 00:51:52.760 align:middle line:84% Given that this event occurred, you want to find the 00:51:52.760 --> 00:51:56.560 align:middle line:84% conditional probability of that sub event. 00:51:56.560 --> 00:51:58.960 align:middle line:84% So we're asking the question, out of the total probability 00:51:58.960 --> 00:52:00.660 align:middle line:84% of these two, what fraction of that 00:52:00.660 --> 00:52:02.790 align:middle line:90% probability is assigned here? 00:52:02.790 --> 00:52:05.300 align:middle line:84% And this is lambda one delta, after we 00:52:05.300 --> 00:52:07.040 align:middle line:90% ignore the other terms. 00:52:07.040 --> 00:52:09.170 align:middle line:90% This is lambda two delta. 00:52:09.170 --> 00:52:15.040 align:middle line:84% So that fraction is going to be lambda one, over lambda one 00:52:15.040 --> 00:52:16.770 align:middle line:90% plus lambda two. 00:52:16.770 --> 00:52:17.640 align:middle line:90% What does this tell you? 00:52:17.640 --> 00:52:21.820 align:middle line:84% If lambda one and lambda two are equal, given that I saw an 00:52:21.820 --> 00:52:25.580 align:middle line:84% arrival here, it's equally likely to be red or green. 00:52:25.580 --> 00:52:29.716 align:middle line:84% But if the reds have a much higher arrival rate, when I 00:52:29.716 --> 00:52:32.700 align:middle line:84% see an arrival here, it's more likely this 00:52:32.700 --> 00:52:34.050 align:middle line:90% number will be large. 00:52:34.050 --> 00:52:38.390 align:middle line:84% So it's more likely to have come from the red process. 00:52:38.390 --> 00:52:40.830 align:middle line:84% OK so we'll continue with this story and do some 00:52:40.830 --> 00:52:42.080 align:middle line:90% applications next time. 00:52:42.080 --> 00:52:43.330 align:middle line:90%