WEBVTT
00:00:00.710 --> 00:00:04.010
The subject of this segment is
the calculation of the PMF of
00:00:04.010 --> 00:00:09.040
the sum of two independent,
discrete random variables.
00:00:09.040 --> 00:00:11.750
This is the simplest example
of a function of two random
00:00:11.750 --> 00:00:16.050
variables, a function of the
form of g of X and Y, where
00:00:16.050 --> 00:00:18.480
the function g happens
to be just the
00:00:18.480 --> 00:00:21.080
sum of the two arguments.
00:00:21.080 --> 00:00:23.140
This is a very important
example, because there are
00:00:23.140 --> 00:00:25.340
many situations where
random variables get
00:00:25.340 --> 00:00:26.960
added to each other.
00:00:26.960 --> 00:00:29.890
We work with discrete random
variables as a warm up.
00:00:29.890 --> 00:00:32.020
And later, we will consider
the case of
00:00:32.020 --> 00:00:34.170
continuous random variables.
00:00:34.170 --> 00:00:38.020
So suppose that we know the PMFs
of X and Y and that we
00:00:38.020 --> 00:00:40.720
want to compute the probability
that the sum is
00:00:40.720 --> 00:00:42.250
equal to 3.
00:00:42.250 --> 00:00:44.840
It always helps to
have a picture.
00:00:44.840 --> 00:00:49.110
The sum of X and Y will
be equal to 3.
00:00:49.110 --> 00:00:51.720
This is an event that can
happen in many ways.
00:00:51.720 --> 00:00:56.010
For example, x could be 3 and Y
could be 0, or X could be 1
00:00:56.010 --> 00:00:58.910
and Y equal to 2.
00:00:58.910 --> 00:01:01.530
The probability of the event of
interest, that the sum is
00:01:01.530 --> 00:01:04.800
equal to 3, is going to be the
sum of the probabilities of
00:01:04.800 --> 00:01:08.390
all the different ways that
this event can happen.
00:01:08.390 --> 00:01:12.060
So it is going to be a
sum of various terms.
00:01:12.060 --> 00:01:15.800
And the typical term would be
the probability, let's say of
00:01:15.800 --> 00:01:20.210
this outcome, which is that
X is equal to 0 and
00:01:20.210 --> 00:01:22.370
Y is equal to 3.
00:01:22.370 --> 00:01:25.930
Another typical term in the sum
will be the probability of
00:01:25.930 --> 00:01:29.270
this outcome here, the
probability that X is equal to
00:01:29.270 --> 00:01:35.479
1, Y is equal to 2, and so on.
00:01:35.479 --> 00:01:38.350
Now, here comes an
important step.
00:01:38.350 --> 00:01:41.560
Because we have assumed that X
and Y are independent, the
00:01:41.560 --> 00:01:43.910
probability of these two
events happening is the
00:01:43.910 --> 00:01:48.180
product of the probabilities of
each one of these events.
00:01:48.180 --> 00:01:51.570
So it is the product of the
probability that X is equal to
00:01:51.570 --> 00:01:55.440
0, where now I'm using PMF
notation, times the
00:01:55.440 --> 00:01:58.600
probability that Y
is equal to 3.
00:01:58.600 --> 00:02:01.720
Similarly, the next term is
the probability that X is
00:02:01.720 --> 00:02:04.190
equal to 1 times the
probability that
00:02:04.190 --> 00:02:05.900
Y is equal to 2.
00:02:05.900 --> 00:02:09.220
Again, we can do this because
we are assuming that our two
00:02:09.220 --> 00:02:12.510
random variables are independent
of each other.
00:02:12.510 --> 00:02:15.846
Now, let us generalize.
00:02:15.846 --> 00:02:23.260
In the general case, the
probability that the sum takes
00:02:23.260 --> 00:02:28.610
on a particular value little z
can be calculated as follows.
00:02:28.610 --> 00:02:31.420
We look at all the different
ways that the sum of little z
00:02:31.420 --> 00:02:32.980
can be obtained.
00:02:32.980 --> 00:02:36.920
One way is that the random
variable X takes on a specific
00:02:36.920 --> 00:02:40.480
value little X. And at the same
time, the random variable
00:02:40.480 --> 00:02:44.890
Y takes the value that's needed
so that the sum of the
00:02:44.890 --> 00:02:52.200
two is equal to little Z. For a
given value of little X, we
00:02:52.200 --> 00:02:56.400
have a particular way that the
sum is equal to Z. And this
00:02:56.400 --> 00:02:59.579
particular way has a certain
probability.
00:02:59.579 --> 00:03:01.660
But little X could
be anything.
00:03:01.660 --> 00:03:04.730
And different choices of little
x give us different
00:03:04.730 --> 00:03:07.640
ways that the event of
interest can happen.
00:03:07.640 --> 00:03:12.740
So we add those probabilities
over all possible X's.
00:03:12.740 --> 00:03:15.800
And then we proceed
as follows.
00:03:15.800 --> 00:03:18.630
We invoke independence of
X and Y to derive this
00:03:18.630 --> 00:03:21.350
probability as a product
of two probabilities.
00:03:21.350 --> 00:03:25.560
And then we use PMF notation
instead of probability
00:03:25.560 --> 00:03:29.170
notation to obtain this
expression here.
00:03:33.010 --> 00:03:39.540
This formula is called the
convolution formula.
00:03:39.540 --> 00:03:42.320
It is the convolution
of two PMFs.
00:03:42.320 --> 00:03:47.420
What convolution means is that
somebody gives us the PMF of
00:03:47.420 --> 00:03:51.079
one random variable, gives
us also the PMF of
00:03:51.079 --> 00:03:52.640
another random variable.
00:03:52.640 --> 00:03:55.540
And when we say we're given the
PMF, it means we're given
00:03:55.540 --> 00:03:59.360
the values of the PMFs for all
the possible choices of little
00:03:59.360 --> 00:04:02.570
X and little y, the arguments
of the two PMFs.
00:04:02.570 --> 00:04:06.790
Then the convolution formula
does a certain calculation and
00:04:06.790 --> 00:04:11.690
spits out now a new PMF, which
is the PMF of the random
00:04:11.690 --> 00:04:17.459
variable Z. Let's now take a
closer look at what it takes
00:04:17.459 --> 00:04:20.350
to carry out of the calculations
involved in this
00:04:20.350 --> 00:04:22.450
convolution formula.
00:04:22.450 --> 00:04:25.210
Let's proceed by a simple
example that will illustrate
00:04:25.210 --> 00:04:26.140
the methodology.
00:04:26.140 --> 00:04:29.410
We're given two PMFs of
two random variables.
00:04:29.410 --> 00:04:32.159
And assuming that they are
independent, the PMF of their
00:04:32.159 --> 00:04:35.400
sum is determined by
this formula here.
00:04:35.400 --> 00:04:37.920
And we want to see what
those terms in this
00:04:37.920 --> 00:04:39.500
summation would be.
00:04:39.500 --> 00:04:41.640
Suppose that we're interested
in the probability that the
00:04:41.640 --> 00:04:43.220
sum is equal to 3.
00:04:43.220 --> 00:04:46.140
Now, the sum is going
to be equal to 3.
00:04:46.140 --> 00:04:48.330
This can happen in
several ways.
00:04:48.330 --> 00:04:54.580
We could have X equal to
1 and and Y equal to 2.
00:04:54.580 --> 00:04:56.750
This combination is
one way that the
00:04:56.750 --> 00:04:58.700
sum of 3 can be obtained.
00:04:58.700 --> 00:05:03.030
And that combination has a
probability of 1/3 times 3/6.
00:05:03.030 --> 00:05:06.010
And that would be one of the
terms in this summation.
00:05:06.010 --> 00:05:09.320
Another way that the sum of 3
can be obtained is by having X
00:05:09.320 --> 00:05:12.630
equal to 4 and y equal
to minus 1.
00:05:12.630 --> 00:05:17.750
And by multiplying this
probability 2/3 with 2/6, we
00:05:17.750 --> 00:05:21.900
obtain another contribution
to this summation.
00:05:21.900 --> 00:05:25.690
However, keeping track of these
correspondences here can
00:05:25.690 --> 00:05:29.970
become a little complicated if
we have richer our PMFs.
00:05:29.970 --> 00:05:33.460
So an alternative way of
arranging the calculation is
00:05:33.460 --> 00:05:34.940
the following.
00:05:34.940 --> 00:05:41.930
Let us take the PMF of Y, flip
it along this vertical axis.
00:05:41.930 --> 00:05:44.870
So these two terms would go to
the left side, and this term
00:05:44.870 --> 00:05:46.860
will go to the right
hand side.
00:05:46.860 --> 00:05:49.850
And then draw it underneath
the PMF of X.
00:05:49.850 --> 00:05:52.150
This is what we obtain.
00:05:52.150 --> 00:05:56.500
Then let us take this drawing
here and shift it to
00:05:56.500 --> 00:05:58.590
the right by 3.
00:05:58.590 --> 00:06:02.140
So the entry of minus 2 goes
to 1, minus 1 goes to 2,
00:06:02.140 --> 00:06:03.990
and 1 goes to 4.
00:06:03.990 --> 00:06:08.710
So what have we accomplished by
these two transformations?
00:06:08.710 --> 00:06:16.810
Well, the term that had
probability 3/6 and which were
00:06:16.810 --> 00:06:21.940
to be multiplied with the
probability 1/3 on that side,
00:06:21.940 --> 00:06:25.520
now this 3/6 sits here.
00:06:25.520 --> 00:06:28.260
So we have this correspondence.
00:06:28.260 --> 00:06:31.850
And we need to multiply
1/3 by 3/6.
00:06:31.850 --> 00:06:36.170
Similarly, the multiplication of
2/3 with 2/6 corresponds to
00:06:36.170 --> 00:06:39.980
the multiplication of this
probability here times the
00:06:39.980 --> 00:06:43.110
probability of this term here.
00:06:43.110 --> 00:06:47.100
So when the diagrams are
arranged this way, then we
00:06:47.100 --> 00:06:49.320
have a simpler job to do.
00:06:49.320 --> 00:06:53.380
We look at corresponding terms,
those that sit on top
00:06:53.380 --> 00:06:58.150
of each other, multiply them,
do that for all the possible
00:06:58.150 --> 00:07:02.450
choices, and then add those
products together.
00:07:02.450 --> 00:07:05.430
And this is what we do if
we're shifting by 3.
00:07:05.430 --> 00:07:09.470
Now, if we wanted to find the
probability that Z equal to 4,
00:07:09.470 --> 00:07:12.320
we would be doing the same
thing, except that this
00:07:12.320 --> 00:07:16.270
diagram would need to be shifted
by one more unit to
00:07:16.270 --> 00:07:20.100
the right so that we have
a total shift of 4.
00:07:20.100 --> 00:07:23.500
So we just repeat this procedure
for all possible
00:07:23.500 --> 00:07:28.540
values of Z which corresponds
to taking this diagram here
00:07:28.540 --> 00:07:32.790
and shifting it progressively
by different amounts.
00:07:32.790 --> 00:07:36.680
This turns out to be a fairly
simple and systematic way of
00:07:36.680 --> 00:07:38.830
arranging the calculations,
at least if you're
00:07:38.830 --> 00:07:40.430
doing them by hand.
00:07:40.430 --> 00:07:43.920
Of course, an alternative is to
carry out the calculations
00:07:43.920 --> 00:07:45.040
on a computer.
00:07:45.040 --> 00:07:49.000
This is a pretty simple formula
that is not hard to
00:07:49.000 --> 00:07:50.930
implement on a computer.