WEBVTT
00:00:00.292 --> 00:00:01.750
PROFESSOR: In this
segment, we will
00:00:01.750 --> 00:00:04.360
discuss the multinomial
model and the multinomial
00:00:04.360 --> 00:00:06.970
probabilities, which are
a nice generalization
00:00:06.970 --> 00:00:09.190
of the binomial probabilities.
00:00:09.190 --> 00:00:11.050
The setting is as follows.
00:00:11.050 --> 00:00:13.030
We are dealing with
balls and the balls come
00:00:13.030 --> 00:00:14.250
into different colors.
00:00:14.250 --> 00:00:17.380
There are r possible
different colors.
00:00:17.380 --> 00:00:19.860
We pick a ball at random,
and when we do that,
00:00:19.860 --> 00:00:23.530
there is a certain probability
Pi that the ball that we picked
00:00:23.530 --> 00:00:25.580
has ith color.
00:00:25.580 --> 00:00:28.990
Now, we repeat this process
n times independently.
00:00:28.990 --> 00:00:32.409
Each time we get a ball
that has a random color.
00:00:32.409 --> 00:00:35.740
And we're interested in the
following kind of question.
00:00:35.740 --> 00:00:38.560
Somebody fixes for
us certain numbers--
00:00:38.560 --> 00:00:44.090
n1, n2, up to nr that
add up to n, and asks us,
00:00:44.090 --> 00:00:46.300
what is the probability
that when you carry out
00:00:46.300 --> 00:00:50.110
the experiment, you get exactly
n1 balls of the first color,
00:00:50.110 --> 00:00:54.040
exactly n2 balls of the
second color, and so on?
00:00:54.040 --> 00:00:59.740
So the numbers n1, n2, up to
nr are fixed given numbers.
00:00:59.740 --> 00:01:01.480
For a particular choice
of those numbers,
00:01:01.480 --> 00:01:04.750
we want to calculate
this probability.
00:01:04.750 --> 00:01:07.420
Now of course, this is
a more general model.
00:01:07.420 --> 00:01:10.940
It doesn't necessarily deal
with balls of different colors.
00:01:10.940 --> 00:01:15.520
For example, we might have
an experiment that gives us
00:01:15.520 --> 00:01:20.100
random numbers, where the
numbers range from 1 up to r,
00:01:20.100 --> 00:01:24.460
and at each time we get a
random number with probability
00:01:24.460 --> 00:01:27.940
Pi we get a number
which is equal to i.
00:01:27.940 --> 00:01:32.560
So we could use this to
model die rolls, for example.
00:01:32.560 --> 00:01:34.300
And there's actually
a special case
00:01:34.300 --> 00:01:36.820
of this problem, which
should be familiar.
00:01:36.820 --> 00:01:39.220
Suppose that we have
only two colors,
00:01:39.220 --> 00:01:41.530
and instead of
thinking of colors,
00:01:41.530 --> 00:01:43.660
let us think of the
two possibilities
00:01:43.660 --> 00:01:47.080
as being heads or tails.
00:01:47.080 --> 00:01:49.180
And we can make the
following analogy.
00:01:49.180 --> 00:01:55.746
Somebody gives us numbers
n1 and n2 that add up to n.
00:01:55.746 --> 00:01:57.370
And we're interested
in the probability
00:01:57.370 --> 00:01:59.920
that we get n1 of
the first color
00:01:59.920 --> 00:02:02.420
and n2 of the second color.
00:02:02.420 --> 00:02:05.850
Well, we could think of this
as a setting in which we
00:02:05.850 --> 00:02:08.380
are asking for the
probability that we obtain
00:02:08.380 --> 00:02:14.650
k heads and n minus k tails.
00:02:17.650 --> 00:02:20.770
So the question of what is
the probability that we obtain
00:02:20.770 --> 00:02:24.700
k heads and n minus k
tails is of the same kind
00:02:24.700 --> 00:02:28.320
as what is the probability that
we get n1 of the first color
00:02:28.320 --> 00:02:30.880
and n2 of the second color.
00:02:30.880 --> 00:02:34.690
Now, if heads have a
probability p of occurring,
00:02:34.690 --> 00:02:38.560
and tails has a probability
of 1 minus p of occurring,
00:02:38.560 --> 00:02:40.720
then we would have
the following analogy.
00:02:40.720 --> 00:02:43.270
The probability of
obtaining the first color,
00:02:43.270 --> 00:02:47.110
which correspond to heads,
that would be equal to p.
00:02:47.110 --> 00:02:49.630
The probability of obtaining
the second color, which
00:02:49.630 --> 00:02:54.130
correspond to tails,
this would be 1 minus p.
00:02:54.130 --> 00:02:56.620
Now, the probability
of obtaining k
00:02:56.620 --> 00:03:00.790
heads in those n
independent trials--
00:03:00.790 --> 00:03:02.890
we know what it is.
00:03:02.890 --> 00:03:04.930
By the binomial
probabilities, it
00:03:04.930 --> 00:03:09.650
is n choose k times
p to the k times one
00:03:09.650 --> 00:03:14.020
minus p to the power n minus k.
00:03:14.020 --> 00:03:19.150
Now we can translate this
answer to the multinomial case
00:03:19.150 --> 00:03:23.620
where we're dealing with colors,
and we do these substitutions.
00:03:23.620 --> 00:03:31.090
So n choose k is n factorial
divided by k factorial.
00:03:31.090 --> 00:03:37.690
In this case, k is the same
as n1, so we get n1 factorial.
00:03:37.690 --> 00:03:40.900
And then we are going to have
here n minus k factorial.
00:03:40.900 --> 00:03:44.020
But n minus k corresponds to n2.
00:03:44.020 --> 00:03:47.860
So here we get an n2 factorial.
00:03:47.860 --> 00:03:58.120
And then p corresponds to p1
and p2 corresponds to 1 minus p.
00:03:58.120 --> 00:04:01.230
So we get here p1 times p2.
00:04:01.230 --> 00:04:06.920
n to the power n minus k,
again, by analogy, is n2.
00:04:06.920 --> 00:04:10.750
So this is the form of the
multinomial probabilities
00:04:10.750 --> 00:04:13.530
for the special case where
we're dealing with two colors.
00:04:16.149 --> 00:04:19.250
Let us now look at
the general case.
00:04:19.250 --> 00:04:22.370
Let us start with an
example, to be concrete.
00:04:22.370 --> 00:04:26.560
Suppose that the number
of colors is equal to 3,
00:04:26.560 --> 00:04:33.910
and that we're going to
pick n equal to 7 balls.
00:04:33.910 --> 00:04:36.160
We carry out the
experiments, and we
00:04:36.160 --> 00:04:39.310
might obtain an
outcome which would
00:04:39.310 --> 00:04:41.390
be a sequence of this type.
00:04:41.390 --> 00:04:44.650
So the first ball
had the color 1,
00:04:44.650 --> 00:04:46.840
the second ball had
the first color,
00:04:46.840 --> 00:04:49.540
the third ball had
the third color,
00:04:49.540 --> 00:04:53.800
the fourth ball had the
first color, and so on.
00:04:53.800 --> 00:04:57.760
And suppose that
this was the outcome.
00:04:57.760 --> 00:05:01.060
One way of summarizing what
happened in this outcome
00:05:01.060 --> 00:05:15.040
would be to say that we
had four 1s, we had two 2s,
00:05:15.040 --> 00:05:18.380
and we had one 3.
00:05:18.380 --> 00:05:21.860
We could say that this is
the type of the outcome.
00:05:21.860 --> 00:05:23.830
It's of type 4, 2, 1--
00:05:23.830 --> 00:05:26.560
that is, we obtained
four of the first color,
00:05:26.560 --> 00:05:29.500
two of the second color,
and one of the third color.
00:05:32.530 --> 00:05:34.640
This is one possible outcome.
00:05:34.640 --> 00:05:36.760
What is the probability
of obtaining
00:05:36.760 --> 00:05:39.690
this particular outcome?
00:05:39.690 --> 00:05:42.630
The probability of obtaining
this particular outcome
00:05:42.630 --> 00:05:46.710
is, using independence, the
probability that we obtain
00:05:46.710 --> 00:05:51.960
color 1 in the first trial,
color 1 in the second trial,
00:05:51.960 --> 00:05:57.390
color 3 in the third trial,
color 1 in the fourth trial,
00:05:57.390 --> 00:06:01.980
color 2 in the next trial,
color 2 in the next trial,
00:06:01.980 --> 00:06:05.090
color 1 in the last trial.
00:06:05.090 --> 00:06:08.670
And we put all the
factors together,
00:06:08.670 --> 00:06:13.090
and we notice that this
is p1 to the fourth p2
00:06:13.090 --> 00:06:16.220
to the second times p3.
00:06:19.510 --> 00:06:23.590
It's not a coincidence that the
exponents that we have up here
00:06:23.590 --> 00:06:26.560
are exactly the
count that we had
00:06:26.560 --> 00:06:32.050
when we specified the type
of this particular outcome.
00:06:32.050 --> 00:06:35.200
Generalizing from
this example, we
00:06:35.200 --> 00:06:37.960
realize that the
probability of obtaining
00:06:37.960 --> 00:06:44.010
a particular sequence of a
certain type, that probability
00:06:44.010 --> 00:06:46.180
is of this form.
00:06:46.180 --> 00:06:49.210
For each color, we
have the probability
00:06:49.210 --> 00:06:52.660
of that color raised to
the power of how many times
00:06:52.660 --> 00:06:57.280
that particular color
appears in a sequence.
00:06:57.280 --> 00:07:02.110
So any particular sequence of
this type has this probability.
00:07:02.110 --> 00:07:04.510
What we're interested
in is to find
00:07:04.510 --> 00:07:07.150
the total probability
of obtaining
00:07:07.150 --> 00:07:10.270
some sequence of this type.
00:07:10.270 --> 00:07:12.250
How can we find
this probability?
00:07:12.250 --> 00:07:14.170
Well, we will take
the probability
00:07:14.170 --> 00:07:16.780
of each sequence of this type--
00:07:16.780 --> 00:07:19.900
which is this much,
and it's the same
00:07:19.900 --> 00:07:22.190
for any particular sequence--
00:07:22.190 --> 00:07:28.160
and multiply with the number
of sequences of this type.
00:07:28.160 --> 00:07:32.050
So how many sequences are
there of a certain type?
00:07:32.050 --> 00:07:34.400
Let us look back at our example.
00:07:34.400 --> 00:07:37.450
We had seven trials.
00:07:37.450 --> 00:07:42.700
So let us number here
the different trials.
00:07:42.700 --> 00:07:46.630
And when I tell you that
a particular sequence was
00:07:46.630 --> 00:07:50.800
obtained, that's the
same as telling you
00:07:50.800 --> 00:08:00.190
that in this set of trials,
we had the first color.
00:08:03.780 --> 00:08:09.000
In this set of trials,
the fifth and sixth trial,
00:08:09.000 --> 00:08:13.360
we had the second color.
00:08:13.360 --> 00:08:19.870
And in this trial, the third
trial, we had the third color.
00:08:19.870 --> 00:08:22.630
This is an alternative
way of telling you
00:08:22.630 --> 00:08:24.500
what sequence we obtained.
00:08:24.500 --> 00:08:26.530
I tell you at
which trials we had
00:08:26.530 --> 00:08:29.170
the first color, at which trials
we had the second, at which
00:08:29.170 --> 00:08:32.251
trials we had the third.
00:08:32.251 --> 00:08:34.070
But What do we have here?
00:08:34.070 --> 00:08:40.929
Here we have a partition of the
set of numbers from 1 up to 7
00:08:40.929 --> 00:08:43.299
into three subsets.
00:08:43.299 --> 00:08:46.300
And the cardinalities
of those subsets
00:08:46.300 --> 00:08:52.540
are the numbers that appear here
in the type of the sequence.
00:08:52.540 --> 00:08:58.650
The conclusion is that a
sequence of certain type
00:08:58.650 --> 00:09:03.330
is equivalent, or can be
alternatively specified,
00:09:03.330 --> 00:09:11.190
by giving you a partition
over this set of tosses, which
00:09:11.190 --> 00:09:16.440
is the set from 1 up to n,
how many trials we've had,
00:09:16.440 --> 00:09:20.620
a partition into subsets
of certain sizes.
00:09:20.620 --> 00:09:23.820
So this allows us now
to count the number
00:09:23.820 --> 00:09:26.430
of sequences of a certain type.
00:09:26.430 --> 00:09:29.730
It's exactly the same as
the number of partitions,
00:09:29.730 --> 00:09:31.650
and we know what this is.
00:09:31.650 --> 00:09:34.500
And putting everything
together, the probability
00:09:34.500 --> 00:09:37.710
of obtaining a sequence
of a certain type
00:09:37.710 --> 00:09:41.820
is equal to the count
of how many sequences
00:09:41.820 --> 00:09:44.130
do we have of the
certain type, which
00:09:44.130 --> 00:09:47.660
is the same as the number of
partitions of a certain type,
00:09:47.660 --> 00:09:51.600
times the probability of
any particular sequence
00:09:51.600 --> 00:09:56.050
of that type that
we're interested in.
00:09:56.050 --> 00:09:58.980
So this is a formula
that generalizes the one
00:09:58.980 --> 00:10:03.240
that we saw before for the case
where we have only two colors,
00:10:03.240 --> 00:10:08.140
and which corresponded to
the coin tossing setting.
00:10:08.140 --> 00:10:09.690
And it is a useful
model, because you
00:10:09.690 --> 00:10:13.380
can think of many situations in
which you have repeated trials,
00:10:13.380 --> 00:10:17.130
and at each trial,
you obtain one out
00:10:17.130 --> 00:10:20.640
of a finite set of
possible results.
00:10:20.640 --> 00:10:23.190
There are different
possible results.
00:10:23.190 --> 00:10:25.560
You repeat those
trials independently.
00:10:25.560 --> 00:10:30.010
And you may be interested in
the question of how many results
00:10:30.010 --> 00:10:34.657
of the first kind, of the second
kind, and so on there will be.