1
00:00:00,292 --> 00:00:01,750
PROFESSOR: In this
segment, we will
2
00:00:01,750 --> 00:00:04,360
discuss the multinomial
model and the multinomial
3
00:00:04,360 --> 00:00:06,970
probabilities, which are
a nice generalization
4
00:00:06,970 --> 00:00:09,190
of the binomial probabilities.
5
00:00:09,190 --> 00:00:11,050
The setting is as follows.
6
00:00:11,050 --> 00:00:13,030
We are dealing with
balls and the balls come
7
00:00:13,030 --> 00:00:14,250
into different colors.
8
00:00:14,250 --> 00:00:17,380
There are r possible
different colors.
9
00:00:17,380 --> 00:00:19,860
We pick a ball at random,
and when we do that,
10
00:00:19,860 --> 00:00:23,530
there is a certain probability
Pi that the ball that we picked
11
00:00:23,530 --> 00:00:25,580
has ith color.
12
00:00:25,580 --> 00:00:28,990
Now, we repeat this process
n times independently.
13
00:00:28,990 --> 00:00:32,409
Each time we get a ball
that has a random color.
14
00:00:32,409 --> 00:00:35,740
And we're interested in the
following kind of question.
15
00:00:35,740 --> 00:00:38,560
Somebody fixes for
us certain numbers--
16
00:00:38,560 --> 00:00:44,090
n1, n2, up to nr that
add up to n, and asks us,
17
00:00:44,090 --> 00:00:46,300
what is the probability
that when you carry out
18
00:00:46,300 --> 00:00:50,110
the experiment, you get exactly
n1 balls of the first color,
19
00:00:50,110 --> 00:00:54,040
exactly n2 balls of the
second color, and so on?
20
00:00:54,040 --> 00:00:59,740
So the numbers n1, n2, up to
nr are fixed given numbers.
21
00:00:59,740 --> 00:01:01,480
For a particular choice
of those numbers,
22
00:01:01,480 --> 00:01:04,750
we want to calculate
this probability.
23
00:01:04,750 --> 00:01:07,420
Now of course, this is
a more general model.
24
00:01:07,420 --> 00:01:10,940
It doesn't necessarily deal
with balls of different colors.
25
00:01:10,940 --> 00:01:15,520
For example, we might have
an experiment that gives us
26
00:01:15,520 --> 00:01:20,100
random numbers, where the
numbers range from 1 up to r,
27
00:01:20,100 --> 00:01:24,460
and at each time we get a
random number with probability
28
00:01:24,460 --> 00:01:27,940
Pi we get a number
which is equal to i.
29
00:01:27,940 --> 00:01:32,560
So we could use this to
model die rolls, for example.
30
00:01:32,560 --> 00:01:34,300
And there's actually
a special case
31
00:01:34,300 --> 00:01:36,820
of this problem, which
should be familiar.
32
00:01:36,820 --> 00:01:39,220
Suppose that we have
only two colors,
33
00:01:39,220 --> 00:01:41,530
and instead of
thinking of colors,
34
00:01:41,530 --> 00:01:43,660
let us think of the
two possibilities
35
00:01:43,660 --> 00:01:47,080
as being heads or tails.
36
00:01:47,080 --> 00:01:49,180
And we can make the
following analogy.
37
00:01:49,180 --> 00:01:55,746
Somebody gives us numbers
n1 and n2 that add up to n.
38
00:01:55,746 --> 00:01:57,370
And we're interested
in the probability
39
00:01:57,370 --> 00:01:59,920
that we get n1 of
the first color
40
00:01:59,920 --> 00:02:02,420
and n2 of the second color.
41
00:02:02,420 --> 00:02:05,850
Well, we could think of this
as a setting in which we
42
00:02:05,850 --> 00:02:08,380
are asking for the
probability that we obtain
43
00:02:08,380 --> 00:02:14,650
k heads and n minus k tails.
44
00:02:17,650 --> 00:02:20,770
So the question of what is
the probability that we obtain
45
00:02:20,770 --> 00:02:24,700
k heads and n minus k
tails is of the same kind
46
00:02:24,700 --> 00:02:28,320
as what is the probability that
we get n1 of the first color
47
00:02:28,320 --> 00:02:30,880
and n2 of the second color.
48
00:02:30,880 --> 00:02:34,690
Now, if heads have a
probability p of occurring,
49
00:02:34,690 --> 00:02:38,560
and tails has a probability
of 1 minus p of occurring,
50
00:02:38,560 --> 00:02:40,720
then we would have
the following analogy.
51
00:02:40,720 --> 00:02:43,270
The probability of
obtaining the first color,
52
00:02:43,270 --> 00:02:47,110
which correspond to heads,
that would be equal to p.
53
00:02:47,110 --> 00:02:49,630
The probability of obtaining
the second color, which
54
00:02:49,630 --> 00:02:54,130
correspond to tails,
this would be 1 minus p.
55
00:02:54,130 --> 00:02:56,620
Now, the probability
of obtaining k
56
00:02:56,620 --> 00:03:00,790
heads in those n
independent trials--
57
00:03:00,790 --> 00:03:02,890
we know what it is.
58
00:03:02,890 --> 00:03:04,930
By the binomial
probabilities, it
59
00:03:04,930 --> 00:03:09,650
is n choose k times
p to the k times one
60
00:03:09,650 --> 00:03:14,020
minus p to the power n minus k.
61
00:03:14,020 --> 00:03:19,150
Now we can translate this
answer to the multinomial case
62
00:03:19,150 --> 00:03:23,620
where we're dealing with colors,
and we do these substitutions.
63
00:03:23,620 --> 00:03:31,090
So n choose k is n factorial
divided by k factorial.
64
00:03:31,090 --> 00:03:37,690
In this case, k is the same
as n1, so we get n1 factorial.
65
00:03:37,690 --> 00:03:40,900
And then we are going to have
here n minus k factorial.
66
00:03:40,900 --> 00:03:44,020
But n minus k corresponds to n2.
67
00:03:44,020 --> 00:03:47,860
So here we get an n2 factorial.
68
00:03:47,860 --> 00:03:58,120
And then p corresponds to p1
and p2 corresponds to 1 minus p.
69
00:03:58,120 --> 00:04:01,230
So we get here p1 times p2.
70
00:04:01,230 --> 00:04:06,920
n to the power n minus k,
again, by analogy, is n2.
71
00:04:06,920 --> 00:04:10,750
So this is the form of the
multinomial probabilities
72
00:04:10,750 --> 00:04:13,530
for the special case where
we're dealing with two colors.
73
00:04:16,149 --> 00:04:19,250
Let us now look at
the general case.
74
00:04:19,250 --> 00:04:22,370
Let us start with an
example, to be concrete.
75
00:04:22,370 --> 00:04:26,560
Suppose that the number
of colors is equal to 3,
76
00:04:26,560 --> 00:04:33,910
and that we're going to
pick n equal to 7 balls.
77
00:04:33,910 --> 00:04:36,160
We carry out the
experiments, and we
78
00:04:36,160 --> 00:04:39,310
might obtain an
outcome which would
79
00:04:39,310 --> 00:04:41,390
be a sequence of this type.
80
00:04:41,390 --> 00:04:44,650
So the first ball
had the color 1,
81
00:04:44,650 --> 00:04:46,840
the second ball had
the first color,
82
00:04:46,840 --> 00:04:49,540
the third ball had
the third color,
83
00:04:49,540 --> 00:04:53,800
the fourth ball had the
first color, and so on.
84
00:04:53,800 --> 00:04:57,760
And suppose that
this was the outcome.
85
00:04:57,760 --> 00:05:01,060
One way of summarizing what
happened in this outcome
86
00:05:01,060 --> 00:05:15,040
would be to say that we
had four 1s, we had two 2s,
87
00:05:15,040 --> 00:05:18,380
and we had one 3.
88
00:05:18,380 --> 00:05:21,860
We could say that this is
the type of the outcome.
89
00:05:21,860 --> 00:05:23,830
It's of type 4, 2, 1--
90
00:05:23,830 --> 00:05:26,560
that is, we obtained
four of the first color,
91
00:05:26,560 --> 00:05:29,500
two of the second color,
and one of the third color.
92
00:05:32,530 --> 00:05:34,640
This is one possible outcome.
93
00:05:34,640 --> 00:05:36,760
What is the probability
of obtaining
94
00:05:36,760 --> 00:05:39,690
this particular outcome?
95
00:05:39,690 --> 00:05:42,630
The probability of obtaining
this particular outcome
96
00:05:42,630 --> 00:05:46,710
is, using independence, the
probability that we obtain
97
00:05:46,710 --> 00:05:51,960
color 1 in the first trial,
color 1 in the second trial,
98
00:05:51,960 --> 00:05:57,390
color 3 in the third trial,
color 1 in the fourth trial,
99
00:05:57,390 --> 00:06:01,980
color 2 in the next trial,
color 2 in the next trial,
100
00:06:01,980 --> 00:06:05,090
color 1 in the last trial.
101
00:06:05,090 --> 00:06:08,670
And we put all the
factors together,
102
00:06:08,670 --> 00:06:13,090
and we notice that this
is p1 to the fourth p2
103
00:06:13,090 --> 00:06:16,220
to the second times p3.
104
00:06:19,510 --> 00:06:23,590
It's not a coincidence that the
exponents that we have up here
105
00:06:23,590 --> 00:06:26,560
are exactly the
count that we had
106
00:06:26,560 --> 00:06:32,050
when we specified the type
of this particular outcome.
107
00:06:32,050 --> 00:06:35,200
Generalizing from
this example, we
108
00:06:35,200 --> 00:06:37,960
realize that the
probability of obtaining
109
00:06:37,960 --> 00:06:44,010
a particular sequence of a
certain type, that probability
110
00:06:44,010 --> 00:06:46,180
is of this form.
111
00:06:46,180 --> 00:06:49,210
For each color, we
have the probability
112
00:06:49,210 --> 00:06:52,660
of that color raised to
the power of how many times
113
00:06:52,660 --> 00:06:57,280
that particular color
appears in a sequence.
114
00:06:57,280 --> 00:07:02,110
So any particular sequence of
this type has this probability.
115
00:07:02,110 --> 00:07:04,510
What we're interested
in is to find
116
00:07:04,510 --> 00:07:07,150
the total probability
of obtaining
117
00:07:07,150 --> 00:07:10,270
some sequence of this type.
118
00:07:10,270 --> 00:07:12,250
How can we find
this probability?
119
00:07:12,250 --> 00:07:14,170
Well, we will take
the probability
120
00:07:14,170 --> 00:07:16,780
of each sequence of this type--
121
00:07:16,780 --> 00:07:19,900
which is this much,
and it's the same
122
00:07:19,900 --> 00:07:22,190
for any particular sequence--
123
00:07:22,190 --> 00:07:28,160
and multiply with the number
of sequences of this type.
124
00:07:28,160 --> 00:07:32,050
So how many sequences are
there of a certain type?
125
00:07:32,050 --> 00:07:34,400
Let us look back at our example.
126
00:07:34,400 --> 00:07:37,450
We had seven trials.
127
00:07:37,450 --> 00:07:42,700
So let us number here
the different trials.
128
00:07:42,700 --> 00:07:46,630
And when I tell you that
a particular sequence was
129
00:07:46,630 --> 00:07:50,800
obtained, that's the
same as telling you
130
00:07:50,800 --> 00:08:00,190
that in this set of trials,
we had the first color.
131
00:08:03,780 --> 00:08:09,000
In this set of trials,
the fifth and sixth trial,
132
00:08:09,000 --> 00:08:13,360
we had the second color.
133
00:08:13,360 --> 00:08:19,870
And in this trial, the third
trial, we had the third color.
134
00:08:19,870 --> 00:08:22,630
This is an alternative
way of telling you
135
00:08:22,630 --> 00:08:24,500
what sequence we obtained.
136
00:08:24,500 --> 00:08:26,530
I tell you at
which trials we had
137
00:08:26,530 --> 00:08:29,170
the first color, at which trials
we had the second, at which
138
00:08:29,170 --> 00:08:32,251
trials we had the third.
139
00:08:32,251 --> 00:08:34,070
But What do we have here?
140
00:08:34,070 --> 00:08:40,929
Here we have a partition of the
set of numbers from 1 up to 7
141
00:08:40,929 --> 00:08:43,299
into three subsets.
142
00:08:43,299 --> 00:08:46,300
And the cardinalities
of those subsets
143
00:08:46,300 --> 00:08:52,540
are the numbers that appear here
in the type of the sequence.
144
00:08:52,540 --> 00:08:58,650
The conclusion is that a
sequence of certain type
145
00:08:58,650 --> 00:09:03,330
is equivalent, or can be
alternatively specified,
146
00:09:03,330 --> 00:09:11,190
by giving you a partition
over this set of tosses, which
147
00:09:11,190 --> 00:09:16,440
is the set from 1 up to n,
how many trials we've had,
148
00:09:16,440 --> 00:09:20,620
a partition into subsets
of certain sizes.
149
00:09:20,620 --> 00:09:23,820
So this allows us now
to count the number
150
00:09:23,820 --> 00:09:26,430
of sequences of a certain type.
151
00:09:26,430 --> 00:09:29,730
It's exactly the same as
the number of partitions,
152
00:09:29,730 --> 00:09:31,650
and we know what this is.
153
00:09:31,650 --> 00:09:34,500
And putting everything
together, the probability
154
00:09:34,500 --> 00:09:37,710
of obtaining a sequence
of a certain type
155
00:09:37,710 --> 00:09:41,820
is equal to the count
of how many sequences
156
00:09:41,820 --> 00:09:44,130
do we have of the
certain type, which
157
00:09:44,130 --> 00:09:47,660
is the same as the number of
partitions of a certain type,
158
00:09:47,660 --> 00:09:51,600
times the probability of
any particular sequence
159
00:09:51,600 --> 00:09:56,050
of that type that
we're interested in.
160
00:09:56,050 --> 00:09:58,980
So this is a formula
that generalizes the one
161
00:09:58,980 --> 00:10:03,240
that we saw before for the case
where we have only two colors,
162
00:10:03,240 --> 00:10:08,140
and which corresponded to
the coin tossing setting.
163
00:10:08,140 --> 00:10:09,690
And it is a useful
model, because you
164
00:10:09,690 --> 00:10:13,380
can think of many situations in
which you have repeated trials,
165
00:10:13,380 --> 00:10:17,130
and at each trial,
you obtain one out
166
00:10:17,130 --> 00:10:20,640
of a finite set of
possible results.
167
00:10:20,640 --> 00:10:23,190
There are different
possible results.
168
00:10:23,190 --> 00:10:25,560
You repeat those
trials independently.
169
00:10:25,560 --> 00:10:30,010
And you may be interested in
the question of how many results
170
00:10:30,010 --> 00:10:34,657
of the first kind, of the second
kind, and so on there will be.