1
00:00:00,640 --> 00:00:03,900
We have already introduced the
concept of the conditional PMF
2
00:00:03,900 --> 00:00:08,720
of a random variable, X, given
an event A. We will now
3
00:00:08,720 --> 00:00:12,150
consider the case where we
condition on the value of
4
00:00:12,150 --> 00:00:18,690
another random variable Y. That
is, we let A be the event
5
00:00:18,690 --> 00:00:22,500
that some other random variable,
Y, takes on a
6
00:00:22,500 --> 00:00:26,310
specific value, little y.
7
00:00:26,310 --> 00:00:28,760
In this case, we're talking
about a conditional
8
00:00:28,760 --> 00:00:33,240
probability of the
form shown here.
9
00:00:33,240 --> 00:00:34,420
The conditional probability--
10
00:00:34,420 --> 00:00:38,110
that X takes on a specific
value, given that the random
11
00:00:38,110 --> 00:00:42,160
variable Y takes on another
specific value.
12
00:00:42,160 --> 00:00:46,300
And we use this notation to
indicate those conditional
13
00:00:46,300 --> 00:00:48,200
probabilities.
14
00:00:48,200 --> 00:00:52,750
As usual, the subscripts
indicate the situation that
15
00:00:52,750 --> 00:00:54,070
we're dealing with.
16
00:00:54,070 --> 00:00:56,280
That is, we're dealing with the
distribution of the random
17
00:00:56,280 --> 00:00:59,310
variable X and we're
conditioning on values of the
18
00:00:59,310 --> 00:01:02,260
other random variable, Y.
19
00:01:02,260 --> 00:01:05,990
Using the definition now of
conditional probabilities this
20
00:01:05,990 --> 00:01:10,020
can be written as the
probability that both events
21
00:01:10,020 --> 00:01:15,450
happen divided by the
probability of the
22
00:01:15,450 --> 00:01:16,700
conditioning event.
23
00:01:21,840 --> 00:01:25,490
We can turn this expression
into PMF notation.
24
00:01:25,490 --> 00:01:28,330
And this leads us to
this definition
25
00:01:28,330 --> 00:01:30,390
of conditional PMFs.
26
00:01:30,390 --> 00:01:34,289
The conditional PMF is defined
to be the ratio
27
00:01:34,289 --> 00:01:35,900
of the joint PMF--
28
00:01:35,900 --> 00:01:38,440
this is the probability
that we have here--
29
00:01:38,440 --> 00:01:41,330
by the corresponding
marginal PMF.
30
00:01:41,330 --> 00:01:44,200
And this is the probability
that we have here.
31
00:01:44,200 --> 00:01:47,190
Now, remember that conditional
probabilities are only defined
32
00:01:47,190 --> 00:01:51,210
when the conditioning event has
a positive probability,
33
00:01:51,210 --> 00:01:53,770
when this denominator
is positive.
34
00:01:53,770 --> 00:01:58,200
Similarly, the conditional PMF
will only be defined for those
35
00:01:58,200 --> 00:02:03,990
little y that have positive
probability of occurring.
36
00:02:03,990 --> 00:02:08,680
Now, the conditional PMF is a
function of two arguments,
37
00:02:08,680 --> 00:02:10,570
little x and little y.
38
00:02:10,570 --> 00:02:14,500
But the best way of thinking
about the conditional PMF is
39
00:02:14,500 --> 00:02:21,270
that we fix the value, little
y, and then view this
40
00:02:21,270 --> 00:02:25,100
expression here as
a function of x.
41
00:02:25,100 --> 00:02:28,390
As a function of x, it gives
us the probabilities of the
42
00:02:28,390 --> 00:02:32,690
different x's that may occur in
the conditional universe.
43
00:02:32,690 --> 00:02:37,480
And these probabilities must,
of course, sum to 1.
44
00:02:37,480 --> 00:02:39,579
Again, we're keeping y fixed.
45
00:02:39,579 --> 00:02:42,510
We live in a conditional
universe where y takes on a
46
00:02:42,510 --> 00:02:43,870
specific value.
47
00:02:43,870 --> 00:02:46,110
And here we have the
probabilities of the different
48
00:02:46,110 --> 00:02:47,840
x's in that universe.
49
00:02:47,840 --> 00:02:49,385
And these sum to 1.
50
00:02:52,130 --> 00:02:56,370
Note that if we change the value
of little y, we will, of
51
00:02:56,370 --> 00:02:59,740
course, get a different
conditional PMF for the random
52
00:02:59,740 --> 00:03:04,090
variable X. So what we're really
dealing with in this
53
00:03:04,090 --> 00:03:08,910
instance is that we have a
family of conditional PMFs,
54
00:03:08,910 --> 00:03:13,500
one conditional PMF for every
possible value of little y.
55
00:03:13,500 --> 00:03:16,170
And for every possible value
of little y, we have a
56
00:03:16,170 --> 00:03:20,380
legitimate PMF who's
values add to 1.
57
00:03:20,380 --> 00:03:22,320
Let's look at an example.
58
00:03:22,320 --> 00:03:25,630
Consider the joint PMF
given in this table.
59
00:03:25,630 --> 00:03:31,690
Let us condition on the event
that Y is equal to 2, which
60
00:03:31,690 --> 00:03:34,480
corresponds to this row
in the diagram.
61
00:03:39,520 --> 00:03:44,829
We need to know the value of the
marginal at this point, so
62
00:03:44,829 --> 00:03:50,800
we start by calculating the
probability of Y at value 2.
63
00:03:50,800 --> 00:03:53,560
And this is found by adding
the entries in
64
00:03:53,560 --> 00:03:55,400
this row of the table.
65
00:03:55,400 --> 00:03:58,920
And we find that this
is 5 over 20.
66
00:03:58,920 --> 00:04:01,700
Then we can start calculating
entries of
67
00:04:01,700 --> 00:04:03,730
the conditional PMF.
68
00:04:03,730 --> 00:04:09,230
So for example, the probability
that X takes on
69
00:04:09,230 --> 00:04:14,580
the value of 1 given that Y
takes the value of 2, it is
70
00:04:14,580 --> 00:04:20,320
going to be this entry, which
is 0, divided by 5/20, which
71
00:04:20,320 --> 00:04:22,360
gives us 0.
72
00:04:22,360 --> 00:04:27,710
We can find the next entry, the
probability of X taking
73
00:04:27,710 --> 00:04:33,180
the value of 2, given that Y
takes the value of 2 will be
74
00:04:33,180 --> 00:04:38,400
this entry, 1/20 divided
by 5/20.
75
00:04:38,400 --> 00:04:39,730
So it's going to be 1/5.
76
00:04:42,990 --> 00:04:46,180
And we can continue with
the other two entries.
77
00:04:46,180 --> 00:04:53,490
And we can actually even plot
the result once we're done.
78
00:04:53,490 --> 00:04:59,560
And what we have is that at 1,
we have a probability of 0.
79
00:04:59,560 --> 00:05:02,860
At 2, we have a probability
of 1/5.
80
00:05:07,588 --> 00:05:13,200
At 3, we have a probability
of 3/20 divided
81
00:05:13,200 --> 00:05:19,070
5/20, which is 3/5.
82
00:05:19,070 --> 00:05:23,665
And at 4, we have, again,
a probability of 1/5.
83
00:05:26,520 --> 00:05:31,040
So what we have plotted here
is the conditional PMF.
84
00:05:31,040 --> 00:05:36,060
It's a PMF in the variable x,
where x ranges over the
85
00:05:36,060 --> 00:05:40,710
possible values, but where we
have fixed the value of y to
86
00:05:40,710 --> 00:05:43,120
be equal to 2.
87
00:05:43,120 --> 00:05:46,560
Now, we could have found this
conditional PMF even faster
88
00:05:46,560 --> 00:05:50,600
without doing any divisions
by following the intuitive
89
00:05:50,600 --> 00:05:53,200
argument that we have
used before.
90
00:05:53,200 --> 00:05:55,740
We live in this conditional
universe.
91
00:05:55,740 --> 00:05:59,150
We have conditioned on
Y being equal to 2.
92
00:05:59,150 --> 00:06:03,740
The conditional probabilities
will have the same proportions
93
00:06:03,740 --> 00:06:06,660
as the original probabilities,
except that they needed to be
94
00:06:06,660 --> 00:06:09,050
scaled so that they add to 1.
95
00:06:09,050 --> 00:06:13,160
So they should be in the
proportions of 0, 1, 3, 1.
96
00:06:13,160 --> 00:06:17,670
And for these to add to 1, we
need to put everywhere a
97
00:06:17,670 --> 00:06:19,640
denominator of 5.
98
00:06:19,640 --> 00:06:24,470
So the proportions are indeed
0, 1, 3, and 1.
99
00:06:24,470 --> 00:06:30,730
Pictorially, the conditional PMF
has the same form as the
100
00:06:30,730 --> 00:06:36,640
corresponding slice of the joint
PMF, except, again, that
101
00:06:36,640 --> 00:06:40,159
the entries of that slice are
renormalized so that the
102
00:06:40,159 --> 00:06:42,810
entries add to 1.
103
00:06:42,810 --> 00:06:45,480
And finally, an observation--
104
00:06:45,480 --> 00:06:49,280
we can take the definition of
the conditional PMF and turn
105
00:06:49,280 --> 00:06:53,300
it around by moving the
denominator to the other side
106
00:06:53,300 --> 00:06:57,040
and obtain a formula, which
is a version of the
107
00:06:57,040 --> 00:06:59,450
multiplication rule.
108
00:06:59,450 --> 00:07:03,850
The probability that X takes a
value little x and Y takes a
109
00:07:03,850 --> 00:07:07,880
value little y is the product
or the probability that Y
110
00:07:07,880 --> 00:07:11,080
takes this particular value
times the conditional
111
00:07:11,080 --> 00:07:15,420
probability that X takes on the
particular value little x,
112
00:07:15,420 --> 00:07:19,970
given that Y takes on the
particular value little y.
113
00:07:19,970 --> 00:07:23,960
We also have a symmetrical
relationship if we interchange
114
00:07:23,960 --> 00:07:29,430
the roles of X and Y. As we
discussed earlier in this
115
00:07:29,430 --> 00:07:32,520
course, the multiplication rule
can be used to specify
116
00:07:32,520 --> 00:07:34,210
probability models.
117
00:07:34,210 --> 00:07:36,990
One way of modeling two
random variables is by
118
00:07:36,990 --> 00:07:39,580
specifying the joint PMF.
119
00:07:39,580 --> 00:07:43,060
But we now have an alternative,
indirect, way
120
00:07:43,060 --> 00:07:45,260
using the multiplication rule.
121
00:07:45,260 --> 00:07:49,580
We can first specify the
distribution of Y and then
122
00:07:49,580 --> 00:07:54,200
specify the conditional
PMF of X for any given
123
00:07:54,200 --> 00:07:56,510
value of little y.
124
00:07:56,510 --> 00:08:00,540
And this completely determines
the joint PMF, and so we have
125
00:08:00,540 --> 00:08:04,560
a full probability model.
126
00:08:04,560 --> 00:08:07,540
We can also provide similar
definitions of conditional
127
00:08:07,540 --> 00:08:10,460
PMFs for the case where we're
dealing with more than two
128
00:08:10,460 --> 00:08:12,120
random variables.
129
00:08:12,120 --> 00:08:16,080
In this case, notation is
pretty self-explanatory.
130
00:08:16,080 --> 00:08:21,930
By looking at this expression
here, you can probably guess
131
00:08:21,930 --> 00:08:26,230
that this stands for the
probability that random
132
00:08:26,230 --> 00:08:31,720
variable X takes on a specific
value, conditional on the
133
00:08:31,720 --> 00:08:36,380
random variables Y and
Z taking on some
134
00:08:36,380 --> 00:08:38,520
other specific values.
135
00:08:38,520 --> 00:08:41,429
Using the definition of
conditional probabilities,
136
00:08:41,429 --> 00:08:48,270
this is the probability that all
events happen divided by
137
00:08:48,270 --> 00:08:51,810
the probability of the
conditioning event, which, in
138
00:08:51,810 --> 00:08:56,930
our case, is the event that Y
takes on a specific value and
139
00:08:56,930 --> 00:09:01,280
simultaneously, Z takes another
specific value.
140
00:09:01,280 --> 00:09:07,080
In PMF notation, this is the
ratio of the joint PMF of the
141
00:09:07,080 --> 00:09:12,000
three random variables together,
divided by the joint
142
00:09:12,000 --> 00:09:19,670
PMF of the two random variables
Y and Z. As another
143
00:09:19,670 --> 00:09:24,280
example, we could have an
expression like this, which,
144
00:09:24,280 --> 00:09:28,370
again, stands for the
probability that these two
145
00:09:28,370 --> 00:09:35,120
random variables take on
specific values, conditional
146
00:09:35,120 --> 00:09:41,530
on this random variable taking
on another value.
147
00:09:41,530 --> 00:09:45,020
Finally, we can have versions of
the multiplication rule for
148
00:09:45,020 --> 00:09:47,460
the case where we're
dealing with more
149
00:09:47,460 --> 00:09:49,360
than two random variables.
150
00:09:49,360 --> 00:09:51,770
Recall the usual multiplication
rule.
151
00:09:51,770 --> 00:09:55,580
For three events happening
simultaneously, let's apply
152
00:09:55,580 --> 00:09:59,430
this multiplication rule for the
case where the event, A,
153
00:09:59,430 --> 00:10:04,490
stands for the event that the
random variable X takes on a
154
00:10:04,490 --> 00:10:06,050
specific value.
155
00:10:06,050 --> 00:10:11,160
Let B be the event that Y takes
on a specific value, and
156
00:10:11,160 --> 00:10:15,600
C be the event that the random
variable Z takes
157
00:10:15,600 --> 00:10:17,470
on a specific value.
158
00:10:17,470 --> 00:10:20,720
Then we can take this relation,
the multiplication
159
00:10:20,720 --> 00:10:23,820
rule, and translate it
into PMF notation.
160
00:10:23,820 --> 00:10:27,470
The probability that all three
events happen is equal to the
161
00:10:27,470 --> 00:10:32,690
product of the probability that
the first event happens.
162
00:10:32,690 --> 00:10:36,000
Then we have the conditional
probability that the second
163
00:10:36,000 --> 00:10:39,550
event happens given that the
first happened, times the
164
00:10:39,550 --> 00:10:43,270
conditional probability that
the third event happens--
165
00:10:43,270 --> 00:10:45,210
this one-- given that the first
166
00:10:45,210 --> 00:10:46,650
two events have happened.