WEBVTT

00:00:00.390 --> 00:00:03.920
Just in order to get some more
familiarity with joint PDFs,

00:00:03.920 --> 00:00:06.770
let us look at independent
normals.

00:00:06.770 --> 00:00:10.330
Actually, this is an important
example because noise is often

00:00:10.330 --> 00:00:13.580
modeled by normal random
variables, and noise terms

00:00:13.580 --> 00:00:16.870
that show up at different
parts of a system, or at

00:00:16.870 --> 00:00:20.300
different times, are often
assumed to be independent.

00:00:20.300 --> 00:00:22.570
Suppose that we have two
standard normal random

00:00:22.570 --> 00:00:26.510
variables, X and Y, with zero
means and unit variances.

00:00:26.510 --> 00:00:29.700
If their independent, their
joint PDF is the product of

00:00:29.700 --> 00:00:32.980
the marginal PDFs and
takes this form.

00:00:32.980 --> 00:00:37.640
This is just the PDF of a
standard normal X and the PDF

00:00:37.640 --> 00:00:40.830
of a standard normal Y
and we multiply them.

00:00:40.830 --> 00:00:45.400
If we are to plot this joint
PDF we obtain this figure.

00:00:45.400 --> 00:00:50.070
It looks like a bell which is
centered at the origin--

00:00:50.070 --> 00:00:53.500
at the point with coordinates
zero, zero.

00:00:53.500 --> 00:00:58.150
One way to think about what is
going on here is to rewrite

00:00:58.150 --> 00:01:05.980
this expression as 1 over 2pi,
and then the exponential of

00:01:05.980 --> 00:01:10.830
minus 1/2 x squared
plus y squared.

00:01:14.900 --> 00:01:22.250
If we look at the unit circle in
xy space, which is the set

00:01:22.250 --> 00:01:28.210
of points at which x squared
plus y squared is equal to 1,

00:01:28.210 --> 00:01:34.150
then, on that circle, the PDF
takes a constant value because

00:01:34.150 --> 00:01:37.370
this quantity is constant
on that circle.

00:01:37.370 --> 00:01:40.789
And the same is true for
any other circle.

00:01:40.789 --> 00:01:44.870
On any circle the PDF takes a
constant value, of course, a

00:01:44.870 --> 00:01:46.450
different constant.

00:01:46.450 --> 00:01:49.991
So the circles centered at the
origin are the so-called

00:01:49.991 --> 00:01:52.259
contours of the joint PDF.

00:01:52.259 --> 00:01:57.780
On each contour the joint
PDF is a constant.

00:01:57.780 --> 00:01:59.830
Let us now generalize.

00:01:59.830 --> 00:02:03.110
Consider two independent normal
random variables, but

00:02:03.110 --> 00:02:09.340
with general means mu x and mu
y, and variances sigma x

00:02:09.340 --> 00:02:12.660
squared and sigma y squared.

00:02:12.660 --> 00:02:16.870
The joint is, again, the product
of the marginal PDFs

00:02:16.870 --> 00:02:19.760
and, therefore, takes
this form.

00:02:19.760 --> 00:02:24.410
This looks intimidating but, in
fact, it is pretty simple.

00:02:24.410 --> 00:02:28.060
This part is just a normalizing
constant.

00:02:28.060 --> 00:02:30.770
It is the constant that's needed
so that the joint PDF

00:02:30.770 --> 00:02:33.500
integrates to 1.

00:02:33.500 --> 00:02:37.460
What we have here is the
negative exponential of a

00:02:37.460 --> 00:02:40.680
quadratic function of x and y.

00:02:40.680 --> 00:02:43.710
Let us plot the contours
of this quadratic.

00:02:43.710 --> 00:02:46.600
Remember that contour is the
set of points where the

00:02:46.600 --> 00:02:49.610
quadratic takes a
constant value.

00:02:49.610 --> 00:02:52.480
And by consequence,
the joint PDF also

00:02:52.480 --> 00:02:54.910
takes a constant value.

00:02:54.910 --> 00:02:58.740
If you have set this quadratic
to a constant, what you have

00:02:58.740 --> 00:03:01.320
is the equation that describes
an ellipse.

00:03:01.320 --> 00:03:05.300
And it is an ellipse whose
principal axes run along the x

00:03:05.300 --> 00:03:09.490
and y directions, and those
ellipses are all centered at

00:03:09.490 --> 00:03:13.590
this particular point,
mu x, mu y.

00:03:13.590 --> 00:03:19.390
The joint PDF is largest when
the exponent is equal to zero.

00:03:19.390 --> 00:03:23.270
And this happens when x is
equal to mu x, and y

00:03:23.270 --> 00:03:24.760
is equal to mu y.

00:03:24.760 --> 00:03:27.450
That is, right at the center
of the ellipse.

00:03:27.450 --> 00:03:31.150
That's where the joint
PDF is largest.

00:03:31.150 --> 00:03:35.470
As you move to ellipses that are
further out on this outer

00:03:35.470 --> 00:03:38.329
ellipse, this expression
is a constant.

00:03:38.329 --> 00:03:41.100
It's the exponential
of the negative of

00:03:41.100 --> 00:03:43.260
some positive numbers.

00:03:43.260 --> 00:03:47.540
So you get a smaller value
for the joint PDF.

00:03:47.540 --> 00:03:50.860
If you move to a further ellipse
further out, then

00:03:50.860 --> 00:03:54.030
again, the joint PDF will be a
constant, but it's going to be

00:03:54.030 --> 00:03:56.770
a smaller constant.

00:03:56.770 --> 00:04:00.240
Now, for the case of standard
normals, the joint PDF was

00:04:00.240 --> 00:04:02.300
circularly symmetric.

00:04:02.300 --> 00:04:06.300
The contours were actually
circles, instead of ellipses.

00:04:06.300 --> 00:04:08.600
But this is not the
case in general.

00:04:08.600 --> 00:04:13.390
For example, suppose that the
variance of Y is bigger than

00:04:13.390 --> 00:04:18.100
the variance of X. Then you get
a shape as the one shown

00:04:18.100 --> 00:04:19.700
in this figure.

00:04:19.700 --> 00:04:24.070
Since the variance of Y is
larger, we expect Y to take

00:04:24.070 --> 00:04:29.300
values over a bigger range,
and to be larger typically

00:04:29.300 --> 00:04:33.630
than the values of X. And so
the bell shape that we have

00:04:33.630 --> 00:04:37.490
for the joint PDF is stretched
in the y direction.

00:04:37.490 --> 00:04:41.120
It extends further out in the
y direction than it does in

00:04:41.120 --> 00:04:43.159
the x direction.

00:04:43.159 --> 00:04:47.290
To conclude, the joint PDF of
two independent normals has

00:04:47.290 --> 00:04:49.240
the shape of a bell.

00:04:49.240 --> 00:04:53.130
The center of the bell is
determined by the means.

00:04:53.130 --> 00:04:56.300
Furthermore, the bell is
stretched in the x and y

00:04:56.300 --> 00:05:00.150
directions by an amount that is
determined by the variances

00:05:00.150 --> 00:05:02.310
of x and y.

00:05:02.310 --> 00:05:05.580
However, the stretching
is always along

00:05:05.580 --> 00:05:07.960
the coordinate axes.

00:05:07.960 --> 00:05:11.610
If you wanted a bell that
stretches in some diagonal

00:05:11.610 --> 00:05:15.880
direction, or if you have
contours that are ellipses but

00:05:15.880 --> 00:05:20.790
with some different kinds of
axes, then you will have

00:05:20.790 --> 00:05:23.740
dependence between the
two random variables.

00:05:23.740 --> 00:05:26.710
In that case, we will be dealing
with a so-called

00:05:26.710 --> 00:05:31.470
bivariate normal distribution,
but we will not pursue this

00:05:31.470 --> 00:05:33.130
any further at this point.