WEBVTT

00:00:00.220 --> 00:00:03.420
Let us now discuss a little bit
the simplest estimation

00:00:03.420 --> 00:00:06.740
problem that there is, the
problem of estimating the mean

00:00:06.740 --> 00:00:10.130
of a certain probability
distribution, and we will take

00:00:10.130 --> 00:00:13.830
this occasion to introduce some
additional terminology

00:00:13.830 --> 00:00:17.430
and discuss some desirable
properties of estimators.

00:00:17.430 --> 00:00:19.620
So the context is as follows.

00:00:19.620 --> 00:00:23.390
We have n random variables
that are independent, and

00:00:23.390 --> 00:00:24.880
they're identically
distributed.

00:00:24.880 --> 00:00:27.090
They are drawn from some
distribution that has a

00:00:27.090 --> 00:00:30.560
certain mean theta and
some variance.

00:00:30.560 --> 00:00:33.970
We assume that we do not know
the value of the mean, and we

00:00:33.970 --> 00:00:35.490
want to estimate it.

00:00:35.490 --> 00:00:39.090
The most natural way of
estimating the mean is to form

00:00:39.090 --> 00:00:42.760
the sample mean, that is, we
take the n observations and

00:00:42.760 --> 00:00:45.530
take their average.

00:00:45.530 --> 00:00:51.040
Notice, that this quantity, the
sample mean or, in this

00:00:51.040 --> 00:00:54.900
case, it is the estimator that
we're using, is a random

00:00:54.900 --> 00:00:59.370
variable because its value is
determined by the values of

00:00:59.370 --> 00:01:03.500
the random variables
X1 up to Xn.

00:01:03.500 --> 00:01:06.880
Let us discuss some properties
of this estimator.

00:01:06.880 --> 00:01:11.070
The first property is that the
expected value of this

00:01:11.070 --> 00:01:13.720
estimator is equal
to the true mean.

00:01:13.720 --> 00:01:16.870
This is because the expected
value of each one of the Xs is

00:01:16.870 --> 00:01:20.140
theta, and therefore, the
expected value of this ratio

00:01:20.140 --> 00:01:22.390
is theta as well.

00:01:22.390 --> 00:01:26.240
Now, this is a relation
that's true for all

00:01:26.240 --> 00:01:28.130
possible values of theta.

00:01:31.270 --> 00:01:34.800
Let us appreciate the content
of this statement.

00:01:34.800 --> 00:01:38.509
Let us think what this
expectation actually is.

00:01:38.509 --> 00:01:41.390
More generally, suppose that
we're dealing with some

00:01:41.390 --> 00:01:46.210
estimator, which is some
function of the data.

00:01:46.210 --> 00:01:53.050
Then, the expected value of this
estimator is using the

00:01:53.050 --> 00:01:56.340
expected value rule, and
assuming that we're dealing

00:01:56.340 --> 00:02:01.140
with a discrete random variable
X, the expected value

00:02:01.140 --> 00:02:04.315
of theta hat is determined
as follows.

00:02:07.940 --> 00:02:12.050
And so we see that the expected
value for estimator

00:02:12.050 --> 00:02:15.310
depends, or is affected,
by what the true

00:02:15.310 --> 00:02:17.210
value of theta is.

00:02:17.210 --> 00:02:20.400
So this is a quantity that
generally depends on theta.

00:02:20.400 --> 00:02:24.660
And what we want in order to
have a so-called unbiased

00:02:24.660 --> 00:02:29.130
estimator is that no matter
what theta is, this

00:02:29.130 --> 00:02:33.760
expectation evaluates
to the true unknown

00:02:33.760 --> 00:02:37.170
value equal to theta.

00:02:37.170 --> 00:02:40.760
In general, having this
property, having an unbiased

00:02:40.760 --> 00:02:43.370
estimator, is a desirable one.

00:02:43.370 --> 00:02:47.070
We do not want our estimates to
be systematically high or

00:02:47.070 --> 00:02:49.840
systematically low, no
matter what the true

00:02:49.840 --> 00:02:52.740
value of theta is.

00:02:52.740 --> 00:02:54.510
A second property of
the sample mean

00:02:54.510 --> 00:02:56.630
estimator is the following.

00:02:56.630 --> 00:03:00.010
By the weak law of large
numbers, we know that the

00:03:00.010 --> 00:03:04.890
sample mean converges to the
true mean in the sense of

00:03:04.890 --> 00:03:07.670
convergence in probability.

00:03:07.670 --> 00:03:13.450
Once more, this is a property
that's true, no matter what

00:03:13.450 --> 00:03:18.570
the underlying unknown value
little theta is.

00:03:18.570 --> 00:03:23.050
When this is true, this
convergence is true, for all

00:03:23.050 --> 00:03:27.070
values of little theta, then we
say that our estimator is

00:03:27.070 --> 00:03:30.650
consistent or that we
have consistency.

00:03:30.650 --> 00:03:33.320
Having a consistent estimator
is definitely a

00:03:33.320 --> 00:03:35.240
very desirable property.

00:03:35.240 --> 00:03:39.220
We would like, when we obtain
more and more data, that our

00:03:39.220 --> 00:03:43.350
estimator will give us
the correct value.

00:03:43.350 --> 00:03:46.690
Finally, we would like to say
something about the size of

00:03:46.690 --> 00:03:48.670
the estimation error.

00:03:48.670 --> 00:03:50.290
This is measured--

00:03:50.290 --> 00:03:53.920
one way of measuring it, but
which is the most common, it's

00:03:53.920 --> 00:03:57.460
measured in terms of the
mean squared error.

00:03:57.460 --> 00:03:59.420
So theta is the unknown value.

00:03:59.420 --> 00:04:00.840
This is our estimator.

00:04:00.840 --> 00:04:02.070
This is the error.

00:04:02.070 --> 00:04:05.680
We square the error, and
we take the average.

00:04:05.680 --> 00:04:09.810
What we've got here for this
specific example of the sample

00:04:09.810 --> 00:04:12.670
mean estimator is
the following.

00:04:12.670 --> 00:04:17.279
Since it is unbiased, we have
a random variable minus the

00:04:17.279 --> 00:04:21.870
mean of that random variable,
so this is just the variance

00:04:21.870 --> 00:04:23.120
of the estimator.

00:04:26.240 --> 00:04:30.770
And for the sample mean, we
know that its variance is

00:04:30.770 --> 00:04:33.200
sigma squared over n.

00:04:33.200 --> 00:04:37.520
So this gives us some very
specific knowledge about how

00:04:37.520 --> 00:04:41.320
the mean squared error behaves
as we change n.

00:04:41.320 --> 00:04:45.600
In this particular example, the
mean squared error did not

00:04:45.600 --> 00:04:47.540
depend on theta.

00:04:47.540 --> 00:04:51.040
It's the same no matter what
the true theta is.

00:04:51.040 --> 00:04:54.340
But in other situations and
with other estimators, you

00:04:54.340 --> 00:04:57.280
might actually obtain here
a function of theta.