3.4 Error Estimates for the Monte Carlo Method

The Error in Estimating the Variance

The variance of $$y$$ is given the symbol, $$\sigma _ y^2$$, and is defined as,

 $\sigma _ y^2 = E[(y-\mu _ y)^2] \label{equ:variance}$ (3.52)

We saw in the previous unit that an unbiased estimator of $$\sigma _ y^2$$ is $$s_ y^2$$, that is:

 $E[s_ y^2] = \sigma _ y^2.$ (3.53)

Note, you should try proving this result.

To quantify the uncertainty in this estimator, we would like to determine the standard error,

 $\sigma _{s^2_ y} \equiv \left\{ E\left[(s_ y^2-\sigma _ y^2)^2\right]\right\} ^{1/2}.$ (3.54)

Unfortunately, this standard error is not known for general distributions of $$y$$. However, if $$y$$ has a normal distribution, then,

 $\sigma _{s^2_ y} = \frac{\sigma _ y^2}{\sqrt {N/2}} \label{equ:se_ s2y_ normal}$ (3.55)

Under the assumption of $$y$$ being normally distributed, the distribution of $$s^2_ y$$ is also related to the chi-squared distribution. Specifically, $$(N-1)s^2_ y/\sigma ^2_ y$$ has a chi-square distribution with $$N-1$$ degrees of freedom. Note that the requirement that $$y$$ be normally distributed is much more restrictive than the requirements for the mean error estimates to hold. For the mean error estimates, the standard error, $$\sigma _{\overline{y}} = \sigma _ y/\sqrt {N}$$, is exact regardless of the distribution of $$y$$. The application of the central limit theorem which gives that$$\overline{y}$$ is normally distributed only requires that the number of samples is large but does not constrain the distribution of $$y$$ itself (beyond requiring that $$f(y)$$ is continuous).

Standard Deviation

Typically, the standard deviation of $$y$$ is estimated using $$s_ y$$, i.e. the square root of the variance estimator. This estimate, however, is biased,

 $E[s_ y] \neq \sigma _ y.$ (3.56)

The standard error for this estimate is only known exactly when $$y$$ is normally distributed. In that case,

 $\sigma _{s_ y} \equiv \left\{ E[(s_ y-\sigma _ y)^2]\right\} ^{1/2} = \frac{\sigma _ y}{\sqrt {2N}}.$ (3.57)