5.05a, 5.05b - Sample means and unbiased estimates

You already know

For independent observations, expectations and variances add. Multiplying a random variable by $a$ multiplies its variance by $a^2$. A normal distribution is written $\mathrm{N}(\mu, \sigma^2)$, with variance in the second position.

Sample mean and sampling distribution

For a randomly and independently selected sample $X_1,\ldots,X_n$ from one population, the sample mean is $\overline X = (X_1 + \cdots + X_n)/n$. Its sampling distribution describes the values of $\overline X$ across repeated samples of that same size. After observing values $x_1,\ldots,x_n$, the calculated value is $\overline x$.

Moments of a sample mean

For independent observations from a population with mean $\mu$ and finite variance $\sigma^2$,

These results hold for every positive sample size $n$; normality is not required.

In the formula booklet.

Standard error of the mean

Under the same independent-sampling conditions,

This is the standard error of the sample mean. It is the square root of the sample-mean variance printed in the booklet, rather than a separately printed formula.

Not in the formula booklet: learn it.

Worked example 1: averaging thirty-six readings

Question. Independent readings have population mean $100$ and variance $144$. Find the expectation, variance and standard error of the mean of $36$ readings.

Plan. Keep the mean, divide the variance by the sample size, then take its square root.

Why this step? The spread needed is the spread of repeated averages, not the spread of individual readings.

Answer. The expectation is $100$, the variance is $4$, and the standard error is $2$.

Check. $12/\sqrt{36}=2$. The standard error is one sixth of the individual standard deviation.

Worked example 2: quadrupling the sample size

Question. The same population has mean $100$ and variance $144$. Find the expectation, variance and standard error of the mean of $144$ independent readings.

Plan. Use the new sample size in the same sampling formulae.

Why this step? Four times as many readings divide the variance by $4$, so they divide its square root by $2$.

Answer. The expectation is $100$, the variance is $1$, and the standard error is $1$.

Check. Relative to $36$ readings, $144$ readings halve the standard error from $2$ to $1$. Twice the precision needs four times the data.

Watch out

Wrong: $\mathrm{sd}(\overline X)=\sigma/n$. Right: $\mathrm{Var}(\overline X)=\sigma^2/n$, so $\mathrm{sd}(\overline X)=\sigma/\sqrt n$. Repeating one reading $n$ times does not create an independent sample and does not reduce its uncertainty.

When a sample mean is normal

For a random independent sample of size $n$:

• If $X\sim\mathrm{N}(\mu,\sigma^2)$, then $\overline X\sim\mathrm{N}(\mu,\sigma^2/n)$ exactly, for every $n$.
• For any population with mean $\mu$ and finite positive variance $\sigma^2$, $\overline X$ is approximately normal when $n$ is sufficiently large. Use $\overline X\approx\mathrm{N}(\mu,\sigma^2/n)$.

The second statement is the central limit theorem. A working guide is approximately $n>25$; this is not a guarantee for every severely skewed population. The results, rather than their proofs, are required.

In the formula booklet.

Worked example 3: an exactly normal average

Question. A random variable has distribution $\mathrm{N}(24,100)$. For $100$ independent observations, find the probability that the sample mean exceeds $26$.

Plan. Write the exact distribution of the mean and standardise the threshold.

Why this step? The standard deviation in the denominator is $1$, the standard error, rather than $10$, the individual standard deviation.

Answer. The probability is $0.0228$, to $3$ significant figures.

Check. The threshold is two standard errors above the mean, so a small upper-tail probability is sensible.

Worked example 4: a large sample from a non-normal population

Question. A population has mean $24$ and variance $100$, but is not normally distributed. For a random independent sample of size $100$, estimate the probability that the sample mean exceeds $26$.

Plan. Use the central limit theorem for the sample mean, then use its exact mean and variance in the normal approximation.

Why this step? The central limit theorem justifies the normal shape of $\overline X$. It does not justify the variance formula, which already holds for independent samples of every size.

Answer. The estimated probability is $0.0228$.

Check. The same mean, variance and sample size give the same numerical normal calculation as before. Only its exact or approximate status has changed.