FS2 2.3 - Mean and variance of continuous variables

FS2 2.3 - Mean and variance of continuous variables

A density describes the whole distribution; its mean and variance summarise location and spread. You will calculate these summaries by integration, find the expected value of a function of a continuous variable, and use your results to judge how well a model describes observations.

Average values using probability as the weight

If a machine's mean reading is 23\frac23, is its mean squared reading necessarily 49\frac49? Squaring the average feels natural. Yet a machine with density f(x)=2xf(x)=2x on [0,1][0,1] has mean squared reading 12\frac12, which is larger. You will find exactly where that extra amount comes from.

You already know

A continuous probability is an area under a density. The density is non-negative, its total area is 11, and definite integration uses upper minus lower antiderivative values.

Imagine dividing the support into very narrow strips. A strip near xx has probability approximately f(x) Δxf(x)\,\Delta x. Its contribution to a weighted average is therefore approximately xf(x) Δxx f(x)\,\Delta x. Adding all such contributions gives an integral.

Expectation and variance

The expectation E(X)=μ\mathrm E(X)=\mu is the probability-weighted mean of XX. The variance Var⁡(X)=σ2\operatorname{Var}(X)=\sigma^2 is the expectation of the squared distance from that mean. For example, positive and negative readings can give mean zero while still having positive variance. Zero mean does not mean zero spread.

The mean is not the average height of the density curve. The curve supplies the weights for averaging the horizontal values.

Mean of a continuous random variable

For a density ff, when the expectation exists,

E(X)=μ=∫−∞∞xf(x) dx\mathrm E(X)=\mu=\int_{-\infty}^{\infty}x f(x)\,\mathrm{d}x

Integrate only over the support, splitting the integral if its density has several formulas.

In the formula booklet.

There is no additional division by the support's width: the weights already add to 11. This differs from the mean value of an ordinary function over an interval. For a variable restricted to [a,b][a,b], a finite mean lies in [a,b][a,b], providing a useful check.

Not every continuous model has a finite mean. For example, the valid density 2x−22x^{-2} on [2,∞)[2,\infty) gives ∫2∞x(2x−2) dx=∫2∞2/x dx\int_2^\infty x(2x^{-2})\,\mathrm{d}x=\int_2^\infty2/x\,\mathrm{d}x, which diverges. Do not force a finite answer from a divergent integral.

Key idea

Expectation averages values using probability weights. For a continuous variable, “value times probability” becomes xf(x) dxx f(x)\,\mathrm{d}x.

Choose the integral that uses both ingredients.

Find the first two moments before the variance

The second moment E(X2)\mathrm E(X^2) is the mean of squared values. It is different from the square of the mean, [E(X)]2[\mathrm E(X)]^2; their difference measures spread.

Variance and the second moment

Provided the required moments are finite,

Var⁡(X)=∫−∞∞(x−μ)2f(x) dx=E(X2)−μ2\operatorname{Var}(X) =\int_{-\infty}^{\infty}(x-\mu)^2 f(x)\,\mathrm{d}x =\mathrm E(X^2)-\mu^2

where

E(X2)=∫−∞∞x2f(x) dx\mathrm E(X^2)=\int_{-\infty}^{\infty}x^2 f(x)\,\mathrm{d}x

The standard deviation is the non-negative square root σ=Var⁡(X)\sigma=\sqrt{\operatorname{Var}(X)}.

The standard-deviation definition is also supplied.

In the formula booklet.

To see why the shorter variance formula works, expand (x−μ)2(x-\mu)^2 inside the integral:

E(X2)−2μ E(X)+μ2∫f(x) dx=E(X2)−2μ2+μ2\mathrm E(X^2)-2\mu\,\mathrm E(X)+\mu^2\int f(x)\,\mathrm{d}x =\mathrm E(X^2)-2\mu^2+\mu^2

The last integral is 11, not the width of the support.

Worked example 1: a rising linear density

Question. Find the mean and variance of the density f(x)=2xf(x)=2x on [0,1][0,1], with density zero elsewhere.

Plan. Calculate E(X)\mathrm E(X) and E(X2)\mathrm E(X^2), then subtract the square of the first from the second.

E(X)=∫01x(2x) dx=[2x33]01=23first momentE(X2)=∫01x2(2x) dx=[x42]01=12second momentVar⁡(X)=12−(23)2=118subtract the squared mean\begin{aligned} \mathrm E(X)&=\int_0^1x(2x)\,\mathrm{d}x =\left[\frac{2x^3}{3}\right]_0^1 =\frac23 && \text{first moment}\\ \mathrm E(X^2)&=\int_0^1x^2(2x)\,\mathrm{d}x =\left[\frac{x^4}{2}\right]_0^1 =\frac12 && \text{second moment}\\ \operatorname{Var}(X)&=\frac12-\left(\frac23\right)^2 =\frac1{18} && \text{subtract the squared mean} \end{aligned}

Why square 23\frac23, rather than the integrand? Variance subtracts [E(X)]2[\mathrm E(X)]^2 after the two separate expectations have been found.

Answer. E(X)=23\mathrm E(X)=\frac23 and Var⁡(X)=118\operatorname{Var}(X)=\frac1{18}.

Check. The mean is inside [0,1][0,1], the variance is positive, and direct expansion gives ∫01(x−23)2(2x) dx=118\int_0^1(x-\frac23)^2(2x)\,\mathrm{d}x=\frac1{18}.

Worked example 2 increases the density's power by one. Its normalising coefficient changes so that the total area remains 11.

Worked example 2: more weight near the upper endpoint

Question. Find the mean and variance when f(x)=3x2f(x)=3x^2 on [0,1][0,1], with density zero elsewhere.

Plan. Repeat the same two moment integrals with the new density.

E(X)=∫013x3 dx=[3x44]01=34weight each valueE(X2)=∫013x4 dx=[3x55]01=35weight each squareVar⁡(X)=35−916=380use a common denominator\begin{aligned} \mathrm E(X)&=\int_0^1 3x^3\,\mathrm{d}x =\left[\frac{3x^4}{4}\right]_0^1 =\frac34 && \text{weight each value}\\ \mathrm E(X^2)&=\int_0^1 3x^4\,\mathrm{d}x =\left[\frac{3x^5}{5}\right]_0^1 =\frac35 && \text{weight each square}\\ \operatorname{Var}(X)&=\frac35-\frac9{16} =\frac3{80} && \text{use a common denominator} \end{aligned}

Why does the mean move right? Compared with 2x2x, this density places more probability near the upper end of the same interval.

Answer. E(X)=34\mathrm E(X)=\frac34 and Var⁡(X)=380\operatorname{Var}(X)=\frac3{80}.

Check. ∫013x2 dx=1\int_0^1 3x^2\,\mathrm{d}x=1. The new mean exceeds 23\frac23, while the variance is smaller than 118\frac1{18}, consistent with stronger concentration near 11.

If XX is a length in centimetres, its mean and standard deviation are in centimetres, while its variance is in square centimetres. A variance quoted as a standard deviation changes both the value and the units.

Watch out

Wrong: Var⁡(X)=E(X2)−E(X)\operatorname{Var}(X)=\mathrm E(X^2)-\mathrm E(X). Right: subtract [E(X)]2[\mathrm E(X)]^2. A negative variance is a signal to check the density, limits and arithmetic.

Increase the density's power once more.

Transform the value before taking its expectation

A payment, area or squared error may be a function of the random measurement. Keep the original density and change the value being averaged. You do not need a new density for the transformed variable.

Expectation of a function

For a continuous random variable with density ff, provided the expectation exists,

E(g(X))=∫−∞∞g(x)f(x) dx\mathrm E(g(X))=\int_{-\infty}^{\infty}g(x)f(x)\,\mathrm{d}x

In the formula booklet.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account