FS1 1.1 - Mean and variance of discrete distributions

FS1 1.1 - Mean and variance of discrete distributions

A probability distribution tells you how often each value occurs; expectation and variance turn that information into a centre and a measure of spread. You will calculate both, find expectations of transformed values, work with linear coding, and use the results to judge whether a model describes the situation sensibly.

Expectation as a probability-weighted mean

A game pays the square of your score. Can you predict the average prize just by squaring the average score?

Start with a score XX taking values −1,1,3-1,1,3, with probabilities 1/4,1/2,1/41/4,1/2,1/4. In a long run of independent plays under this model, the middle score occurs about twice as often as either extreme. A simple average of the three distinct values would ignore that unequal weighting.

You already know

From A-level Mathematics, a discrete random variable has separate possible values and an associated probability distribution. Each probability is between 00 and 11, inclusive, and the probabilities over all possible values sum to 11. Check these conditions before treating a table as a distribution.

Multiply each outcome by its probability, then add. This is the probability version of dividing a frequency-weighted total by the total frequency.

Expectation

The expectation, or mean, of XX is its probability-weighted average, denoted E(X)=μ\mathrm E(X)=\mu. It describes the long-run average under the model, not a guaranteed result on the next trial. It need not be one of the possible values of XX.

The probabilities already add to one, so do not divide the resulting weighted sum by the number of outcomes again.

Mean of a discrete distribution

If XX takes values xx with probabilities p(x)=P(X=x)p(x)=\mathrm P(X=x), then

E(X)=μ=∑xxP(X=x)\mathrm E(X)=\mu=\sum_x x\mathrm P(X=x)

Sum over every possible value, using a valid probability distribution. For infinitely many possible values, the expectation must exist; all tables in this lesson are finite. In the formula booklet.

Negative outcomes contribute negative terms, while an outcome of zero contributes zero.

Worked example 1: unequal probability weights

Question. Find E(X)\mathrm E(X) for the distribution below.

xx−1-11133
P(X=x)\mathrm P(X=x)1/41/41/21/21/41/4

Plan. Check the probabilities, then add the outcome-probability products.

14+12+14=1valid non-negative probabilitiesE(X)=(−1)14+112+314weight each outcome=−1+2+34=1add exactly\begin{aligned} \frac14+\frac12+\frac14&=1 && \text{valid non-negative probabilities} \\ \mathrm E(X)&=(-1)\frac14+1\frac12+3\frac14 && \text{weight each outcome} \\ &=\frac{-1+2+3}{4}=1 && \text{add exactly} \end{aligned}

Why this step? Each probability represents the proportion of trials contributing that score to the long-run total.

Answer. E(X)=1\mathrm E(X)=1.

Check. The distribution is symmetric about 11: scores two units either side have equal probability, so 11 is a sensible balance point.

Worked example 2 changes only the highest score, from 33 to 55. Its probability stays fixed, so you can predict which way the mean moves.

Worked example 2: the mean need not be attainable

Question. Find the mean for scores −1,1,5-1,1,5 with respective probabilities 1/4,1/2,1/41/4,1/2,1/4.

Plan. Keep the same weights and update the final weighted term.

E(X)=−14+12+54changed highest score=64=32collect fractions\begin{aligned} \mathrm E(X)&=-\frac14+\frac12+\frac54 && \text{changed highest score} \\ &=\frac64=\frac32 && \text{collect fractions} \end{aligned}

Why this step? Increasing one score by 22, with probability 1/41/4, increases the mean by 2(1/4)=1/22(1/4)=1/2.

Answer. E(X)=3/2\mathrm E(X)=3/2, even though the score can never equal 3/23/2.

Check. The mean lies between the smallest and largest possible values, −1-1 and 55, and is half a unit above the previous mean.

Expectation is a balance point, not another outcome to add to the table.

Watch out

Wrong: divide ∑xP(X=x)\sum x\mathrm P(X=x) by the number of listed values. Right: the probabilities already supply normalised weights, so the sum itself is the mean.

Change the highest score once more in your turn.

Variance measures squared spread

The mean alone cannot tell you whether scores cluster tightly or vary widely, so measure their squared distances from the mean.

The average signed deviation is always zero: E(X−μ)=0\mathrm E(X-\mu)=0. Squaring before averaging prevents positive and negative deviations cancelling. The result is the variance σ2\sigma^2; its non-negative square root σ\sigma is the standard deviation.

Variance of a discrete distribution

For a valid distribution with finite second moment,

Var⁡(X)=σ2=∑x(x−μ)2P(X=x)=E(X2)−μ2\operatorname{Var}(X)=\sigma^2 =\sum_x(x-\mu)^2\mathrm P(X=x) =\mathrm E(X^2)-\mu^2

where μ=E(X)\mu=\mathrm E(X) and E(X2)=∑xx2P(X=x)\mathrm E(X^2)=\sum_x x^2\mathrm P(X=x). In the formula booklet.

The shortcut follows by expanding the square: E(X2−2μX+μ2)=E(X2)−2μ2+μ2\mathrm E(X^2-2\mu X+\mu^2)=\mathrm E(X^2)-2\mu^2+\mu^2. Variance is non-negative, and is zero exactly when the variable is constant with probability one. Its units are the square of the variable's units; standard deviation returns to the original units. Use σ=Var⁡(X)\sigma=\sqrt{\operatorname{Var}(X)}; this square-root relation is not printed as a separate formula in the booklet, so learn it.

Worked example 3: calculate the second moment first

Question. Find the variance of scores −1,1,3-1,1,3, with probabilities 1/4,1/2,1/41/4,1/2,1/4.

Plan. Use the mean 11, calculate the weighted squares, then subtract the square of the mean.

E(X)=1weighted meanE(X2)=(−1)214+1212+3214=3square values, retain probabilitiesVar⁡(X)=3−12=2subtract the squared mean\begin{aligned} \mathrm E(X)&=1 && \text{weighted mean} \\ \mathrm E(X^2)&=(-1)^2\frac14+1^2\frac12+3^2\frac14=3 && \text{square values, retain probabilities} \\ \operatorname{Var}(X)&=3-1^2=2 && \text{subtract the squared mean} \end{aligned}

Why this step? E(X2)\mathrm E(X^2) and (E(X))2(\mathrm E(X))^2 are different quantities: the square goes inside the weighted sum in the first one.

Answer. Var⁡(X)=2\operatorname{Var}(X)=2, so the standard deviation is 2\sqrt2.

Check. Using squared deviations directly gives 4(1/4)+0(1/2)+4(1/4)=24(1/4)+0(1/2)+4(1/4)=2.

Worked example 4 changes only the highest score to 55, as in the earlier mean calculation. Recalculate both moments; the old mean no longer applies.

Worked example 4: both moments change

Question. Find the variance of scores −1,1,5-1,1,5, with probabilities 1/4,1/2,1/41/4,1/2,1/4.

Plan. Use the new mean 3/23/2 and calculate the new second moment exactly.

E(X)=32new weighted meanE(X2)=14+12+254=7weighted squared scoresVar⁡(X)=7−(32)2=194subtract the new squared mean\begin{aligned} \mathrm E(X)&=\frac32 && \text{new weighted mean} \\ \mathrm E(X^2)&=\frac14+\frac12+\frac{25}4=7 && \text{weighted squared scores} \\ \operatorname{Var}(X)&=7-\left(\frac32\right)^2=\frac{19}4 && \text{subtract the new squared mean} \end{aligned}

Why this step? A wider upper tail changes the centre as well as the spread, so holding the old mean fixed would not calculate variance.

Answer. Var⁡(X)=19/4\operatorname{Var}(X)=19/4.

Check. Direct squared deviations give (25/4)(1/4)+(1/4)(1/2)+(49/4)(1/4)=19/4(25/4)(1/4)+(1/4)(1/2)+(49/4)(1/4)=19/4.

Keep exact fractions or full calculator precision until the final answer. A rounded mean gets squared, which can magnify an early rounding error.

Watch out

Wrong: Var⁡(X)=E(X2)−E(X)\operatorname{Var}(X)=\mathrm E(X^2)-\mathrm E(X). Right: subtract (E(X))2(\mathrm E(X))^2. A negative variance is a warning to check the arithmetic, probabilities and brackets.

Use a new set of scores for the next calculation.

Expectations and variances of transformed values

To average a function of a random value, apply the function to each possible outcome before taking the probability-weighted mean.

The outcome probabilities do not get squared when the outcomes do. If different original outcomes give the same transformed value, add their probabilities when writing the new distribution.

Expectation of a function

For a valid discrete distribution and a real-valued function gg, provided the expectation exists,

E(g(X))=∑xg(x)P(X=x)\mathrm E(g(X))=\sum_x g(x)\mathrm P(X=x)

If Y=g(X)Y=g(X) has a finite second moment, calculate its variance as Var⁡(Y)=E((g(X))2)−(E(g(X)))2\operatorname{Var}(Y)=\mathrm E((g(X))^2)-(\mathrm E(g(X)))^2. This uses the supplied expectation and variance formulae. In the formula booklet.

For Y=X2Y=X^2, this means Var⁡(X2)=E(X4)−(E(X2))2\operatorname{Var}(X^2)=\mathrm E(X^4)-(\mathrm E(X^2))^2. The required fourth power comes from squaring the transformed variable, not from an extra rule to memorise.

Worked example 5: a square can merge outcomes

Question. For scores −1,1,3-1,1,3 with probabilities 1/4,1/2,1/41/4,1/2,1/4, let Y=X2Y=X^2. Find E(Y)\mathrm E(Y) and Var⁡(Y)\operatorname{Var}(Y).

Plan. Form the transformed values, merge repeated outcomes, then apply the usual moment formulae.

yy1199
P(Y=y)\mathrm P(Y=y)3/43/41/41/4
E(Y)=134+914=3average squared scoresE(Y2)=1234+9214=21average fourth powers of XVar⁡(Y)=21−32=12variance of the new variable\begin{aligned} \mathrm E(Y)&=1\frac34+9\frac14=3 && \text{average squared scores} \\ \mathrm E(Y^2)&=1^2\frac34+9^2\frac14=21 && \text{average fourth powers of }X \\ \operatorname{Var}(Y)&=21-3^2=12 && \text{variance of the new variable} \end{aligned}

Why this step? Both X=−1X=-1 and X=1X=1 produce Y=1Y=1, so its probability is 1/4+1/2=3/41/4+1/2=3/4.

Answer. E(Y)=3\mathrm E(Y)=3 and Var⁡(Y)=12\operatorname{Var}(Y)=12.

Check. Directly from XX, E(X4)=(1+2+81)/4=21\mathrm E(X^4)=(1+2+81)/4=21, matching E(Y2)\mathrm E(Y^2).

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account