SD8 - Sums of independent variables

SD8 - Sums of independent variables

Totals and differences of random quantities have their own means and spreads. You will combine discrete or continuous variables, see why independence matters for variance, and distinguish repeated observations from repeated use of the same variable.

Adding means and independent variances

Two independent measurements each have standard deviation 2 millimetres. Does their sum have standard deviation 4 millimetres? The answer depends on adding squared deviations before taking a square root.

The mean of a total is the total of the means. Imagine averaging paired observations: averaging each row's sum gives the same result as adding the two column averages. This does not require independence.

Means and variances of a sum

For variables with finite expectations,

E(X+Y)=E(X)+E(Y).E(X+Y)=E(X)+E(Y).

Independence is not needed for this equality. If X,YX,Y are independent and have finite variances, then

Var⁡(X+Y)=Var⁡(X)+Var⁡(Y).\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y).

These rules hold for discrete variables, continuous variables or a combination of the two. They are not separately printed in the formula booklet.

Why is variance different? Put μX=E(X)\mu_X=E(X), μY=E(Y)\mu_Y=E(Y). Expanding the squared deviation gives

Var⁡(X+Y)=E([(X−μX)+(Y−μY)]2)=Var⁡(X)+Var⁡(Y)+2E((X−μX)(Y−μY)).\begin{aligned} \operatorname{Var}(X+Y) &=E\bigl([(X-\mu_X)+(Y-\mu_Y)]^2\bigr)\\ &=\operatorname{Var}(X)+\operatorname{Var}(Y) +2E\bigl((X-\mu_X)(Y-\mu_Y)\bigr). \end{aligned}

For independent variables, the expectation of a product factors into the product of expectations. The last expectation is therefore

E(X−μX)E(Y−μY)=0×0=0.E(X-\mu_X)E(Y-\mu_Y)=0\times0=0.

This factorisation works through sums, integrals or both. It is why independence makes variance addition valid. Without independence, the cross term may remain; the separate means and variances alone are generally insufficient. Independence is sufficient, though some dependent variables also happen to have a zero cross term.

Worked example 1: a total of two observations

Question. Independent measurements X,YX,Y, in millimetres, each have mean 10 and standard deviation 2. Find the mean, variance and standard deviation of S=X+YS=X+Y.

Plan. Add means, square each standard deviation before adding variances, then take one square root.

E(S)=10+10=20,Var⁡(S)=22+22=8.E(S)=10+10=20,\qquad \operatorname{Var}(S)=2^2+2^2=8. SD⁡(S)=8=22.\operatorname{SD}(S)=\sqrt8=2\sqrt2.

Answer. Mean 20 mm20\,\mathrm{mm}, variance 8 mm28\,\mathrm{mm}^2, standard deviation 22 mm2\sqrt2\,\mathrm{mm}.

Check. Its standard deviation is about 2.83 mm2.83\,\mathrm{mm}, larger than either individual standard deviation but smaller than their sum 4. Squaring it recovers 8.

Keep XX's moments, but replace the second independent observation by a second copy of the same XX. The two contributions keep the same means and variances, but their dependence changes.

Worked example 2: using one observation twice

Question. E(X)=10E(X)=10 and SD⁡(X)=2\operatorname{SD}(X)=2. Find the mean, variance and standard deviation of T=X+XT=X+X.

Plan. Simplify T=2XT=2X before applying a one-variable scale rule.

E(T)=2(10)=20,Var⁡(T)=22Var⁡(X)=4(4)=16.E(T)=2(10)=20,\qquad \operatorname{Var}(T)=2^2\operatorname{Var}(X)=4(4)=16. SD⁡(T)=16=4.\operatorname{SD}(T)=\sqrt{16}=4.

Answer. Mean 20, variance 16, standard deviation 4, in the corresponding measurement units.

Check. Each deviation from the mean is doubled exactly: T−20=2(X−10)T-20=2(X-10). Its squared size is four times the original, rather than twice.

If X1,X2X_1,X_2 were independent copies of XX, their sum would instead have variance 4+4=84+4=8. A subscript denotes a separate observation only when the model says so; identical distributions do not imply independence. Likewise, two variables sharing a random source need care: A=X+YA=X+Y and B=X+ZB=X+Z both contain XX, even if the original X,Y,ZX,Y,Z are independent.

Watch out

Wrong: SD⁡(X+Y)=SD⁡(X)+SD⁡(Y)\operatorname{SD}(X+Y)=\operatorname{SD}(X)+\operatorname{SD}(Y) for independent variables. Add variances, then take the square root. For X+XX+X, combine to 2X2X; one nonconstant variable is not independent of itself.

Try separate observations with different spreads.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account