2.02g - Variance, standard deviation and grouped estimates

2.02g - Variance, standard deviation and grouped estimates

Calculate mean and standard deviation from individual values, summary statistics and frequency tables. You will also estimate them from grouped data, while keeping the calculator's divisor, the units and the lost information clear.

Raw data and summaries give the same spread

Can three numbers tell you how variable a thousand measurements are? If you know the count, their total and their sum of squares, you can recover their mean and standard deviation without seeing the individual measurements. You cannot recover the shape of their distribution: summaries are powerful, but they are not a photograph.

Start with four repair times: 3,5,5,73, 5, 5, 7 minutes. Their mean is 55 minutes; their deviations are −2,0,0,2-2, 0, 0, 2. Adding those deviations gives zero, so ordinary averaging would make the variation disappear. Squaring prevents that cancellation.

You already know

The mean is the total divided by the number of observations. Variance averages squared deviations from the mean; standard deviation takes its square root, restoring the measurement units.

Write vv for the variance of the data here. H240 uses the root mean square deviation, with divisor nn. The mean formula xˉ=∑x/n\bar x=\sum x/n is not in the formula sheet: learn it.

Raw-data standard deviation

For n≥1n\ge1 observations,

SD=∑(x−xˉ)2n=∑x2n−xˉ2\mathrm{SD}=\sqrt{\frac{\sum(x-\bar x)^2}{n}} =\sqrt{\frac{\sum x^2}{n}-\bar x^2}

In the formula sheet.

Variance is the square of SD, so v=∑(x−xˉ)2/n=∑x2/n−xˉ2v=\sum(x-\bar x)^2/n=\sum x^2/n-\bar x^2. This variance identity is derived by squaring the supplied SD expressions, rather than printed separately. Why do the two variance expressions agree? Expand and add the squares:

∑(x−xˉ)2=∑x2−2xˉ∑x+nxˉ2=∑x2−nxˉ2\sum(x-\bar x)^2 =\sum x^2-2\bar x\sum x+n\bar x^2 =\sum x^2-n\bar x^2

The last equality uses ∑x=nxˉ\sum x=n\bar x. Dividing by nn gives the shortcut. Variance is non-negative, and is zero exactly when every observation is the same.

Worked example 1: use the deviations

Question. Find the mean, variance and standard deviation of repair times 3,5,5,73, 5, 5, 7 minutes.

Plan. Find the mean, square each deviation, then average and take a square root.

xˉ=3+5+5+74=5average the four times∑(x−5)2=4+0+0+4=8square before addingv=84=2divide by the countSD=2≈1.41restore the time units\begin{aligned} \bar x&=\frac{3+5+5+7}{4}=5 && \text{average the four times} \\ \sum(x-5)^2&=4+0+0+4=8 && \text{square before adding} \\ v&=\frac84=2 && \text{divide by the count} \\ \mathrm{SD}&=\sqrt2\approx1.41 && \text{restore the time units} \end{aligned}

Why this divisor? We are describing the spread of these four observations using H240's root mean square definition.

Answer. Mean 55 minutes, variance 2 min22\,\mathrm{min}^2, SD 1.411.41 minutes, to 33 significant figures.

Check. The deviations have size at most 22 minutes, so an SD between 00 and 22 minutes is sensible.

Now change only the way the same data are supplied: replace the list by its summaries. The answer must stay the same.

Worked example 2: reconstruct from summaries

Question. Four repair times have ∑x=20\sum x=20 and ∑x2=108\sum x^2=108. Calculate their mean and standard deviation.

Plan. Use the total for the mean and the sum of squares for the shortcut.

xˉ=204=5use the totalv=1084−52=2subtract the squared meanSD=2≈1.41take the square root\begin{aligned} \bar x&=\frac{20}{4}=5 && \text{use the total} \\ v&=\frac{108}{4}-5^2=2 && \text{subtract the squared mean} \\ \mathrm{SD}&=\sqrt2\approx1.41 && \text{take the square root} \end{aligned}

Why not square 2020 first? ∑x2\sum x^2 adds the squared measurements; (∑x)2(\sum x)^2 squares their total. They are different quantities.

Answer. Mean 55 minutes and SD 1.411.41 minutes, to 33 significant figures.

Check. 32+52+52+72=1083^2+5^2+5^2+7^2=108, agreeing with the supplied summary and the first calculation.

A calculator can perform either route. Clear earlier entries, choose one-variable statistics, enter the values and check the count and mean. Select the SD output defined with divisor nn. Many calculators call this σx\sigma_x, while sxs_x uses n−1n-1; check your model's definitions. For these four times, the other output is 8/3\sqrt{8/3}, which is a different statistic. Formal estimation of population variance from a sample is outside this topic.

Watch out

Wrong: v=∑x2/n−xˉv=\sum x^2/n-\bar x. Right: v=∑x2/n−xˉ2v=\sum x^2/n-\bar x^2. Keep full-precision totals and mean until the final answer: a small negative result indicates rounding or input trouble, not negative spread.

Try the summary route with another set of measurements.

Frequencies count repeated observations

A frequency table compresses repeated values. It must give exactly the same summaries as writing out every observation separately.

Frequency

The frequency ff of a value xx is the number of observations equal to that value. Values 1,3,51,3,5 with frequencies 2,3,12,3,1 represent six observations: 1,1,3,3,3,51,1,3,3,3,5, not three observations.

Each repeated value contributes fxfx to the total and fx2fx^2 to the sum of squares. The weighted mean is xˉ=∑fx/∑f\bar x=\sum fx/\sum f; this mean formula is not in the formula sheet: learn it.

Statistics from frequencies

For positive total frequency n=∑fn=\sum f,

SD=∑f(x−xˉ)2∑f=∑fx2∑f−xˉ2\mathrm{SD}=\sqrt{\frac{\sum f(x-\bar x)^2}{\sum f}} =\sqrt{\frac{\sum fx^2}{\sum f}-\bar x^2}

In the formula sheet.

Treat the frequency column as repeated observations when using calculator statistics, and verify the reported count.

Worked example 3: repeated daily counts

Question. Daily counts 1,3,51,3,5 have frequencies 2,3,12,3,1. Find the mean and SD.

Plan. Build the frequency-weighted totals before using the divisor-nn formula.

xxfffxfxfx2fx^2
11222222
3333992727
5511552525
n=6,∑fx=16,∑fx2=54sum the columnsxˉ=166=83six observed daysv=546−(83)2=179use the weighted squaresSD=173≈1.37take the square root\begin{aligned} n&=6,\quad \sum fx=16,\quad\sum fx^2=54 && \text{sum the columns} \\ \bar x&=\frac{16}{6}=\frac83 && \text{six observed days} \\ v&=\frac{54}{6}-\left(\frac83\right)^2=\frac{17}{9} && \text{use the weighted squares} \\ \mathrm{SD}&=\frac{\sqrt{17}}3\approx1.37 && \text{take the square root} \end{aligned}

Why square xx before multiplying by ff? Three observations equal to 33 contribute 3×32=273\times3^2=27, not (3×3)2=81(3\times3)^2=81.

Answer. Mean 2.672.67 counts per day and SD 1.371.37 counts, to 33 significant figures.

Check. Expanding the six-value list gives total 1616 and sum of squared values 5454 as well.

Change just the frequency of 55, from 11 to 33. Both the total and the count must respond.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account