MD12 - Variance, standard deviation and grouped estimates

MD12 - Variance, standard deviation and grouped estimates

Use MEI's sample variance and sample standard deviation to describe spread from raw data, frequency tables and grouped estimates. You will interpret calculator outputs, keep the sample divisor clear and explain what grouping conceals.

Sample variance uses divisor n minus one

Why can two correct calculator outputs give different standard deviations for the same four measurements? For machining times 4,6,6,84,6,6,8 minutes, the mean is 66, and the squared deviations add to 88. Dividing by 44 or by 33 gives different measures. The calculator is offering a choice; you need to know which measure the question requests.

You already know

The mean divides the total by the number of observations. Squared deviations measure spread without cancellation; taking a square root restores the original units.

MEI defines sample variance using divisor n−1n-1, and denotes it by s2s^2. The sample mean still uses divisor nn. Its formula xˉ=∑xi/n\bar x=\sum x_i/n is not in the formula sheet: learn it.

Sample variance and sample standard deviation

For a sample of n>1n>1 observations, sample variance is the sum of squared deviations from the sample mean divided by n−1n-1. Sample standard deviation ss is its non-negative square root. Variance has squared measurement units; ss has the original units.

Writing SxxS_{xx} for the squared-deviation total keeps the two different divisors visible.

MEI sample spread

For n>1n>1,

Sxx=∑(xi−xˉ)2=∑xi2−(∑xi)2n=∑xi2−nxˉ2s2=Sxxn−1,s=s2\begin{aligned} S_{xx}&=\sum(x_i-\bar x)^2 =\sum x_i^2-\frac{(\sum x_i)^2}{n} =\sum x_i^2-n\bar x^2 \\ s^2&=\frac{S_{xx}}{n-1},\qquad s=\sqrt{s^2} \end{aligned}

In the formula sheet.

The shortcut follows by expanding the squares and using ∑xi=nxˉ\sum x_i=n\bar x: the two cross terms combine to subtract nxˉ2n\bar x^2. With one observation, n−1=0n-1=0, so this sample variance is undefined. With at least two identical observations it is zero. These are different cases.

Worked example 1: deviations first

Question. Calculate the sample mean, sample variance and sample SD of machining times 4,6,6,84,6,6,8 minutes. Give the SD to 33 significant figures.

Plan. Find the mean, add squared deviations, then divide by one less than the sample size.

xˉ=4+6+6+84=6mean uses fourSxx=(−2)2+02+02+22=8square before addings2=84−1=83sample divisors=83≈1.63restore time units\begin{aligned} \bar x&=\frac{4+6+6+8}{4}=6 && \text{mean uses four} \\ S_{xx}&=(-2)^2+0^2+0^2+2^2=8 && \text{square before adding} \\ s^2&=\frac{8}{4-1}=\frac83 && \text{sample divisor} \\ s&=\sqrt{\frac83}\approx1.63 && \text{restore time units} \end{aligned}

Why not divide the mean by 33? The adjustment belongs to the definition of sample variance, not the mean.

Answer. Mean 66 minutes, sample variance 8/3 min28/3\,\mathrm{min}^2, sample SD 1.631.63 minutes.

Check. The squared-deviation total is non-negative, and both spread measures are zero only when all the observations coincide.

Change only the representation: the next question supplies summaries of the same four times.

Worked example 2: use the summary-statistics shortcut

Question. Four machining times have ∑xi=24\sum x_i=24 and ∑xi2=152\sum x_i^2=152. Determine their sample mean and sample SD, giving the SD to 33 significant figures.

Plan. Use the summaries to obtain SxxS_{xx}, then apply the sample divisor.

xˉ=244=6total divided by countSxx=152−2424=8remove the mean components2=83divide by n−1s=83≈1.63take the square root\begin{aligned} \bar x&=\frac{24}{4}=6 && \text{total divided by count} \\ S_{xx}&=152-\frac{24^2}{4}=8 && \text{remove the mean component} \\ s^2&=\frac83 && \text{divide by }n-1 \\ s&=\sqrt{\frac83}\approx1.63 && \text{take the square root} \end{aligned}

Why find SxxS_{xx} first? Its shortcut contains divisor nn; replacing that by n−1n-1 would change the squared-deviation total itself.

Answer. Mean 66 minutes and sample SD 1.631.63 minutes.

Check. 42+62+62+82=1524^2+6^2+6^2+8^2=152, so the result agrees with the direct calculation.

For any non-constant sample, the divisor-n−1n-1 SD is larger than the divisor-nn SD by the factor n/(n−1)\sqrt{n/(n-1)}. That difference is large for tiny samples and shrinks as nn grows. This is an algebraic comparison of two definitions, not a reason to switch between them.

In one-variable calculator statistics, clear old data, enter the measurements and check nn and xˉ\bar x. Choose the output defined using n−1n-1, commonly sxs_x. The other output, commonly σx\sigma_x, uses nn. Manufacturer labels vary, so confirm the definition against the known four-time example. Keep full-precision totals until the end.

For a real check, the first five people in the current LDS9 NHANES 20032003–20042004 sample have pulse entries 7474, #N/A, 9292, 7070, 6868 beats per minute. Use the four observed pulses: xˉ=76\bar x=76, Sxx=360S_{xx}=360, so s2=360/3=120s^2=360/3=120 and s=120≈11.0s=\sqrt{120}\approx11.0 beats per minute, to 33 significant figures. The missing code is not a fifth pulse equal to zero. The valid count belongs to the variable being analysed.

Watch out

Wrong: s2=∑xi2/(n−1)−xˉ2s^2=\sum x_i^2/(n-1)-\bar x^2. Right: calculate Sxx=∑xi2−(∑xi)2/nS_{xx}=\sum x_i^2-(\sum x_i)^2/n first, then divide the whole SxxS_{xx} by n−1n-1.

Use the shortcut with a new summary.

Frequency weights retain the sample size

A frequency is a count of repeated observations. Compressing the data into a table does not reduce the sample size to the number of rows.

Sample statistics from frequencies

For value xx, frequency ff and n=∑f>1n=\sum f>1,

xˉ=∑fxn,Sxx=∑fx2−(∑fx)2n,s2=Sxxn−1\bar x=\frac{\sum fx}{n},\qquad S_{xx}=\sum fx^2-\frac{(\sum fx)^2}{n},\qquad s^2=\frac{S_{xx}}{n-1}

The sample-spread identities follow from the raw-data formulas by counting repetitions; the weighted-mean formula is not printed on the sheet.

Not in the formula sheet: learn it.

Use the actual count as nn, then choose the sample SD output on the calculator. Its frequency column must be enabled.

Worked example 3: compress repeated values

Question. Defect counts 1,3,51,3,5 have frequencies 2,3,12,3,1. Calculate the sample mean and sample SD to 33 significant figures.

Plan. Count repetitions in each total and apply the sample-spread definition.

xxfffxfxfx2fx^2
11222222
3333992727
5511552525
n=6,∑fx=16,∑fx2=54sum the columnsxˉ=83six observationsSxx=54−1626=343weighted squared deviationss2=34/35=3415sample divisor fives=3415≈1.51original count units\begin{aligned} n&=6,\quad \sum fx=16,\quad\sum fx^2=54 && \text{sum the columns} \\ \bar x&=\frac83 && \text{six observations} \\ S_{xx}&=54-\frac{16^2}{6}=\frac{34}{3} && \text{weighted squared deviations} \\ s^2&=\frac{34/3}{5}=\frac{34}{15} && \text{sample divisor five} \\ s&=\sqrt{\frac{34}{15}}\approx1.51 && \text{original count units} \end{aligned}

Why is the denominator 55? The sample contains six observations, even though only three different values occur.

Answer. Mean 2.672.67 defects and sample SD 1.511.51 defects.

Check. The expanded list 1,1,3,3,3,51,1,3,3,3,5 gives exactly the same calculator result.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account