L4 - Outliers, missing values and data choices | AQA A-Level Mathematics

Investigate possible outliers, distinguish missing values from genuine zeros and choose displays that support a fair conclusion. Use the actual AQA car data to make field-specific cleaning decisions and keep an honest account of what remains uncertain.

Identifying and investigating outliers

Should you delete a car record because its engine size is zero? For an electric car, that zero can make sense; for its mass, zero cannot. A surprising entry needs investigation before a decision, and the decision depends on the variable.

You already know

The interquartile range is $Q_3-Q_1$ . Mean and standard deviation use all observations, while median and IQR are less sensitive to a single distant value. None of these summaries tells you whether an entry is genuine.

An outlier is an observation unusually far from the rest or from the pattern in a diagram. It can be a genuine unusual case, an error or evidence of another population group. A point isolated vertically from a scatter pattern may matter even if neither coordinate is extreme on its own.

Apply the rule stated in a question, including whether the boundary is strict. A common supplied rule flags values below $Q_1-1.5\,\mathrm{IQR}$ or above $Q_3+1.5\,\mathrm{IQR}$ . This rule is not in the formula booklet, and it is not a universal definition to impose when a different rule is supplied.

Worked example 1: apply a strict fence

Question. Illustrative measurements have $Q_1=10$ , $Q_3=18$ . A rule flags values strictly below $Q_1-1.5\,\mathrm{IQR}$ or strictly above $Q_3+1.5\,\mathrm{IQR}$ . Decide which of $8,12,18,27,45$ are flagged.

Plan. Calculate both fences, then compare each observation with them.

\begin{aligned} \mathrm{IQR}&=18-10=8 && \text{quartile spread}\\ \text{lower fence}&=10-1.5(8)=-2 && \text{subtract the allowance}\\ \text{upper fence}&=18+1.5(8)=30 && \text{add the allowance} \end{aligned}

Why this step? The quartile rule identifies possible outliers; it does not establish that they are errors.

Answer. Only $45$ is flagged, since $45>30$ ; none of the values is below $-2$ .

Check. $27$ , the next largest value, is inside the fences.

Change only the final value from $45$ to $30$ , leaving the supplied quartiles and rule fixed. The boundary word now decides the result.

Worked example 2: an observation on the fence

Question. Using the same supplied quartiles and strict rule, decide which of $8,12,18,27,30$ are flagged.

Plan. Retain the fences and test the final observation against the strict inequality.

\begin{aligned} \text{fences}&=-2,30 && \text{same supplied quartiles}\\ 30&\not>30 && \text{equality is not strictly above} \end{aligned}

Why this step? Replacing “above” by “at least” would change the stated rule.

Answer. None is flagged. This does not prove every observation is free of error; it only gives the outcome under this rule.

Check. All five observations lie in the closed interval $[-2,30]$ .

To investigate a flagged entry, check its source, units, recording equipment and population membership. Correct it only if reliable evidence supplies the correction. Otherwise preserve the raw entry and explain any analysis with and without it.

For illustrative waits $8,10,11,12,39$ minutes, including $39$ gives mean $16$ , median $11$ and descriptive standard deviation $\sqrt{134}\approx11.58$ minutes. Omitting it gives mean $10.25$ , median $10.5$ and standard deviation $\sqrt{35/16}\approx1.48$ minutes. The large changes show sensitivity, not permission to discard a valid long wait. For other datasets, examine the effect rather than claiming removal must change every statistic in a particular direction.

Watch out

Wrong: “ $45$ is outside the fence, so delete it.” A fence flags a case; it does not diagnose an error. Right: investigate, preserve the raw value and justify any correction or exclusion.

The next question changes the quartiles and includes an equality case as well as a flagged case.

Cleaning each field honestly

A missing entry says that a measurement is not recorded; a numerical zero records a value, whose meaning depends on the field.

Keep the source sheet and create a separate analysis view. Log the variable, proposed checks, exclusions or reliable corrections, and resulting count. Do not apply one field's exclusions automatically to every other field in a row.

The following four selected entries are authentic AQA records. The dash represents a blank NOX cell, not a numerical code. EngineSize is in $\mathrm{cm}^3$ , Mass in $\mathrm{kg}$ , and emissions in $\mathrm{g\,km^{-1}}$ .

ReferenceNumber	Fuel	EngineSize	Mass	CO2	NOX
$440$	Petrol	$1598$	$1970$	$190$	$0.026$
$588$	Petrol	$2979$	$0$	$230$	—
$3645$	Electric	$0$	$1270$	$0$	—
$1199$	Petrol	$1598$	$0$	$166$	$0.043$

Source: AQA car LDS, $2017$ DfT Stock Vehicle Database, Open Government Licence v3.0. Mass includes $75\,\mathrm{kg}$ for an average driver. Decimal emissions retain their recorded precision.

Worked example 3: a mass-specific filter

Question. Find the mean of the usable masses in this selected extract, assuming reliable corrections are unavailable. Explain the exclusions.

Plan. Reject impossible zero masses for this analysis, then use the positive-mass count as the denominator.

\begin{aligned} n_{\text{usable}}&=2 && \text{records }440,3645\\ \bar m&=\frac{1970+1270}{2}=1620 && \text{mean of usable masses} \end{aligned}

Why this step? A recorded car mass of zero is impossible, whereas an electric car can have a meaningful zero combustion-engine capacity.

Answer. The usable-mass mean is $1620\,\mathrm{kg}$ . Exclude the mass entries for $588$ and $1199$ , retain the original rows and log the decision. Keep electric record $3645$ 's positive mass.

Check. Including the two zero masses would give $3240/4=810\,\mathrm{kg}$ , halving this mean by treating unusable entries as genuine masses.

Change only the analysis field from Mass to NOX; retain the same four records. This changes which entries are available, rather than deleting entire rows.

Worked example 4: an emissions-specific filter

Question. Find the mean recorded NOX in the same selected extract. Explain how the two blanks enter the calculation.

Plan. Use the two observed NOX values and count only those observed measurements.

\begin{aligned} n_{\text{observed}}&=2 && \text{records }440,1199\\ \overline{\mathrm{NOX}}&=\frac{0.026+0.043}{2}=0.0345 && \text{mean of recorded values} \end{aligned}

Why this step? Record $1199$ 's unusable mass says nothing about whether its observed NOX is usable.

Answer. Mean recorded NOX $=0.0345\,\mathrm{g\,km^{-1}}$ . Leave the two blanks missing and exclude them only from this NOX calculation; do not replace them with zero.

Check. Filling blanks with zero would instead give $0.069/4=0.01725$ , inventing two measurements.

The full workbook has $3827$ rows, with $92$ zero masses and $74$ blank NOX cells. A positive-Mass analysis therefore uses $3735$ entries; a recorded-NOX analysis uses $3753$ , before any further justified restrictions. Check these counts in software before calculating summaries. Availability can vary by vehicle type or year, so the observed subset need not represent the missing subset without bias.

Zero EngineSize and recorded CO2 can be meaningful for an electric vehicle. Zero recorded CO2 does not establish zero environmental impact over the car's life. The workbook defines particulate emissions for diesel; a blank for another fuel type can be not applicable rather than an unmeasured diesel value. The source field notes give availability percentages for the whole underlying database; do not substitute those for counts measured in this selected workbook.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account

Last Accessed

Pure Mathematics

Mechanics

Statistics

L4 - Outliers, missing values and data choices

Identifying and investigating outliers

Cleaning each field honestly

Continue this lesson.

Learn faster with Nova