Appendix 1 - Data quality and evidenced geographical argument

Data quality and sources of error

Data quality depends on the use of the evidence, rather than on a single label such as “official” or “primary”.

For an enquiry about river flooding, a water-depth reading may be accurate but not valid evidence of how many households were affected. A record of claims to an insurer may document its customers' losses accurately but omit uninsured households. Representativeness concerns how well the evidence covers the population, places or times to which a conclusion refers. ONS describes statistical quality as fitness for purpose and includes relevance, accuracy, timeliness and comparability; these are questions to investigate, rather than automatic guarantees.

In an original calibration example, a reference tank is known to contain 0.10 m of water. A gauge reads 0.12 m, 0.12 m and 0.13 m. The readings cluster, but all are above the reference. A zero-setting fault could create systematic error, shifting results in one direction. Repeating the same faulty instrument many times would not remove that shift. Random error varies between readings, for example because a fluctuating surface makes the level hard to read. Repeated appropriate measurements can reveal variation and reduce its effect on an estimate; calibration is needed to investigate systematic error.

Instrument resolution limits how finely a value can be recorded. A reading to the nearest centimetre does not justify reporting many decimal places. Repeated values that look precise are not proof of accuracy. Transcription mistakes, a forgotten unit, a moved reference point and inconsistent observer instructions can also change the dataset. Keep original records and correction notes so decisions are traceable.

For qualitative evidence, consistency means documenting how prompts, recording and coding were applied. It does not mean everyone must give the same opinion. An unusual but well-documented account may reveal a real experience that an average hides. A large sample, official logo or detailed spreadsheet cannot compensate for a method that measures the wrong thing.

Comparable data and misleading presentation

A comparison can be numerically correct yet geographically misleading when boundaries, definitions or exposure differ.

Consider this original illustrative table for two fictional neighbourhoods, measured in the same year using the same definition: a home with at least one recorded flood incident.

Westbank has more recorded affected homes: 40 rather than 30. Eastbank has the higher recorded proportion: $30/300 \times 100 = 10\%$, compared with $40/800 \times 100 = 5\%$. Absolute totals address the scale of demand for help; proportions address how widespread recorded impact is among homes. They answer different questions. Neither automatically shows the severity of each incident or future flood probability.

Check the denominator, meaning the quantity against which a value is expressed. “Half of homes” is meaningless without knowing which homes, when and how selected. A household survey of volunteers cannot safely use the total population as though everyone was sampled with known probability. Missing values are not necessarily zero, and a non-response is not a negative answer.

Spatial boundaries also matter. A neighbourhood average can conceal a heavily affected riverside street. Comparing two averages does not demonstrate that every household in one area faces more risk than every household in another. Changing a ward boundary can alter its average without any household's conditions changing. Reconcile boundaries, dates, definitions and units before interpreting change; if this cannot be done, state the comparability limit.

Presentation choices influence the impression. A vertical axis starting at 35 rather than zero can exaggerate the visual difference between bars of 40 and 45; the numbers are still only five units apart. A broken or restricted scale can be legitimate when clearly marked and appropriate, but should not conceal the size of the difference. Percent change from a tiny starting value can look dramatic while the absolute change is small. A map of totals may highlight densely populated areas; a rate map may reveal a different pattern. Check the legend, scale, classification and underlying values before explaining the pattern.