S1 1.1 - Modelling with data

The statistical modelling cycle

If a delivery is late on one day in four, should you expect exactly five late deliveries in the next twenty? A model can predict a useful average without predicting the exact sequence of days.

Consider an illustrative delivery service. We want to predict the number of deliveries more than ten minutes late next month. First define “late” precisely; otherwise different people may record different events. Collect relevant recent delivery records, decide how to describe the uncertainty, and calculate predictions. Finally compare predictions with new observations.

For example, a model might assign probability $0.25$ to a late delivery. It does not claim that every fourth delivery must be late. In $20$ deliveries it predicts an average count of

There could still be $3$, $6$ or another number in a particular month. The prediction concerns repeated comparable sets of deliveries.

The modelling cycle connects mathematics to the original problem:

1. State the practical question and decide what to measure.
2. Make assumptions and formulate a model.
3. Use the model to obtain predictions.
4. Compare predictions with observations in their practical context.
5. Retain, refine or replace the model, then check again.

Data can help formulate a model and later help assess it. When possible, use new observations for checking: a model made to fit one set of records may describe those records well and predict future records poorly. A useful model need not reproduce every detail, but it must be suitable for the question being asked.

Models simplify complicated situations so that we can understand patterns, describe relationships and explore possible changes. A useful explanation identifies what the model helps us understand or solve. Merely calling it “easy” or “cheap” does not explain what it represents, and a model does not automatically give more accurate results.

A prediction of an average count should not be read as a guarantee about one month.

Simplifying assumptions

An assumption leaves out detail deliberately, so that the model can answer a manageable question.

A single probability of lateness treats the deliveries under study as having the same chance of being late. That could be useful if they cover similar routes at similar times. If morning deliveries and evening deliveries behave very differently, pooling them may hide a pattern important to the prediction.

Suppose you model a numbered spinner by giving each of its four sectors probability $1/4$. Equal-sized sectors and a freely turning pointer support that choice. A bent pointer or an uneven pivot might make it unsuitable. “There are four possible results” alone does not justify equal probabilities.

There are three useful questions to ask about an assumption:

Do not confuse an assumption with a fact found in the records. “The model uses probability $0.25$” is a choice. “There were $25$ late deliveries in the $100$ recorded” is an observation. The observation may support the choice without proving it will hold forever.

A model also has a domain of use: the conditions in which it is intended to work. A weekday delivery model may need checking before it is used for public holidays. More detail is useful only if it improves the answer enough to justify the extra complication.