SH3, SH4 - Goodness of fit tests | OCR A-Level Further Mathematics B (MEI)

A goodness of fit test compares observed frequencies with those predicted by a probability model. You will test uniform, binomial and Poisson models, use supplied groupings correctly, and adjust the degrees of freedom when parameters have been estimated from the same data.

Compare observations with a specified model

Could two different probability models both survive a test on the same data? A test looks for sufficient evidence against a proposed model; it does not award a certificate saying that exactly one model is true.

You already know

The chi-squared statistic is $X^2=\sum (O-E)^2/E$ , supplied in the formula booklet. Large values count against the null model. Use the upper chi-squared tail, or a software $p$ -value, and give a cautious population conclusion.

For goodness of fit there is one variable and a proposed distribution, rather than two factors in a contingency table. Under a uniform model on four categories, each category has probability $1/4$ . A sample of $100$ therefore has expected frequency $25$ in each category.

Expected frequencies and degrees of freedom for a fixed model

If the model gives probabilities $p_1,\ldots,p_k$ for $k$ exhaustive, non-overlapping categories and the sample size is $N$ ,

E_i=Np_i

When all model parameters are specified independently of this sample,

\nu=k-1

The loss of one degree of freedom reflects the fixed total frequency.

Not in the formula booklet: learn it.

The statistic and chi-squared critical-value table are supplied, but this expected-count and degrees-of-freedom rule is not. Check that model probabilities sum to $1$ , expected frequencies sum to $N$ , and expected counts are large enough for the approximation.

Worked example 1: a uniform four-category model

Question. A device assigns each item to category A, B, C or D. In a random sample of $100$ independent items, the frequencies are $(18,22,28,32)$ . Test at the $5\%$ level whether a uniform model is suitable for the population category distribution.

Plan. Use equal expected counts, calculate the discrepancy, then use $4-1$ degrees of freedom.

$H_0$ : a uniform model on A, B, C, D is suitable for the population category distribution. $H_1$ : that model is not suitable.

\begin{aligned} E_i&=100\left(\frac14\right)=25 && \text{all four categories}\\ X^2&=\frac{(-7)^2+(-3)^2+3^2+7^2}{25} =\frac{116}{25}=4.64\\ \nu&=4-1=3,\qquad \chi^2_{3,\mathrm{critical}}=7.815\\ 4.64&<7.815 && \text{do not reject }H_0 \end{aligned}

Why this step? The null model predicts probabilities, not identical observed counts. Random frequencies can depart from $25$ without being too unusual under the model.

Answer. There is insufficient evidence at the $5\%$ level to suggest that the uniform model is unsuitable for the population category distribution.

Check. All expected counts are $25$ , their total is $100$ , and software gives $p\approx0.200$ , consistent with not rejecting.

Change only the observed allocation, keeping the sample size, model and level fixed.

Worked example 2: a larger discrepancy

Question. Repeat the test when the observed frequencies are $(10,20,30,40)$ .

Plan. Retain $E_i=25$ and $\nu=3$ , but recalculate the statistic.

$H_0$ : a uniform model is suitable for the population categories. $H_1$ : that model is not suitable.

\begin{aligned} X^2&=\frac{(-15)^2+(-5)^2+5^2+15^2}{25}=20\\ \nu&=3,\qquad \chi^2_{3,\mathrm{critical}}=7.815\\ 20&>7.815 && \text{reject }H_0 \end{aligned}

Why this step? The total is unchanged, so equal probabilities still give $25$ per category. The stronger observed pattern changes the discrepancy.

Answer. There is sufficient evidence at the $5\%$ level to suggest that the uniform model is unsuitable for the population category distribution.

Check. The $p$ -value is approximately $0.000170$ , below $0.05$ , agreeing with the comparison.

A model concerns the distribution that could produce repeated observations. It is not the data that need to “fit themselves” to a distribution.

Watch out

Wrong: “The test proves that the uniform model fits” after a small statistic.

Right: there is insufficient evidence to reject the model. Another model might also be compatible with the same sample.

The next observed allocation lies nearer the decision boundary.

Respect the supplied groups and complete the tails

For a binomial or Poisson model, the categories need not be equally likely. Calculate their model probabilities before multiplying by the sample size. A tail category must include every value it names, including values not observed in the sample.

Small expected counts can make the chi-squared approximation unreliable. Categories are often pooled so that the resulting expected counts are at least $5$ . In examination questions, the grouping is supplied: you need to understand it and use it, rather than decide your own groups.

Pooling categories

Pooling combines categories by adding their observed frequencies and adding their expected frequencies. For a pooled category, calculate one new contribution

\frac{(O_{\mathrm{pooled}}-E_{\mathrm{pooled}})^2}{E_{\mathrm{pooled}}}

Do not add the old unpooled contributions instead. Count the categories after pooling when finding the degrees of freedom.

The pooled-contribution rule follows from the supplied test statistic. The pooling and category-count rules are not printed in the formula booklet. Binomial probabilities are supplied there; their use here is assumed from earlier study.

Worked example 3: a fully specified binomial model

Question. A random sample of $100$ independent batches, each containing $4$ items, gives the following counts of defective items per batch. Test the proposed model $\mathrm B(4,0.2)$ at the $5\%$ level. Use the prescribed categories $0,\ 1,\ \ge2$ .

Number defective	$0$	$1$	$2$	$3$	$4$
Frequency	$35$	$44$	$18$	$3$	$0$

Plan. Find the three grouped probabilities, add observed frequencies within the final group, then test with no parameter estimated.

$H_0$ : $\mathrm B(4,0.2)$ is suitable for the population count per batch. $H_1$ : that model is not suitable.

\begin{aligned} P(X=0)&=0.8^4=0.4096\\ P(X=1)&=4(0.2)(0.8)^3=0.4096\\ P(X\ge2)&=1-0.4096-0.4096=0.1808 \end{aligned}

The grouped observed counts are $(35,44,21)$ , and expected counts are $(40.96,40.96,18.08)$ . Therefore

\begin{aligned} X^2&=\frac{(35-40.96)^2}{40.96} +\frac{(44-40.96)^2}{40.96} +\frac{(21-18.08)^2}{18.08}\\ &=1.5644\ldots\\ \nu&=3-1=2,\qquad \chi^2_{2,\mathrm{critical}}=5.991\\ 1.5644\ldots&<5.991 && \text{do not reject }H_0 \end{aligned}

Why this step? The value $0.2$ was specified, not fitted from these frequencies, so it does not cost an additional degree of freedom.

Answer. There is insufficient evidence at the $5\%$ level to suggest that $\mathrm B(4,0.2)$ is unsuitable for defective-item counts per batch.

Check. The expected counts sum to $100$ and all exceed $5$ . Before pooling, the expectations for $3$ and $4$ would be $2.56$ and $0.16$ , explaining why sparse tail categories need attention.

Change only the proposed success probability, from $0.2$ to $0.25$ . Keep the observations, prescribed grouping and significance level unchanged.

Worked example 4: a different fixed model on the same data

Question. Test $\mathrm B(4,0.25)$ using the same data and the categories $0,\ 1,\ \ge2$ .

Plan. Recalculate the model probabilities, retaining three categories and no fitted parameter.

$H_0$ : $\mathrm B(4,0.25)$ is suitable for population counts per batch. $H_1$ : that model is not suitable.

\begin{aligned} P(X=0)&=0.75^4=0.31640625\\ P(X=1)&=4(0.25)(0.75)^3=0.421875\\ P(X\ge2)&=0.26171875 \end{aligned}

Thus

\begin{aligned} O&=(35,44,21),\qquad E=(31.640625,42.1875,26.171875)\\ X^2&=\frac{(35-31.640625)^2}{31.640625} +\frac{(44-42.1875)^2}{42.1875} +\frac{(21-26.171875)^2}{26.171875}\\ &=1.4566\ldots\\ \nu&=2,\qquad 1.4566\ldots<5.991 && \text{do not reject }H_0 \end{aligned}

Why this step? Changing a specified model probability changes the expected frequencies, but does not mean that probability has been estimated from this sample.

Answer. There is insufficient evidence at the $5\%$ level to suggest that $\mathrm B(4,0.25)$ is unsuitable for the population batch counts.

Check. The expectations again sum to $100$ . Both this model and the previous model survive: the same data need not identify one unique distribution.

Pooling is based on small expected frequencies, not simply a small observed count. An observed zero can be quite informative when its expected count is substantial.

Watch out

Wrong: omit a category because it has no observations, or replace the tail probability by the probability of just its first value.

Right: retain all model outcomes in the supplied categories. For a binomial model, $\ge2$ here includes $2,3,4$ ; for a Poisson model, an upper tail continues indefinitely.

Change the fixed probability again, while retaining the same grouping.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account

Last Accessed

Core Pure

Mechanics (Major and Minor)

Statistics (Major and Minor)

Modelling with Algorithms

Numerical Methods

Extra Pure

Further Pure with Technology

SH3, SH4 - Goodness of fit tests

Compare observations with a specified model

Respect the supplied groups and complete the tails

Continue this lesson.

Learn faster with Nova