FS1 8.1 - Type I and Type II errors | Edexcel A-Level Further Mathematics

A statistical decision can be wrong in two different ways: rejecting a true claim or missing a real change. You will identify each error in context and calculate its probability using the correct population model and the test's fixed rejection rule.

Two ways a test can make an error

A test rejects a null hypothesis using a $5\%$ significance level. Does that mean there is a $5\%$ probability that this rejected hypothesis is actually true?

No. The significance calculation starts by assuming the null is true and asks how often the rule would reject it. That is a different direction of conditioning from asking whether the null is true after seeing a rejection.

You already know

A test has a null hypothesis $H_0$ , an alternative $H_1$ , and a critical region $C$ of outcomes that cause rejection. Earlier Poisson tests calculate probabilities under the null model; a conclusion of insufficient evidence does not prove $H_0$ .

To name an error, you need both the decision and the true population state. The observed count alone does not tell you whether a mistake has occurred.

Type I and Type II errors

A Type I error is rejecting $H_0$ when $H_0$ is true.

A Type II error is failing to reject $H_0$ when $H_0$ is false. Its probability is evaluated at a specified true alternative parameter.

For example, if $H_0$ says a machine's hourly fault mean is unchanged, a Type I error reports evidence of a change when there is none. A Type II error misses a change that really has occurred.

True population state	Reject $H_0$	Do not reject $H_0$
$H_0$ is true	Type I error	Correct decision
The specified alternative is true	Correct detection	Type II error

Neither error probability is the probability that a hypothesis is true. Also, Type I and Type II error probabilities are calculated under different population states, so they are not complements of each other.

Key idea

Keep truth and decision separate: Type I is a false rejection under the null, while Type II is a missed rejection under a specified true alternative.

The opening question reverses the condition. A $5\%$ significance limit controls false rejection when $H_0$ holds; it does not assign probabilities to the hypotheses after the data arrive.

False rejection is calculated under the null model

Once a rejection rule is fixed, its Type I error probability is the probability of that critical region under $H_0$ . Do not alter the cutoff during this calculation.

Probability of a Type I error

For a fixed critical region $C$ and a specified null parameter $\theta_0$ ,

\mathrm P(\text{Type I error}) =\mathrm P_{\theta_0}(X\in C)

The subscript records which true population parameter is used. For a discrete test this actual probability need not equal a stated significance limit. Not in the formula booklet: learn it.

Consider $H_0:\lambda=4$ against $H_1:\lambda>4$ , where $\lambda$ is a population mean number of faults per hour. The statistic $X$ is a one-hour count, with a suitable Poisson model. Large counts lead to rejection.

Worked example 1: the false-alarm probability

Question. The fault test rejects when $X\ge9$ . Find the probability of a Type I error, to four decimal places.

Plan. Assume the null mean really is $4$ , and calculate the inclusive critical tail.

\begin{aligned} X&\sim\operatorname{Po}(4) &&\text{true null model}\\ \mathrm P(\text{Type I error}) &=\mathrm P_{\lambda=4}(X\ge9) &&\text{rejection while }H_0\text{ holds}\\ &=1-\mathrm P_{\lambda=4}(X\le8) &&\text{include outcome }9\\ &=0.0213634\ldots &&\text{Poisson upper tail} \end{aligned}

Why this step? A Type I error assumes the original mean is correct; replacing it with an increased mean would no longer measure a false alarm.

Answer. The Type I error probability is $0.0214$ .

Check. This is below $0.05$ , consistent with the $5\%$ critical region constructed in the earlier Poisson lesson.

Worked example 2 changes only the rejection cutoff from $9$ to $10$ , making the rule more demanding.

Worked example 2: requiring a larger count

Question. Find the Type I error probability if the same hypotheses are tested by rejecting when $X\ge10$ .

Plan. Retain the true null distribution and move the tail boundary.

\begin{aligned} X&\sim\operatorname{Po}(4) &&\text{same true null}\\ \mathrm P(\text{Type I error}) &=\mathrm P_{\lambda=4}(X\ge10)\\ &=1-\mathrm P_{\lambda=4}(X\le9) &&\text{new inclusive tail}\\ &=0.0081322\ldots \end{aligned}

Why this step? The stricter rule rejects fewer outcomes, reducing the probability of a false rejection.

Answer. The probability is $0.0081$ , to four decimal places.

Check. $X\ge10$ is contained in $X\ge9$ , so its probability must be smaller under the same model.

This reduction is not a free improvement in every respect. A rule that rarely rejects can also miss genuine changes; that is a separate calculation under the alternative.

Watch out

Wrong: multiply the critical-tail probability by an invented probability that $H_0$ is true. Right: Type I error is already a conditional calculation under a true null.

Raise the cutoff one more time.

A missed change is calculated under the alternative

Return to the original rule, rejecting for $X\ge9$ . If the mean has genuinely increased, a Type II error occurs when the observed count still lies in the non-rejection region $X\le8$ .

The test's cutoff stays fixed. Change the probability model to the specified true alternative, rather than designing a new critical region using that alternative.

Probability of a Type II error

For fixed critical region $C$ , at a specified true alternative parameter $\theta_1$ ,

\beta(\theta_1) =\mathrm P_{\theta_1}(X\notin C)

This is non-rejection under the true alternative model. Its value can change with $\theta_1$ ; it is not determined by the stated significance level alone. Not in the formula booklet: learn it.

The complement of $\beta(\theta_1)$ concerns a correct detection under that same alternative. By contrast, a Type I probability uses the null parameter, so it cannot generally be found by subtracting $\beta$ from $1$ .

Worked example 3: missing a genuine increase

Question. For the rule $X\ge9$ , find the Type II error probability if the true hourly mean is $6$ , to four decimal places.

Plan. Use the actual mean $6$ , and sum the outcomes that fail to reject.

\begin{aligned} X&\sim\operatorname{Po}(6) &&\text{specified true alternative}\\ \beta(6)&=\mathrm P_{\lambda=6}(X\le8) &&\text{non-rejection region}\\ &=0.8472374\ldots &&\text{Poisson cumulative probability} \end{aligned}

Why this step? The mean really has risen, so the error consists of observing too few faults for the fixed rule to detect the increase.

Answer. The Type II error probability is $0.8472$ .

Check. A mean of $6$ is still below the rejection cutoff $9$ ; missing this change can be quite likely.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account

Last Accessed

Core Pure Mathematics

Further Pure Mathematics 1

Further Pure Mathematics 2

Further Statistics 1

Further Statistics 2

Further Mechanics 1

Further Mechanics 2

Decision Mathematics 1

Decision Mathematics 2

FS1 8.1 - Type I and Type II errors

Two ways a test can make an error

False rejection is calculated under the null model

A missed change is calculated under the alternative

Continue this lesson.

Learn faster with Nova