6.5 - Hypotheses and critical regions

Null and alternative hypotheses

A machine has historically produced a satisfactory item with probability $0.6$. After an adjustment, $15$ of $20$ independently chosen items are satisfactory. Has the machine improved, or is this a lucky sample? The observed proportion $15/20$ describes these items; the question concerns the machine's underlying probability $p$.

A test statistic is a quantity calculated from the observations and used to make the decision. Here, $X$, the number of satisfactory items among $20$, is convenient. Under $H_0$, $X\sim\mathrm B(20,0.6)$. The statistic is random before sampling; its observed value is $15$.

The question you decide to investigate determines the tail:

The first two are one-tailed tests; the third is two-tailed. Choose this direction from the investigation before looking at its result. Choosing whichever tail looks more striking after sampling changes the chance of rejection.

Keep the population and sample size, but change the research question from improvement to any change. Now a deterioration matters too.

A hypothesis is about the population, so it should not say that the sample count or sample proportion equals its observed value.

Translate a different investigation into a parameter statement.

Significance and rejection

A test needs a rule for deciding what counts as unusually far from the null model. Consider $8$ independent tosses of a coin and let $X$ be the number of heads. We investigate whether the coin favours heads, so $H_0:p=0.5$ and $H_1:p>0.5$.

Under $H_0$, a high count can still occur by chance. For example,

A rule that rejects for $X\ge7$ therefore rejects a fair coin on about $3.52\%$ of repeated samples of this size.

The regions concern possible results, not the unknown parameter. Here, $\{7,8\}$ is a rejection region and $\{0,1,\ldots,6\}$ is its acceptance region. “Acceptance” is a conventional region name; it does not mean the null has been proved.

[DIAGRAM: asset_name: A discrete rejection region; asset_slug: hypotheses-binomial-region; caption: Seven or eight heads form a rejection region with null probability 9 over 256; description: A bar chart of B eight comma one half shows counts zero through eight. Counts seven and eight are shaded clay and labelled rejection region. Counts zero through six are pale blue and labelled acceptance region. The total null probability of the rejection region is 0.03515625, below the chosen five percent level.]

For a one-tailed test, choose the largest suitable tail region whose null probability does not exceed the level. A larger region would reject more often than allowed. For a two-tailed test at level $\alpha$, the usual equal-tail rule allocates $\alpha/2$ to each tail. With discrete counts the attained tails need not have exactly equal probabilities, and their total may be below $\alpha$.