4.4(b) - Comparing groups with Mann–Whitney and t-tests

Choosing a test for two groups

Comparing groups is different from testing association between paired variables at the same sites. A group comparison asks whether the distributions or means of two populations differ. Before choosing a test, identify the measurement scale, whether observations are independent and whether the samples are related to each other.

Two independent neighbourhood samples contain different households, with no one-to-one matching between the lists. A before-and-after measurement of the same sites is paired data: each site is its own match. Equal sample sizes do not make samples paired, and different sample sizes do not prevent an independent comparison. The methods taught here concern independent samples; paired questions need methods that retain the matching.

Mann–Whitney U uses ranks and can analyse ordinal or suitable continuous observations without requiring normally distributed populations. Its general comparison concerns distributions and the tendency for one group's values to exceed another's. Interpreting it specifically as a difference in central location needs additional care, particularly similar distribution shapes. It is not automatically a test of medians in every situation.

An independent-samples t-test compares population means for numerical measurements. For small samples, approximately normal population distributions and the absence of severe outliers support the usual inference. Observations must be appropriately independent. Welch's version allows unequal population variances; the pooled version assumes equal population variances. Choosing a rank test does not repair a biased sampling frame or repeated observations masquerading as independent cases.

For an illustrative comparison of ordinal environmental-quality scores from two independent groups of streets, Mann–Whitney may be appropriate. For approximately normal numerical noise measurements from independent sites, a t-test may be suitable. If measurements are repeated on the same streets before and after a redesign, neither independent-sample procedure should be selected just because it is familiar. State the question and evidence structure first.

Hayhoe and colleagues' 2011 Upper Xingu study compared forest and soya headwater catchments using 2007–2009 stream measurements. The published study provides a real example of a group comparison where catchment area, rainfall and shared regional conditions matter. Choosing a test requires examining that data structure; the label “forest versus soya” alone does not establish independence or distributional assumptions.

Mann–Whitney ranks and U

Mann–Whitney begins by combining the two samples and ranking all observations from smallest to largest. Keep each rank's group label. Tied values receive the mean of the ranks they occupy, although ties also affect the appropriate significance calculation. Then find each group's rank sum, which captures whether that group's values tend to occupy high or low positions in the combined order.

For this lesson use the following equivalent convention for the two U statistics. Other sources may reverse the group labels; the smaller U, used for the two-sided decision here, is unchanged.

Worked separated groups

Illustrative journey times: A is 2, 3, 4, 5 minutes and B is 7, 8, 9, 10 minutes. A occupies ranks 1–4. Sum its ranks, calculate U and the complement:

The sum of the two U values equals the product of sample sizes.

Worked overlapping groups

Illustrative environmental scores: C is 1, 4, 7 and D is 2, 3, 5, 6. Combined ranks equal these ordered scores, so C's rank sum is 12:

The groups overlap substantially and the U statistics are equal. The raw rank sums cannot be compared without accounting for different sample sizes.

The null model is that the independent samples come from the same population distribution. Small U values indicate stronger separation of the groups; this decision direction differs from chi-squared, where large values are unusual. For four observations in each group without ties, the supplied exact two-sided 5% critical U is zero. The observed U of zero therefore rejects the null under the table's “at or below” rule. The sample supports a difference in journey-time distributions, with A's observed times lower than B's. It does not establish that neighbourhood alone caused it.

For larger samples or tied ranks, software may use a normal approximation and corrections rather than this small-sample table. Use a procedure suited to the actual data instead of applying the same critical value universally. NIST's rank-comparison guidance explains the rank-sum basis.