9.1.11, 2.2.12, 3.2.2, 4.2.3, D.1.4, D.1.10, D.1.12, D.1.13, D.2.1 - Choosing tests and significance

Levels of measurement

A statistical test must fit the information that a measurement provides, rather than merely the appearance of numbers in a spreadsheet.

An observation coded cooperative or non-cooperative is nominal. Replacing the words with 1 and 2 does not add equal-unit measurement; those numbers are labels. The relevant data for a chi-squared comparison are counts in categories, not means of the category codes.

A rank from least to most anxious is ordinal: the order is meaningful, but the gap between first and second need not equal the gap between second and third. A single rating from “not at all” to “very much” is ordinarily treated as ordinal because we cannot assume that every participant experiences equal psychological distances between adjacent responses.

Response time in seconds has meaningful equal intervals: the difference from 0.3 to 0.4 seconds is the same duration as from 0.4 to 0.5. It also has a meaningful zero, often called ratio measurement; for this qualification's three-way test-choice distinction, such data meet at least interval requirements. Scores expressed numerically do not automatically measure a psychological construct at interval level. Explain what the operation actually records and whether equal units are justified.

The prescribed rank tests can use ordinal data or interval-level scores converted to ranks. Ranking preserves order while discarding exact distances, which makes these tests useful when equal-interval or distribution assumptions required by other tests are not appropriate. A normal-looking graph alone does not convert category labels into interval data.

Choosing among the four tests

Choose a test by asking what relationship is being tested, whether scores are linked, and what level of information the data contain.

A test of difference asks whether conditions or groups differ. A test of correlation asks whether two co-variables vary together. Independent groups provide unrelated sets of scores from different participants; related or paired scores link each observation in one condition to an observation in the other, as in repeated measures or a suitably matched-pairs design. A correlation also contains paired measurements, but it asks about association between co-variables rather than a difference between conditions. Pairing alone therefore does not select Wilcoxon.

For example, different participants rate concentration after silence or background music: a comparison of these ordinal ratings calls for Mann–Whitney U. If the same participants supply meaningfully comparable concentration scores in both conditions, Wilcoxon preserves their paired differences. If each person reports practice hours and receives a recall score, a test of their association calls for Spearman. If independent groups choose one of two revision methods, compare the category counts using chi-squared.

Justify each relevant criterion using the study's facts. “There are numbers” is insufficient. For chi-squared, one participant's repeated category responses are not automatically independent observations; suitable cell counts are also needed, as the chi-squared lesson develops. For Wilcoxon, differences of zero are removed before ranking and before determining the analysis's $N$. For Mann–Whitney, use both group sizes; for Spearman, $N$ counts paired observations. The later test lessons teach each calculation and detailed table conditions.

Hypotheses and tails

The alternative hypothesis states the predicted difference or association, while the null hypothesis supplies the reference claim tested by the analysis.

An alternative hypothesis predicts a relationship. In an experiment it connects the operationalised IV conditions with the measured DV; in a correlation it connects the two operationalised co-variables. A null hypothesis states no population difference or no population association of the specified kind, with any observed sample pattern attributable to sampling variation under that model. It is not a claim that the sample scores must be identical.

A directional hypothesis specifies the direction in advance, such as “Participants recall more words in silence than with music” or “More practice hours are associated with more words recalled.” It normally calls for a one-tailed test. A non-directional hypothesis predicts a difference or association without specifying which way, such as “Recall differs between silence and music.” It normally calls for a two-tailed test, allowing either direction.