4.4 - Non-parametric tests for paired data

Paired sign comparison

Choose a direction, such as $D=\text{after}-\text{before}$. The paired sign test uses only whether each nonzero difference is positive or negative.

The no-systematic-difference null is a population median difference of zero, giving $\mathrm P(D>0)=1/2$ for continuous differences. A directional alternative changes this probability above or below $1/2$.

Worked example 1: improvement as a positive difference

Question. Eight randomly selected people take a scored task before and after training. Higher scores are better. Use a sign test at $5\%$ to investigate whether after-training performance tends to be better.

Assume the sampled pairs are independent and the population differences are continuous.

Plan. Define after minus before, then use the upper binomial tail for improvements.

The differences are $1.1,-0.4,2.3,0.7,-1.8,1.5,3.2,0.2$, so $S=6$. State $H_0$: no systematic population difference, with median $D=0$; $H_1$: after-training scores tend to be higher, with median $D>0$.

Under $H_0$, $S\sim\mathrm B(8,1/2)$, giving

Answer. Since $0.1445>0.05$, do not reject $H_0$. There is insufficient evidence that after-training performance tends to be better.

Check. Directly comparing the two score rows shows improvements for A, C, D, F, G and H: six distinct people, matching the sign count.

Worked example 2: reversing the difference direction

Question. Repeat Worked example 1 using $E=\text{before}-\text{after}$.

Plan. Rewrite the alternative before selecting the probability tail.

Now improvements have negative $E$. The differences are $-1.1,0.4,-2.3,-0.7,1.8,-1.5,-3.2,-0.2$, so the positive count is $S_E=2$.

The null is median $E=0$, and the same improvement claim becomes median $E<0$. Under $H_0$, $S_E\sim\mathrm B(8,1/2)$, so

Answer. The same probability exceeds $0.05$: there is insufficient evidence of improved population performance.

Check. Every pair has exactly one sign, so $S_E=8-S=2$. The events $S\ge6$ and $S_E\le2$ describe exactly the same sign arrangements.

Watch out

A paired sign test counts signs. Ranking differences changes the test to a Wilcoxon matched-pairs test. Do not replace a requested sign test with it.

Worked example 3: a matched-pairs rank test

Question. For eight independent random pairs, after-minus-before differences are $0.6,-0.1,0.8,1.4,-0.3,1.8,2.2,2.6$. Assume a continuous difference distribution symmetric about its median. Test at $5\%$ whether after-values tend to be higher.

Plan. Rank absolute paired differences and compare the negative rank sum with the one-tailed critical value.

State $H_0:m_D=0$, no systematic population difference, and $H_1:m_D>0$. The signed ranks are

Hence $Q=1+2=3$ and $P=36-3=33$. For $n=8$, the $5\%$ one-tailed critical value is $5$. Since $Q=3\le5$, reject $H_0$.

Answer. There is sufficient evidence at the $5\%$ level that after-values tend to be higher in the population comparison.

Check. A negative rank sum at most $3$ comes from only five rank subsets: empty, $\{1\},\{2\},\{3\},\{1,2\}$. Its probability is $5/256=0.01953125<0.05$, agreeing with the table decision.