6.1.6 – Test and refine a program

Test evidence and fitness

Before testing a program, make its requirement precise. Without a requirement, an output can look reasonable but there is nothing objective to compare it with.

Useful test data targets different parts of a requirement:

A value can serve more than one purpose. For example, -1 is invalid and is also immediately beside the lower boundary. The important evidence is not the label alone; it is the expected result and whether the program produces it.

Worked example

Requirement: an integer score from 0 to 100, inclusive, must be classified as Accepted. Every other integer must be classified as Rejected.

Read this complete original program and predict all five outputs without running the code:

def classify_score(score):
    if (score > 0) and (score < 100):
        result = "Accepted"
    else:
        result = "Rejected"
    return (result)

test_scores = [50, 0, 100, -1, 101]
for score in test_scores:
    print(classify_score(score))

Logical reasoning gives this test table:

The normal test passes, but the two permitted endpoints fail. One passing test therefore does not establish fitness for purpose; these failed relevant tests show that the current program does not yet meet the requirement.

The table is a prediction, not a record of execution. When a task allows Run, compare its actual result in a separate column; keep the original expected result unchanged.

Refine from a failure

Refinement should be driven by a reproduced mismatch, not by changing several lines and hoping. Follow the evidence:

1. State the requirement and the exact failing input.
2. Compare the expected result with the result predicted from the code.
3. Locate the first condition or state change that disagrees with the requirement.
4. Make the smallest effective correction.
5. Re-apply the failed test and relevant normal, boundary and invalid tests.

In the score program, the first divergence is the condition. For score equal to 0, score > 0 is False, even though the requirement includes 0. For score equal to 100, score < 100 is False, even though the requirement includes 100. The smallest correction is to include equality at both endpoints.