SYSE.31:4.3 - Make a failure reproducible and interpretable

Retain the tested revision/artifact, inputs, relevant environment and failure observation. Compare a suspected regression with the previous qualified behavior under matching conditions when that comparison can distinguish the cause.

Investigate variable outcomes. Shared mutable fixtures, uncontrolled clocks, order dependence and unavailable services can corrupt feedback. A nondeterministic product race is still a product defect; calling a test “flaky” does not settle which object failed.

Control a suspected source of interference and repeat a discriminating case. Change only what the comparison can account for. A rerun may help diagnosis, but a later pass does not erase an earlier failure under supported conditions.