SYSE.31:4 - Solution
SYSE.31:4.1 - Connect a check to the decision
Name the changed behavior, the assertion under test and the result consumer. Recover the application team’s acceptance meaning rather than deriving it from current implementation. A test that repeats the implementation’s mistaken assumption is not an independent answer to the user question.
Bind each result to the source or artifact actually exercised, relevant configuration, test input and conditions. Distinguish a test description from a completed run. A missing run, lost observation or wrong artifact cannot be reported as a pass.
Begin with a small set that discriminates the consequential failures for the current change. When necessary behavior is not specified well enough to judge, return that precise question to its owner. Do not hide it behind a large existing test count.
SYSE.31:4.2 - Put feedback where it can change the next action
Use fast focused tests for failures they can genuinely detect. Retain integration, acceptance, performance and exploratory work where their distinct results affect reliance. Choose the execution arrangement from those dependencies, not from a fixed proportion of test types.
Run relevant checks close enough to a change that the result remains actionable. Recover the actual waiting, execution and diagnosis time before choosing an improvement: a faster test does not repair a long queue or an unintelligible failure report. Parallel execution is useful only when isolation and capacity preserve the test’s meaning.
A slower test can remain necessary. Make its unresolved result visible to the decision that needs it instead of presenting early fast checks as full acceptance.
SYSE.31:4.3 - Make a failure reproducible and interpretable
Retain the tested revision/artifact, inputs, relevant environment and failure observation. Compare a suspected regression with the previous qualified behavior under matching conditions when that comparison can distinguish the cause.
Investigate variable outcomes. Shared mutable fixtures, uncontrolled clocks, order dependence and unavailable services can corrupt feedback. A nondeterministic product race is still a product defect; calling a test “flaky” does not settle which object failed.
Control a suspected source of interference and repeat a discriminating case. Change only what the comparison can account for. A rerun may help diagnosis, but a later pass does not erase an earlier failure under supported conditions.
SYSE.31:4.4 - Decide who repairs which failure
| Observed problem | Immediate response |
|---|---|
| The tested product violates the agreed behavior. | Return the failing case to the application team and withhold the reliance that required it. |
| The test asserts an obsolete or incorrect expectation. | Resolve the expectation with its owner, repair the test and re-exercise the behavior. |
| The environment or provider failed before the behavior could be observed. | Return that failure for environment/platform repair; retain the missing product evidence. |
| The result cannot be attributed to the candidate or conditions. | Recover attribution or rerun the right case; do not transfer the old pass. |
Use the actual maintainers and their permissions. Platform operators can repair provision; they do not acquire authority to change application behavior, withdraw a source change or approve release merely because they operate the test service.
If a test must be quarantined, identify the claim whose evidence was lost, its current consequence and a usable temporary replacement or stop. Name the repair responsibility and the condition for restoring the check. Quarantine is not an unlimited exemption or a way to improve the displayed pass rate.
SYSE.31:4.5 - Use the exact continuous-integration Method when integration is missing
Apply DORA Continuous integration, “How to implement CI” and “Common pitfalls” for frequent small changes to a shared mainline. Supply the actual mainline, current change, adequate build/tests, accepting application team and permission. Recheck against the current integration base when another change intervenes and check the resulting shared revision.
The first result is a checked integrated mainline revision with qualified feedback, or a blocked/restored mainline and an unresolved change returned to its team. If the mainline breaks, stop qualifying its artifacts; the authorized team repairs or withdraws the offending integration and rebuilds/retests the resulting revision. Separately green branches do not supply that result.
The source’s cadence and TDD guidance need their own fit to the team’s work; they are not universal scheduling or development-method mandates here. A checked mainline is still not an installed application or release permission.
SYSE.31:4.6 - Verify that the arrangement improves useful feedback
Exercise a known passing case, a meaningful failing change and a failure of the feedback mechanism. Check that the responsible person can distinguish them and recover the tested subject. Inspect late-discovered defects: did the arrangement omit a needed question, use the wrong conditions, or fail to deliver its result in time?
Return the checks, their supported questions and conditions, remaining evidence gaps, and the actual failure-response arrangement. Improve from useful detection and repair, not from test count or green percentage alone.