SYSE.46:4.2 - Construct representative contrasts

Select tasks that admit useful progress and cases that require a bounded stop. Vary the location or value of a needed contribution where that distinction can reveal a failure: an adequate current premise, a genuinely missing fact, stale memory, a changed interface, unavailable support or an ambiguous effect.

Use SYSE.49 if the required task/feedback conditions must be built. Qualify fixture fidelity and reset through the appropriate domain and SYSE.33. Keep final cases and their expected outcomes outside adaptive construction or candidate selection. Synthetic consistency alone does not qualify a real-world consequence.

When the question is whether the agent chooses support well, construct matched cases with a sufficient supplied premise, a decisive fact available only through the selected support, and offered material that is irrelevant or misleading. Add verification-required and presently unanswerable cases where the receiving use needs them. Keep result, correctness basis, relevant limits and performing arrangement comparable.

For the optional contribution being tested, compare three regimes:

• No optional support: the selected contribution is unavailable; other means and protected conditions stay as declared.
• Supplied support: the protocol fixes which supported means or operation to use and makes it available. The agent still binds inputs, performs it, interprets and uses the actual return. This intervention supplies no oracle answer or assumed upper bound.
• Agent-selected support: the agent chooses whether and how to obtain the contribution under the same result and protection conditions.

Record the exact access and guidance intervention. When the task requires an external world change, retain its executor, current-state observations, permission and protected controls in every regime. Removing the actuator would defeat the task, not isolate this help decision. A withheld indispensable fact likewise changes access rather than proving an inability to reason. Preserve different valid trajectories that satisfy the result.

For people, use matched tasks and an appropriate order or allocation across participants; record learning, fatigue and carryover that can change the comparison. Repeating the same question after showing its answer would not isolate support choice. HCD.12/.13 supplies any separate claim about acquired unaided capability, transfer or retention. An ordinary tool use with adequate existing evidence needs no new experiment.