RMP.9:4.3 - Decide whether further evidence is worth obtaining

Before designing a new trial or study, ask:

• What materially different result could change the present decision?
• Could a discriminating comparison be obtained under the actual access, capability, material and time constraints?
• Why is its whole burden warranted by that possible change?

These questions select acquisition; they do not require a new form. If existing adequate evidence supports retention or a bounded correction, finish with that answer. If a consequential decision remains open and an informative comparison is attainable and worthwhile, select the needed work.

Otherwise preserve the strongest supported position and its limits. Restrict or stop the unsupported use where the missing basis prevents it. “Further evidence is not worth obtaining now” and “existing evidence is sufficient” answer different questions. Neither an unselected experiment nor an unchanged adequate Method needs an omission certificate.

For a selected comparison, choose the research or Method Engineering contribution that can obtain the required evidence. Preserve the difference among the proposed change, intended trial, actual work, observations, inference and Method decision. A trial plan is not an obtained improvement.

In a statistical simulation, define the aim, generating situations, common estimand, candidate procedures and performance measures before judging results. Account for Monte Carlo uncertainty and failed or non-convergent outputs. Performance calculated only among successful runs can conceal a consequential weakness. Use the full observed result at its applicable reach; a favourable selected case does not establish general fit.

For a qualitative, historical or mixed inquiry, use the criticism and evidence appropriate to that purpose. Do not introduce a statistical benchmark merely to make unlike research traditions share one indicator. A reporting practice can improve inspectability while leaving the Method’s substantive adequacy unresolved.