RMP.6:4.3 - Match the credibility move to the threat
| Move | Comparison to make | What the result cannot establish by itself |
|---|---|---|
| Computational reproduction | Use the identified data, analysis and relevant environment to compare the output actually obtained with the reported result under a justified tolerance. | Agreement does not establish the model’s substantive adequacy, a causal claim or transfer to another setting. Missing access and unsuccessful execution are not discrepant outputs. |
| New-data replication | Compare a target claim using new material and a justified corresponding design. State what is held comparable, what differs and the agreement criterion. | A matching or non-matching significance label alone does not settle effect magnitude, uncertainty, equivalence or every proposed explanation. |
| Robustness or sensitivity | Examine plausible analytical, parameter, missingness or model choices that can change the conclusion under the declared question. | Favorable selected variants do not establish robustness over unexamined choices; a changed target quantity is a different claim, not another interchangeable estimate. |
| Validation for a use | Examine the empirical and conceptual relation between the model, measure or result and its intended use under supplied domain criteria. | Fitting to calibration material or checking implementation does not by itself establish adequacy for the intended use. Use challenging material separate from fitting where the claim requires it. |
| Triangulation or corroboration | Compare what differently limited Methods or sources contribute, their dependence and the relation expected among their results. | Agreement among copies is not independent corroboration; differing results need not be averaged into a common claim. |
| Transfer examination | Identify differences between source and receiving populations, settings, periods, materials or practices that can change the claimed relation. | Success in the source context does not supply an unrestricted target-context result. |
Use a disciplinary variant with its explicit meaning when that changes the examination. The moves are a repertoire, not stages or a maturity scale. Several can be useful together only when their contributions and the relation between their results justify the combination.
A test must bear on the actual threat. Reproducing a simulation cannot replace a missing mechanism comparison. An inter-coder reliability procedure cannot validate a reflexive thematic interpretation merely because both involve interviews. Replicating a unique historical event is not the test of its source chronology; source criticism or corroboration may be.