RMP.9:11 - SoTA-Echoing

The working question is how to improve research practice when criticism leaves a Method decision open, without spending effort on evidence that cannot change the answer. The best-known answer selected here combines situated comparison with a prior decision about whether further evidence is worth obtaining. Method Engineering supplies reusable identity, trial, fit, worth, variant and introduction distinctions. ME.14:4.1–4.2 compares feasible courses and their full burdens, permits an adequate present choice or a retained set with its limitation, and selects further inquiry only when its obtainable result could change the choice enough to warrant the whole burden. RMP.9 adopts these stopping and acquisition distinctions and adapts the comparison to the research question, criticism requirement and affected research result in §4.3. A sufficient existing research account can carry that decision without a separate ME.14 record. At the effort needed to diagnose §5.1’s specified calculation error, retention plus verified repair already answers the use; a further efficacy study adds no discrimination. Where qualification is unresolved, preserve the uncertainty and restrict unsupported use. Reopen this source use if Method Engineering changes the relied-on stopping or acquisition conditions; reconsider a research comparison when its obtainable result becomes worth the whole burden.

For a simulation already selected, the question is which procedure answers the intended research target under the relevant conditions. RMP.9 adopts the common-target design in Morris, White and Crowther (2019), refined by Williams et al. (2024) on inspectable planning/reporting and Pawel et al. on missing outputs and non-convergence. These sources support the selected line over a comparison that silently ranks only successful runs: different failure rates can make that ranking answer another question. In §§4.3 and 5.2, preserve the common estimand, generating situations, performance criteria, Monte Carlo uncertainty and failed runs. This deliberately accepts the effort of diagnosing and interpreting failures within the comparison’s whole burden; apparent simplicity is insufficient reason to discard them. Their treatment follows the study’s aim. The sources discipline an obtained comparison, not its necessity or a universally winning Method. Reopen when a changed target, newly observed failure or stronger treatment of missing outputs changes the applicable comparison.

For reflexive thematic analysis, the question is how to repair weak interpretation while preserving that research purpose. RMP.9 adopts methodological coherence from Braun and Clarke’s RTARG contribution (2024) and their quality guidance. Section 5.3 rejects compulsory coder agreement as the repair for an undeveloped topic list: agreement can leave the interpretive defect intact. The selected work spends analytic effort on developing a coherent interpretation and reflexive scrutiny. This choice forgoes an agreement indicator that serves a different purpose; fixed-category counting would reopen the Method and criteria. Reporting guidance alone does not establish interpretive quality.

For historical extraction, RMP.9 adapts the AHA professional statement’s source-integrity and recoverable-trail requirements to §5.4’s missing marginal dates. Comparing extraction tools by speed alone cannot establish fidelity to the available originals. Correcting the bounded extraction and chronology preserves that criterion and its receiving argument, while accepting the effort of checking sources. This is a reasoned application of a professional constraint, not evidence that one OCR tool is superior. A different collection, unavailable originals or a newly attainable tool comparison reopens the applicable correction or limitation.

These comparisons preserve different research purposes rather than ranking statistical, reflexive and historical work on one scale. Their synthesis and constructed cases explain the selected action and trade-offs; they are not measured evidence that deploying RMP.9 improves research outcomes.