RMP.3:11 - SoTA-Echoing
Practice question. How much and what kind of advance specification preserves a research question while allowing the inquiry its justified flexibility?
Selected line. Adapt question-relative, method-sensitive specification with inspectable commitments and amendments. Compared with either a universal protocol checklist or an unrestricted evolving notebook, this preserves both methodological fit and the timing needed for criticism. It deliberately requires reasons for consequential choices; it does not require more documentation where an adequate domain protocol already supplies them.
| Comparison and source role | Adopt, adapt or reject; effect on the Method and limit |
|---|---|
| Flake and Fried’s Measurement Schmeasurement is a methodological candidate against unexplained measure selection. Goos et al.’s 2026 measurement study supplies bounded failure evidence from Many Labs measures. | Adapt the recoverable construct–measure relation and actual administration/scoring choices in §4.2. Reject publication or replication status as sufficient qualification for a changed measurement use. The sources neither validate the local instrument nor establish a universal research failure rate. |
| Lakens’s Sample Size Justification compares quantitative justifications tied to inferential aims and constraints. | Adapt its question-relative rationale in §4.2. Reject an unexplained inherited count and the treatment of a resource-limited sample as automatically sufficient. Applicable statistical calculations remain supplied Methods; power analysis is not a universal standard for qualitative or historical adequacy. |
| SPIRIT 2025 is a bounded reporting comparator; TOP 2025 distinguishes registration, protocol and analysis-plan practices. | Adapt consequential specification and separate timed commitments in §§4.2–4.4. Reject the checklist or preregistration label as proof of design quality. Trial-specific objects apply where that design has them, not to every inquiry. |
| Haven et al.’s qualitative preregistration study supplies a candidate starting account. Braun and Clarke’s RTARG argument and review challenges generic qualitative reporting and assessment defaults. | Adapt explicit starting commitments with approach-appropriate evolution in §4.4. Reject universal pre-data freezing and universal coder agreement. The Delphi consensus and reflexive-TA critique do not establish one procedure for all qualitative traditions. |
| AHA’s professional standards and the Cochrane Handbook, chapter 2 supply source-critical and question-led selection comparators. | Adapt provenance, contrary material and question-relative selection in §§4.2 and 4.4. These practices cannot restore a lost archive or supply the actual historical or synthesis conclusion. |
The combined arrangement is a reasoned methodological synthesis, not a demonstrated universal improvement over every research practice. Reopen the affected choice when a changed construct, approach, source corpus or access condition defeats its research connection, or when a better domain procedure preserves that connection and its criticism with less burden.