Jump to text
On this page
Copy and source
Published source confirmed at last check

Source changed 2026-11-16 21:18:08 UTC · snapshot created 2026-11-16 21:19:40 UTC · last check 2026-11-17 19:15:10 UTC

Part III - Analyze Claims and Choose Credibility Work

RMP.5 - Analyze and Criticize the Research Claim under Its Method

Status: Stable

Type: Research Method Practice practitioner method pattern

RMP.5:1 - Problem Frame

Use this pattern to establish what claim a body of research material, an analysis or an emerging interpretation supports. Start from the proposed claim or the question the analysis is developing. Trace a consequential connection from the material through the analysis to the claim and examine which criticisms affect its support.

A simulation may reproduce a reported curve while its relation to field failures remains untested. Several valid downstream readings may establish a pattern while leaving its cause unresolved. In each case, identify the result obtained and the evidence needed for the stronger interpretation.

The ClaimCandidate states the bounded answer. Its CriticismRecord identifies the examinations performed, their bearing on the answer and any remaining uncertainty. An annotated analysis, an existing notebook or a concise qualified answer can carry both.

Use an adequate supplied analysis within its scope. For inherited material, establish the origin, method and limits needed for the present claim. The resulting answer can support a current use, narrow an earlier claim or identify the point open to challenge.

Use a qualified disciplinary method directly for a routine calculation, qualification or interpretation with adequate criteria. If an inference still needs a source, design premise or criterion, identify it and the affected claim. Access permissions and decisions about action have their own grounds. Use RMP.6 if a remaining credibility question warrants comparing ways to address it.

RMP.5:2 - Problem

An association can be misread as a cause, or repeated readings as independent cases. A theme may be reported as a prevalence estimate, archival silence as proof that an event never happened, or a calculation as validation of a model for a new use. Each stronger conclusion needs a connection that the reported result alone may leave open.

Additional analysis is worthwhile when it can clarify the inference at issue. An adequate result can be reused. If a required comparison is missing, more detail about the surviving material may support a narrower claim while leaving that comparison open.

Identify the analytical choices and contrary material that bear on the claim. Judge exploratory, interpretive and source-critical work using criteria appropriate to the method employed.

RMP.5:3 - Forces

ForceTension
A useful answerThe strongest requested claim may exceed the evidence while a narrower conclusion answers part or all of the question.
Method fitFamiliar analytical tools are accessible, while unlike questions require different forms of criticism.
Discovery and commitmentA relation noticed during analysis can justify a new question. A claim of advance commitment depends on what was recorded or otherwise established before the relevant results were known.
ReuseExisting analysis can answer the question, while a changed input, assumption or use may invalidate part of it.
Contrary materialA coherent account is easier to communicate, while discordant cases and plausible rivals can change its scope.
InspectabilityAnother critic needs the consequential reasoning, while indiscriminate recording can obscure it or expose protected material.
CompletionThe available evidence can justify ending with a bounded or unresolved answer. A further examination may be worthwhile if its attainable results could change that answer or its use.

RMP.5:4 - Solution

Identify the claim, the material and the method it relies on. Use an adequate existing analysis or perform the analysis the question requires. Examine the assumptions, contrary evidence or alternative interpretations that could affect its support. Return the bounded answer, the criticisms examined and any remaining limits.

RMP.5:4.1 - Recover the claim and one consequential connection

Specify the phenomenon, cases, population, setting, period or corpus that the claim concerns, the proposed relation and the conditions for its use. Distinguish description, interpretation, comparison, causal explanation and prediction when they require different evidence. If the claim emerged during analysis, identify when and how.

Trace the connection needed to judge the claim.

ConnectionWhat to recover
Material to observation or source claimWhich records, specimens, encounters, documents or computations are available, what they represent, and their selection, transformation, loss and access limits.
Observation or source to analysisThe method and units used, the assumptions, consequential choices or departures, and when those choices were made relative to seeing the results.
Analysis to proposed answerThe reasoning that makes the result bear on the exact claim, including the comparison or interpretation and the scope it can support.

Use the relevant RMP.3 plans and RMP.4 records when they supply part of that connection. A plan can establish an intention. Use evidence of conduct for a claim about what occurred, identifying later testimony or reconstruction as such. A missing record or unperformed comparison limits the inferences that depend on it.

Retain an existing analysis that answers the question for the relevant material, method, assumptions and use. If a premise has changed or a live criticism exposes a gap, reconsider the portion that depends on it. A new filename or the age of an otherwise applicable analysis supplies no such change by itself.

RMP.5:4.2 - Use the analysis that fits the question

Choose an analysis justified for the subject, material and inference. Use the criteria and assumptions appropriate to that discipline. The following examples show why a criticism that matters to one claim may be irrelevant to another.

Research usePractical analytical moveCriticism that can change the answer
Quantitative or experimental estimateIdentify the target quantity, units of sampling or assignment and comparison. Report the estimated magnitude and its justified uncertainty, including relevant exclusions, multiple comparisons and choices made before or after seeing results.Dependence, missing data, measurement and analytical choices can change the estimate or its interpretation. A significance label alone gives neither an estimate of effect size nor grounds for attributing a cause.
Observational or field accountRelate observations to the sites, periods, selection and measurement conditions. State the pattern established.Examine confounding, seasonality, selection and any missing comparison when attributing a cause. Repeated measurements of one site can describe variation at that site. Inference across independent sites needs evidence from those units.
Qualitative or reflexive interpretationDevelop the interpretation under the selected approach and theoretical commitments. Connect it to the research material, the researcher’s contribution and relevant discordant cases.A theme may omit a different account or extend beyond what its extracts support. In reflexive thematic analysis, judge the interpretation by its coherence with the approach and its grounding in the material.
Historical or source-critical accountExamine the dating, authenticity, provenance and purpose of the sources, their dependence and the sequence of events in question.Copies may share one source. An archive’s survival and access conditions can explain a silence. A document may establish when a change occurred while leaving its cause open. Check material that could support a contrary chronology.
Computational or model claimIdentify the target, derivation or implementation, data, environment, fitted quantities and relevant assumptions. Interpret the output for the stated target.Reproducing an output can support a claim about the implemented calculation. Calibration, verification, sensitivity and validation for a field use each examine a different connection.
Mixed inquiryAnalyze each component using its appropriate method and specify the relation claimed between the results.Check dependence, meanings, targets and disagreements before combining results. Preserve a disagreement that changes what the components jointly support.

Apply the kind of criticism the claim needs. A descriptive estimate can answer a descriptive question. In a qualitative inquiry, establish the sampling, saturation or reliability criterion from the approach and question where it is relevant. Historical claims can be examined through surviving evidence. If a disciplinary criterion is still needed, identify the claim it limits and a competent source where one is available.

RMP.5:4.3 - Criticize the consequential inference

Start with a threat that could affect the answer. Include other threats on which the claim depends. For each examination that matters, establish the following.

  1. which claim or inference could fail;
  2. which observations, alternative analysis or source comparison could distinguish the possible answers or their degree of support,
  3. what was actually examined and what was found;
  4. whether the finding sustains or changes the answer, its scope or its uncertainty.

Compare plausible rivals and defensible analytical choices, including contrary material, relevant exclusions and missing evidence. If a threat has not been examined, say which inference remains uncertain. Failure to obtain a record is evidence about the search or access. A conclusion about the proposed event needs grounds linking that absence to the event.

If an analysis chosen after seeing results reveals a relation, report the discovery, its connection to the original question and its present support. Record what was known when the choice was made so a critic can distinguish exploration from an advance commitment.

For a sensitivity analysis, identify the choices examined and their effects. Limit robustness claims to the scope the comparison supports. If a variant changes the target quantity or question, identify the separate claim each result answers.

RMP.5:4.4 - State the answer the examination supports

Give the answer warranted for the stated question and use. The following outcomes distinguish what the examination established.

  • support the bounded claim to the extent established by the material and Method;
  • narrow or qualify the claim because a condition, rival or limitation matters;
  • defeat the proposed claim or one of its necessary premises;
  • leave the named claim unresolved while preserving what is known.

State which part of the requested claim the answer supports. Downstream measurements may establish a descriptive pattern after a causal comparison is lost. A computational result can stand even when its proposed field interpretation has been rejected.

Interpret an estimate and its uncertainty interval under the method that produced them. An interval that includes zero can be compatible with zero and with nonzero effects. To assess practical sufficiency or equivalence, apply the relevant threshold or margin and the comparison rule.

Return the answer and the examinations on which it relies. A remaining limitation may justify a different conclusion or a proposal for further study. Time and resources affect what can be examined, not the strength of the available evidence.

RMP.5:4.5 - Retain the account needed for criticism and use

Keep the material, consequential choices, actual examinations, results and limitations accessible to the extent the use requires. A concise answer with its basis, a qualified notebook entry or a reference to the relevant calculation may suffice.

Provide the source, analytical choice or reason on which later reliance depends, within the permitted access conditions. A reconstruction can support claims about an earlier choice and its timing to the extent surviving evidence warrants them. Identify it as a reconstruction and state which conclusions remain open.

Apply the conditions protecting participants, confidentiality and rights. An authorized critic may examine material unavailable in a public summary. State which conclusion each can support and whether a qualified result from the authorized inspection answers the public reader’s question.

RMP.5:4.6 - Finish or return the specific missing premise

Finding about the present useUseful continuation
The analysis and its criticism already answer the bounded questionReturn that answer and its material qualification.
The research question or proposed receiving use has changedRecover the changed question with RMP.1 or compatible supplied inputs.
A needed comparison is unavailable or a premise of the selected design is defeatedState the consequence for the proposed conclusion. If the task requires a different design, use RMP.2. A justified decision to conduct no new study can finish the inquiry.
A specified operation cannot supply the intended contrastState the resulting limit. Use RMP.3 to revise the operation when a workable specification is needed, keeping any change of research question explicit.
The examination needs a connection to material or a transformation that has not been recoveredUse an adequate existing record or inspection. Where the history still needs work, use RMP.4 to establish what survives, what can be reconstructed and the limit on the conclusion.
A needed disciplinary result, criterion or permission is still missingIdentify the dependent question. Obtain the contribution from a qualified or authorized source when it is needed and available, or state the remaining limit.
A remaining credibility question could change use of the claimUse an existing qualified answer when available. Use RMP.6 if it is worthwhile to compare ways of examining the remaining threat.

A research finding can change the grounds for an engineering, policy or management decision. When the task also calls for that decision, apply its criteria and identify or exercise the relevant authority.

RMP.5:4.7 - What changes in practice

The practitioner can show why a conclusion follows, what criticism was considered, and what that examination confirmed, changed or left open. This can be a complete answer to the present question.

RMP.5:5 - Archetypal Grounding

The teaching cases use stipulated events, measurements and conditions.

RMP.5:5.1 - The seal model runs, but the field claim is still open

In a Pump-P4 analysis, investigators calibrate a model on accelerated-test observations. A run using the retained code, data and environment reproduces the reported degradation curve. The proposed field conclusion is that the maintenance interval can be extended.

The run supports the computational result under the stated inputs. Extending the field interval would also require evidence that the accelerated and field conditions preserve the mechanism relevant to that use. No qualified comparison establishes that relation in this case. Report the reproduced curve, its domain of calculation and the unresolved field-use claim.

If a qualifying parameter analysis shows that plausible material values reverse the comparison, include that sensitivity in the answer. Rerunning the same inputs examines computational repeatability. Use RMP.6 when a further sensitivity or field-validation question would affect use enough to warrant investigation. The present model result can stand, and the applicable authority makes any decision about the maintenance interval.

RMP.5:5.2 - Downstream salinity without the reference reach

The team’s RMP.3 specification calls for paired reference and downstream readings during the same thaw. Its RMP.4 trace supports the downstream readings, documents the missed reference visit and identifies no comparable reference measurement from that event. The thaw has ended.

Use the qualified measurements to describe downstream change, keeping repeated readings tied to site and time. Without the reference observations, the planned paired comparison is unavailable. The downstream series alone leaves the proposed outfall and the relevant hydrological alternatives unresolved as causes.

Return the observed pattern and the unresolved attribution. A qualified reference observation already made during that thaw, if found, could support the original comparison. Later authorized matched observations can address a new period using the conditions they actually cover.

RMP.5:5.3 - A directive date and an earlier practice

A historical claim says the practice first began after Directive-D and was caused by it. Authenticated records establish when the directive was issued. The earlier workshop series is inaccessible. Its catalogue identifies material to seek, but the evidence available here leaves the possible earlier practice open.

An authenticated earlier ledger then establishes that the practice occurred before issuance. This defeats the proposed chronology of its first occurrence. The cause of that earlier practice and any effect of the directive on later adoption remain separate questions.

RMP.5:5.4 - A situated theme is not a workforce percentage

A reflexive thematic analysis examines a small set of maintenance-team interviews under an explicit interpretive approach. A proposed theme presents record keeping as a means of blame. The supporting extracts describe defensive documentation, while another account treats the same records as useful protection during shift handover.

Examine how the investigator developed the theme and how their position shaped the reading. Include the discordant account. Here the extracts support a tension between blame and practical protection. State that interpretation and its grounds in the existing annotated material.

The interpretation concerns these accounts. Estimating a workforce percentage would need a suitable sampling and measurement design. Attributing change to a training programme would need a credible causal comparison. Coder agreement addresses neither question by itself.

RMP.5:5.5 - Repeated measurements and an uncertain contrast

Eight independent specimens are each measured three times under a supplied measurement method. The 24 readings can describe within-specimen variation. Use the eight specimens as the independent experimental units and account for the repeated readings in the analysis.

Suppose the estimated contrast is 2 scale units, with an uncertainty interval from −1 to 5 scale units. Report that estimate and interval with the comparison and methodological assumptions. If the supplied equivalence rule requires the entire interval to lie within (−6, 6) scale units, it meets the rule. It fails the same rule with a margin of (−2, 2). Without an applicable margin or practical threshold, these results alone leave equivalence and practical sufficiency open.

RMP.5:6 - Bias-Annotation

Recurring biasEffect on the answerCorrective move
Desired-conclusion biasSupportive material determines the story while contrary material disappears.Follow the consequential inference and retain examinations that narrow or defeat it.
Technique prestigeA familiar statistic, code run or interview format is treated as universal assurance.Recover the selected Method and the claim its result can support.
Retrospective certaintyAn exploratory choice is described as though it preceded seeing the material.Preserve its actual timing and the claim’s resulting qualification.
Record-volume biasVolume is taken as assurance of inspectability or support.Identify which material the inference depends on and whether the intended reader can use it or a qualified examination of it.
Closure pressureAn unresolved claim is presented as settled or used to prescribe another study.Give the bounded answer. Consider further acquisition when it could improve the answer enough to justify the work.

RMP.5:7 - Conformance Checklist

Use the questions below to assess the claim under the applicable research method.

  • The proposed or emerging claim and a consequential connection to actual material can be identified.
  • The target, relation, scope and use distinguish the descriptive, interpretive, causal or predictive claim actually made.
  • Supplied analysis is reused when its material, Method, assumptions and use remain adequate.
  • The account identifies consequential analytical choices, when they were made and any departure from the plan that affects the claim.
  • The analysis addresses the units, dependence, missing evidence, source relations and uncertainty on which its conclusion depends.
  • The conclusion accounts for relevant contrary material and defensible alternatives, including any that leave it unchanged.
  • The account distinguishes inability to inspect a record from an observed absence in a source that was examined. When absence is used as evidence against a claim, the expected recording, survival and coverage have grounds.
  • The result states the supported scope of the claim, any premise defeated and the remaining uncertainty.
  • The criticism account describes examinations actually made and retains the limits needed for the receiving use.
  • If a premise fails, the account identifies the affected claim. Where further work needs a correction or clarification, it identifies what would supply it (§4.6).
  • Proposed credibility testing has grounds in its attainable contribution and the cost of obtaining it.
  • When the finding informs an action, the applicable criteria, protections, access conditions and decision authority remain identifiable.

RMP.5:8 - Common Anti-Patterns and How to Avoid Them

Anti-patternWhy it failsPractical correction
“The graph proves the explanation”The observations can support a pattern while leaving its cause open.State what the comparison establishes and the further grounds for the proposed explanation, or the limit that remains.
“More readings give more independent cases”Repeated measurements can share the same underlying unit and dependence.Recover the unit and the justified uncertainty model.
“The model reruns, so it is valid in the field”Execution and model-to-world adequacy are different relations.Preserve the computational result and examine the proposed use separately.
“The archive is silent, so it did not happen”Source purpose, selection, loss or access conditions can explain a silence.Identify which records were examined and whether the event would be expected to leave an accessible record in that set.
“Every qualitative analysis needs coder agreement”The criterion can contradict the selected qualitative approach.Use its justified account of interpretation, researcher contribution and criticism.
“A limitation means the whole analysis failed”A bounded finding may answer the question.State the supported result and identify any further question on which the receiving work depends.

RMP.5:9 - Consequences

A reader can use the supported descriptive or computational result while limiting an unsupported extension. Existing suitable analysis can supply the answer directly, with enough of its basis to permit the examination required.

Judging the method and inference takes professional work. When a requested conclusion cannot be established, give the reason and any narrower finding the evidence supports. A specific evidential limit can itself answer the question. A separate decision may use that answer under its own criteria.

RMP.5:10 - Rationale

Analysis transforms or interprets material. Inference relates the analytical result to a claim. A correct calculation can therefore be used to draw an unwarranted conclusion. Examining that connection can improve the answer with the data already available.

Choose criticisms that can expose error in the particular inference. A sampling unit matters to an experimental estimate, a source’s dependence matters to historical attribution, and an investigator’s interpretation matters to a reflexive theme. Each method can be criticized on grounds appropriate to its use.

A bounded answer can complete the present work. Consider further testing when its attainable findings and cost justify a separate investigation.

RMP.5:11 - SoTA-Echoing

How can researchers relate an analytical result to a claim and examine the inference under the method that produced it?

The 2026 SCORE analytical-robustness study examines defensible reanalyses of the same data in social and behavioural research. The findings give grounds to identify analytical choices that affect a result. A quantitative threshold for another use needs justification in its own setting.

Braun and Clarke’s 2024 RTARG discussion and their author guidance explain how methodological coherence and the researcher’s interpretive contribution bear on reflexive thematic analysis. Apply those criteria to the approach they address.

The American Historical Association’s professional standards ground careful use of sources, inspectable interpretation and engagement with contrary evidence. They support distinguishing what an inspected record establishes from an inference about events on which the surviving archive is silent.

The National Academies’ 2019 account distinguishes computational reproducibility from replicability with new data. A successful rerun supports a claim about the computation under the conditions tested. A claim about transfer to the field needs evidence for that use.

Reconsider an analytical conclusion if new methodological evidence, changed material or a new intended use affects its grounds. Obtain a disciplinary measurement rule, causal evidence or permission from the source that supplies that particular contribution when needed.

RMP.5:12 - Relations

  • Use RMP.3 to examine or develop operational choices when their relation to the research question needs work. A suitable existing protocol can supply those choices.

  • Use RMP.4 when conduct, transformations, loss or access need to be documented or recovered. Inspectable material permits an examination. Whether the claim follows depends on that examination and its findings.

  • Use a compatible question and design already available, or RMP.1 and RMP.2 where the question changes or the selected design needs reconsideration.

  • Use RMP.6 when a remaining credibility question warrants comparing available responses. A new test requires a useful attainable result and a feasible arrangement.

  • Use FPF F.1 to find or select a source, F.0.1 to recover a source-local meaning that matters, and A.10 when the evidence for a claim or its provenance needs clarification or reproducible tracing. Use an adequate source result directly.

  • Apply the disciplinary methods and criteria relevant to the inference. Use existing access and protection conditions. An action or change in those conditions needs the corresponding authority.

  • A bounded RMP.5 answer may finish the inquiry. Use RMP.7 if that answer is needed for a compatible corpus synthesis, or RMP.8 when revisiting a broader claim or returning a change to its receiving use.

RMP.5:End

RMP.6 - Choose a Credibility Test That Bears on the Research Claim, or Answer without New Testing

Status: Stable

Type: Research Method Practice practitioner method pattern

RMP.6:1 - Problem Frame

Use this pattern to assess which credibility examination bears on a research claim, whether a new examination is worth obtaining, or what an existing test establishes. Identify the claim, its current basis, the intended use and the particular uncertainty. Use an existing RMP.5 result or another source that supplies those inputs.

Ask what each possible result would establish. A successful rerun can establish computational reproducibility under its conditions. A question about field degradation also needs evidence relating the model to the field. Another interview may deepen an interpretation. Estimating the prevalence of that interpretation requires a suitable population basis.

A CredibilityTestDecision may use a qualified existing result, select a worthwhile new examination, close with a qualified answer or identify the input still needed to choose. The current research account can carry the conclusion. To rely on a test as performed, establish what was done and what it found.

Return the claim with the support the available evidence warrants. Leave a remaining uncertainty explicit when further testing is unavailable or its expected contribution does not justify the burden.

For routine execution of a qualified engineering or assurance test, use its domain method and criteria. RMP.6 addresses what such a test can establish about the research claim. Obtain any outstanding criteria, protection, permissions, funding or authority from the practice or holder responsible for that contribution when the task requires it.

RMP.6:2 - Problem

“Repeat it,” “validate it” and “triangulate it” can name useful work or an expensive mismatch. The proposed test may target the implementation while the real concern is transfer, use copies of the same source as though they were independent, or reproduce a significance label without stating what agreement means.

Interpret the outcome of the examination. Lack of access can prevent a computation. An inconclusive replication can leave the effect open. A different result can reflect an error, changed population, changed method or another condition. Agreement may support a bounded claim while leaving a stronger interpretation unsettled.

Choose an examination by what its result can establish and whether the complete work of obtaining and interpreting it is feasible and worthwhile. A sufficient existing finding can close the question.

RMP.6:3 - Forces

ForceTension
Relevant challengeRepetition is recognizable, while the consequential threat may concern another relation entirely.
ComparabilityA new examination needs a meaningful comparison, while changed materials, context or Methods may change the claim being tested.
Evidence and resourcesStronger support can be valuable, while preparation, access, interpretation and displaced work can dominate the benefit.
Existing resultsA qualified prior examination can finish the question, while a changed premise or receiving use may limit its reuse.
Shared resourcesIndividual tests can be feasible while their combined programme is not.
Protection and authorityThe merits of a requirement can be disputed while its current terms continue to govern conduct. Changing it requires the applicable authority.
Honest returnA receiver needs an answer, while a selected or attempted test may leave the claim unresolved.

RMP.6:4 - Solution

Identify the claim, the uncertainty and the question the answer must serve. Use an adequate existing examination. If further testing is at issue, compare its attainable contributions and costs, including the option to conclude with the present evidence. Report separately the decision to obtain a test, its performance and its result when those events occur.

RMP.6:4.1 - Name what could fail and what would change

Identify the claim and scope, the analysis or sources supporting it, the intended use and any consequential uncertainty. Use the relevant RMP.5 result or other suitable inputs already available.

Examples of questions that distinguish possible credibility contributions include the following.

  • Is the reported computation obtainable from the identified data and analysis?
  • Does a plausible analytical choice change the substantive interpretation?
  • Does new corresponding material support the target relation?
  • Does the model or measure have support for this intended use?
  • Do apparently corroborating accounts have independent evidential contributions?
  • Which difference between the original and receiving context can change the relation?

Select the question whose answer could affect the claim, its support or its use. Distinguish a possible threat from evidence that it has occurred. Whether to investigate it also depends on what can be learned and at what cost.

Use the method and criteria that make the proposed result interpretable. If a missing domain criterion, source relation or suitable comparison could change the selection, identify that gap. The success criterion needs subject or decision grounds independent of the outcome the investigator hopes to see.

RMP.6:4.2 - Reuse the examination that already answers

Use an existing examination whose material, method, criterion, assumptions and scope answer the present question. If a premise or receiving use changes, assess the affected part. Repeat or extend a test when the question remains consequential and the additional examination can justify its cost.

Check the dates and scope that matter to the particular examination. A record may establish when calibration, permission or a criterion expires. An earlier result whose relevant conditions still hold can remain usable.

When the existing basis answers the question, return that answer with the qualification needed to use it.

RMP.6:4.3 - Match the credibility move to the threat

ExaminationComparisonScope of the result
Computational reproductionCompare a computed output with the reported result using the identified data, analysis, relevant environment and justified tolerance.Agreement supports reproducibility under the conditions tested. Claims about field adequacy, causation or transfer need their own evidence. If access or execution fails, no comparable output has been obtained.
New-data replicationCompare estimates for the target claim using new material and a corresponding design. Specify what was held comparable, what changed and the agreement criterion.Use magnitude, uncertainty and the applicable criterion to assess the result. Agreement or disagreement between significance labels alone leaves equivalence and cause open.
Robustness or sensitivityExamine plausible analytical choices, parameters, missingness or model assumptions under the declared question.State what changed over the examined range. A different target quantity calls for a separate claim and comparison.
Validation for a useExamine the empirical and conceptual relation between the model, measure or result and its intended use under supplied domain criteria.Fitting to calibration material or checking implementation does not by itself establish adequacy for the intended use. Use challenging material separate from fitting where the claim requires it.
Triangulation or corroborationCompare what different methods or sources establish, how their limitations are related and what agreement or disagreement would mean.Copies may trace to one account. A discrepancy can reveal a serious difference in target or method that a single average would conceal.
Transfer examinationIdentify differences in populations, settings, periods, materials or practices that affect the claimed relation.Limit the transferred claim to conditions for which the relation has adequate grounds.

Use the meaning and criterion of the examination in the relevant discipline. Select one or combine several when the question needs their distinct contributions and their results can be related meaningfully.

Match the test to the uncertainty. A simulation rerun can answer a computational question while leaving a mechanism-to-field relation open. Judge a reflexive interpretation by its grounding in the material. To establish historical chronology, examine the dates, origins and dependence of surviving sources.

RMP.6:4.4 - Compare contribution with the whole obtainable burden

Compare the attainable options, including adequate prior results and returning the present qualified answer. A single targeted examination can be sufficient.

For each serious option, identify what the attainable evidence could distinguish, what different results would imply, and what would remain open. Estimate the work of obtaining and interpreting it, including whichever of the following it requires.

  • access to sources, participants, specimens, data and code;
  • qualified personnel, instruments, preparation and computation;
  • actual performance, interpretation and any independent examination required for the proposed use;
  • participant effort, protection, legitimate restrictions and permissions;
  • timing, delay, shared capacity, necessary records and displaced work.

Use the precision the choice requires. A qualitative comparison or supplied bound can settle it. Probabilities, costs and credibility scores need suitable grounds when they affect the choice. Include the contribution to knowledge, later inquiry and other receiving questions where relevant.

Check joint feasibility when tests draw on shared capacity. If each of two examinations can require four to six non-overlapping investigator-days and six plus six is possible, ten days do not assure both. State that result. For a requested plan, compare a sufficient subset, a different arrangement or ending with the present evidence. A shorter plan still needs a worthwhile contribution.

RMP.6:4.5 - Return the current decision and specify only selected work

Give the result warranted by the question and the available facts.

Present conditionResult
An adequate existing examination answers the selected credibility questionReturn that answer, its basis and the qualification needed for use.
An additional examination has a relevant contribution and a worthwhile feasible arrangementFor a proposal, identify the comparison, required inputs, criterion, possible results and remaining limits. Select or commission the examination within the applicable authority when that is the task.
The question can finish without additional testingReturn the supported claim, qualified or adverse finding, or precise uncertainty. Include the limit that matters to the recipient.
A missing input prevents this selectionIdentify the input, the affected choice and what would permit reconsideration. Give its source if known.

Use the existing research account or give the conclusion directly. An unknown can remain a limitation of a completed answer. Leave a selection open when the missing input prevents that particular decision.

Before undertaking selected new work, specify the target claim, material, design correspondence, method, criterion or tolerance, interpretation-changing differences and practical arrangements, including when to stop. Use RMP.2 or RMP.3 if the design or its operational choices need development. Preserve evidence of actual performance and deviations when the examination is conducted, using RMP.4 where needed.

A test selection or authorization can establish a plan. Report a performed test and a favorable or adverse result from evidence of the work and its findings.

RMP.6:4.6 - Interpret an actual return without changing its kind

To interpret a completed or attempted examination, identify what was done, how it relates to the intended claim and criterion, and the effects of any deviations. Qualify deviations by their evidence and justification.

Actual returnInterpretation and continuation
The comparison was completed and supports the target relation under its criterionState that bounded support and the limits left outside the comparison.
A completed comparison yields a consequential discrepancyConfirm the target and assess whether the discrepancy reflects changed material, method, execution, context or a defect in the claim. State the result, leaving the cause open if the evidence does not distinguish it.
A completed comparison is inconclusiveIdentify what remains unresolved and what the observations still support.
Access, execution or interpretation failedReport what could not be obtained and why. Distinguish an absent output from a computed difference outside the comparison tolerance.

For a replication, preserve the estimate, uncertainty, design correspondence and success criterion. Identify and justify a change of criterion, including when it was made. Apply agreement or discrepancy to the particular claim and scope for which the comparison provides evidence.

Return the credibility finding directly when it answers the question. For further work, use RMP.5 to reconsider the affected claim, RMP.2 for a defeated design premise, RMP.3 for an operational mismatch, or RMP.4 for a needed connection to the original work. Frame a materially different research question on its own grounds.

RMP.6:4.7 - When a protective requirement itself is disputed

If the necessity or proportionality of a record, independent check, evidence threshold or participation condition is disputed, consider two questions. What protection does it provide at what cost, and who can change the requirement under the applicable rules?

Identify what is protected, the unwanted outcome, the baseline and the time horizon. Compare the expected incremental protection, uncertainty, burden and any protective work displaced by the requirement. Where a change is under consideration, compare the practical options for retaining, revising, replacing or removing it. A label or cost estimate alone cannot establish the protection it provides.

Use research evidence for the causal, measurement and evidential questions. Apply the responsible practice’s protection duties, value criteria and authority to any amendment. A binding requirement governs conduct until it expires under its terms or a competent authority changes it.

RMP.6:4.8 - Preserve a useful answer for its receiver

Return the claim, the credibility question and the supported answer. Include the source, reason or limit needed for its intended use. An adequate existing research account or test report can carry the result.

When access is restricted, identify what the public material establishes, what the permitted critic could examine and what other recipients can use from a qualified inspection. A lost record or unperformed test limits the specific conclusions that depend on it.

RMP.6:5 - Archetypal Grounding

These examples use stipulated data, tests and conditions.

RMP.6:5.1 - From a reproduced model to a relevant field comparison

A run of the Pump-P4 model with its identified data and code reproduces the reported accelerated-degradation curve. The computation is supported under those conditions. Its relevance to a longer field maintenance interval remains open. What would a further rerun establish about that use?

Another run with the same data and implementation could test repetition of the calculation. It would add no evidence about the missing relation between the accelerated and field mechanisms. Return the supported model result and state the field-use uncertainty if no worthwhile relevant examination is obtainable.

Now suppose a qualified independent field dataset becomes accessible in time and can distinguish the proposed degradation mechanisms. Assess a comparison under suitable domain criteria. If the whole comparison is feasible and its expected contribution warrants its cost, specify how it relates to the field use and how support, discrepancy or inconclusiveness would affect that use. A decision to obtain the comparison is prospective. The performed comparison supplies its finding. The maintenance interval remains a decision for the applicable authority.

RMP.6:5.2 - A blocked reproduction is not a contrary result

An analyst receives a published model claim and code, but the data needed to execute its reported analysis are not accessible under the available rights. The analyst cannot obtain the target computation.

The analyst can report the access restriction and what the code or description establishes. There is no computed output to compare and hence no observed discrepancy. If permitted access becomes available, reassess the need, feasibility and cost of reproduction. Perform a selected reproduction under its stated criterion when authorized to do so.

If a later run produces a different value, compare the target, inputs, environment and justified tolerance before judging the discrepancy and its implications for the reported claim.

RMP.6:5.3 - Historical corroboration and a qualitative receiving change

The directive’s date is established. In this case the earlier series is inaccessible and the available sources leave earlier practice unresolved. Three copies of one account add no independent evidence of earlier practice. Their shared origin can itself be established. The question can end with the remaining uncertainty.

If an authenticated earlier ledger becomes accessible, examine what it records when that would answer the chronology question. A qualified finding that the practice preceded the directive defeats the proposed sequence. The cause of the earlier practice and the directive’s effect on later adoption require different evidence.

A recipient asks whether most workers share a theme identified in a reflexive analysis of interviews. That is a new prevalence question. Agreement among coders would not supply a population basis. Give the warranted interpretation and identify the sampling and measurement the stronger claim needs. If the question concerns a discordant account that the interpretation may have omitted, examine that material under the selected qualitative approach.

RMP.6:5.4 - Two useful examinations and ten days

A research team has ten available investigator-days. Each of two proposed examinations needs four to six days of non-overlapping effort from that team. The conditions allow both to require six days in the same programme.

The arrangements 6 + 4 and 4 + 6 each fit ten days. The admissible 6 + 6 takes twelve. The pair is therefore not assured within ten days. If a programme is needed, consider a sufficient single examination, a changed arrangement or ending with the existing uncertainty.

A provider then supplies a qualified shared-preparation arrangement that bounds the combined effort by eight days and preserves both comparisons. The pair now fits ten days. To choose whether to undertake it, also consider the attainable credibility results, remaining limits and the work those eight days would displace.

RMP.6:5.5 - Two different merits answers, one authority boundary

A research protocol requires an additional independent check of a consequential record transformation. Its marginal protection is questioned.

In the first case, examination records show that the additional check detects a transformation error missed by the baseline controls. The error can change a protected exposure classification. The check fits the available window and displaces no more protective work. These findings support retaining the protection or an alternative that preserves it.

In the second case, the proposed check repeats the same source and procedure. The records show no additional ability to detect the protected failure under these conditions, and the extra check displaces an examination that can expose a consequential error. The comparison supports revising or replacing the requirement under the stated premises.

Report the evidence, uncertainty, burden and protection that led to each recommendation. The requirement continues to govern while it remains in force. A participant authorized to amend it can make the decision within that authority. Others seeking an amendment must obtain the appropriate decision.

RMP.6:6 - Bias-Annotation

Recurring biasLikely driftCorrective move
Repetition biasAny repeat is treated as a stronger test of the substantive claim.Name the precise relation being challenged and the result that could change it.
Favorable-outcome biasCriteria or tested variants are selected after seeing agreement.Preserve the intended criterion, justified changes and unexamined range.
Counting methodsEach method or source is counted as independent support.Identify the contribution and dependence of each, and what any disagreement would mean.
Acquisition biasEvery remaining uncertainty is treated as grounds for another study.Compare what an attainable examination could add with its full cost. A bounded answer can finish the task.
Authority shortcutA requirement is defended by its label or bypassed because it is costly.Examine protection merits separately from the competent decision and current force.

RMP.6:7 - Conformance Checklist

Assess the credibility question, the decision if one was requested, and any actual test result under the conditions that apply.

  • The account identifies the bounded claim, its current basis and the use or credibility question at issue. For a proposed new test, it identifies the uncertainty that test would examine.
  • The selected question identifies a possible result that would matter to the support, interpretation or use of the claim.
  • An adequate existing examination is used within its scope.
  • Reproduction, replication, robustness, validation, corroboration and transfer retain their different comparisons and limits.
  • The Method, relevant correspondence and criterion make the selected examination interpretable.
  • For a requested choice, the comparison covers the realistic options and the criteria material to that choice.
  • For proposed testing, its attainable contribution is weighed against the whole cost, including shared resources, necessary protection and displaced work.
  • A no-new-test answer preserves insufficient evidence or uncertainty where it remains.
  • A missing input is tied to the selection or conclusion that depends on it. Further acquisition is considered where the task requires it.
  • The test decision, specification, actual performance and returned result remain distinct.
  • The result distinguishes agreement, discrepancy, inconclusiveness and inability to obtain or interpret the comparison.
  • A conclusion on the merits of a disputed requirement addresses its protected function and burden. Any decision to change it has the applicable authority.
  • The receiving answer states the conclusion, conditions that affect its use and any consequential unresolved question.

RMP.6:8 - Common Anti-Patterns and How to Avoid Them

Anti-patternWhy it failsPractical correction
“The rerun validates the model”Reproducing the computation can leave its intended field use open.Use relevant independent grounds for the field relation, or report the limit if no adequate assessment is available.
“The replication was not significant, so the effect is absent”The label omits magnitude, uncertainty and the chosen comparison.Interpret the actual result under its design and criterion.
“Many copies corroborate the source”Repeated carriers can preserve one dependent account.Examine origin, dependence and the distinct evidential contribution.
“No data access means the claim failed”An unattemptable comparison supplies no contrary output.Return the access limit and the exact examination still unavailable.
“Each test fits, so the programme fits”Combined demands on shared capacity may exceed what is available.Check admissible joint combinations. Report the feasible or infeasible range, and select a useful set when a plan is requested.
“We selected the test, so credibility increased”Selection establishes a plan.Report what already supports the claim and identify the further evidence that would alter it.
“It is required, therefore it protects”A binding rule can supply a duty while leaving its marginal protection unexamined.Assess the protection where that is the question. Any change requires the applicable amendment authority.

RMP.6:9 - Consequences

An existing test can answer the stated credibility question. When relevant new evidence becomes obtainable, its expected contribution and cost may justify further work. An access failure can explain an incomplete examination, and an observed discrepancy can change the support for a particular claim.

Assessing credibility requires subject judgement about comparisons and criteria, and making a test feasible may consume scarce resources. If evidence is unavailable, identify which conclusion remains unsupported. An applicable protective requirement still governs conduct.

RMP.6:10 - Rationale

A test is informative when its possible findings bear on the claim. One targeted comparison can resolve a question that several repetitions of an unrelated computation would leave open.

Assess separately what the available evidence supports and whether it is worthwhile to obtain more. A research plan can be justified while its result is still unknown.

For a protective requirement, consider its purpose, empirical effects, costs and current authority. Evidence of a costly procedure can bear on its merits. Changing that procedure also requires the applicable decision.

RMP.6:11 - SoTA-Echoing

Which examination could change the support for the claim or clarify its limits, under which criterion and at what cost?

National Academies’ 2019 synthesis distinguishes repeating a computation on the same data from examining a claim with new data. Select the comparison that bears on the question.

The 2026 SCORE reproducibility study distinguishes data availability from computational reproduction. The replicability study examines how the criterion for success affects replication findings. The analytical-robustness study considers defensible alternative analyses. Their results concern the sampled fields, comparisons and tolerances in social and behavioural research.

Braun and Clarke’s qualitative guidance and their 2024 RTARG account explain methodological coherence in reflexive thematic analysis. The American Historical Association’s standards guide criticism of sources and their integrity in historical inquiry. Apply each kind of examination to the claims it can assess.

2021 MRC framework relates perspective, context and resource consequences to the question in complex-intervention research. RMP.6 uses that contribution when comparing attainable examinations in context.

Reconsider a comparison when a new method, source, criterion, practical arrangement or proposed use changes what the test can establish or whether it is worth obtaining.

RMP.6:12 - Relations

  • Use RMP.5 if the claim or the criticism it needs remains unclear, or if the test changes what that claim can support. Existing qualified inputs can be used directly.

  • Use RMP.2 when developing or reconsidering a research design is necessary. A justified finding that no new study is needed can finish the question.

  • Use RMP.3 when an examination needs an operational specification and RMP.4 when the course of actual work or the material retained needs to be documented or recovered. An adequate protocol or trace can be used directly.

  • Use an adequate ordinary-practice or source answer directly. RMP.1 helps when a materially changed question needs framing or it remains unclear which practice or source could answer it.

  • Use FPF A.10 when the evidence for a claim needs clarification or reproducible tracing. Use A.10.1 when a material change in a source or claim that earlier uses relied on calls for finding affected uses or revalidating several together. Use C.11.DUA to judge what new evidence could add at its full cost and A.15.9 when obtaining a bounded contribution from another practice.

  • Use applicable disciplinary methods, criteria, protections and permissions. For an action or amendment, apply the decision rules and authority of the receiving practice.

  • A completed credibility examination can inform a compatible corpus appraisal under RMP.7 or a claim or model revision under RMP.8. Use those results when the receiving work needs them. A bounded credibility answer can also finish the inquiry.

RMP.6:End