RMP.6 - Choose a Credibility Test That Bears on the Research Claim, or Answer without New Testing

Type: Research Method Practice practitioner method pattern

RMP.6:1 - Problem Frame

Use this pattern when a bounded research claim is available and someone needs to decide which credibility examination could change its interpretation or proposed use, and whether to obtain that examination. Enter with the claim, its current basis, proposed use and a specific live threat. A new RMP.5 run or passage through the earlier numbered patterns is not required.

Begin by asking what a possible result would change. A successful rerun can answer whether a computation is reproducible; it cannot by itself answer whether the model represents field degradation. Another interview may deepen an interpretation without estimating its prevalence. The useful comparison depends on the claim and the threat.

The first result is a CredibilityTestDecision: use an adequate existing result, select an additional examination, finish with a qualified answer without new testing, or identify a missing input that prevents this particular selection. The result may fit in the current answer or research account. Selecting a test is distinct from specifying it, performing it and obtaining its result.

The practical gain is relevant credibility work and an honest completion when more testing would add little, cannot be obtained or does not warrant its whole burden. An unsupported claim remains unsupported.

For routine execution of an already qualified engineering, assurance or other domain test under supplied criteria, use that Method. RMP.6 addresses the research claim and its credibility relation. It supplies neither missing domain criteria nor safety, ethics, access, funding or receiving-decision authority.

RMP.6:2 - Problem

“Repeat it,” “validate it” and “triangulate it” can name useful work or an expensive mismatch. The proposed test may target the implementation while the real concern is transfer, use copies of the same source as though they were independent, or reproduce a significance label without stating what agreement means.

Tests also have unlike possible outcomes. Failure to access data is not a discrepant computation. An inconclusive replication is not proof of no effect. A different result may expose a defect, a changed population, an altered Method or an unrecognized condition. A favorable result can support one bounded claim while leaving a stronger claim open.

The practitioner needs to select the examination whose result can matter, preserve what it could establish and assess whether the whole arrangement is obtainable and worthwhile. Otherwise every uncertainty can become a permanent demand for another study.

RMP.6:3 - Forces

Force	Tension
Relevant challenge	Repetition is recognizable, while the consequential threat may concern another relation entirely.
Comparability	A new examination needs a meaningful comparison, while changed materials, context or Methods may change the claim being tested.
Evidence and resources	Stronger support can be valuable, while preparation, access, interpretation and displaced work can dominate the benefit.
Existing results	A qualified prior examination can finish the question, while a changed premise or receiving use may limit its reuse.
Shared resources	Individual tests can be feasible while their combined programme is not.
Protection and authority	A requirement’s empirical merits may be disputed, while its current force still governs conduct.
Honest return	A receiver needs an answer, while a selected or attempted test may leave the claim unresolved.

RMP.6:4 - Solution

Recover the claim, threat and receiving question. Use sufficient existing examination results first. Compare real candidate contributions, select only useful obtainable work, and keep the decision separate from actual performance and its interpretation.

RMP.6:4.1 - Name what could fail and what would change

State the exact claim and its scope, the analysis or source basis, the proposed use and the uncertainty that remains. Recover the relevant RMP.5 claim and criticism when supplied; use compatible direct inputs otherwise.

Turn a general concern into a discriminating question. For example:

Is the reported computation obtainable from the identified data and analysis?
Does a plausible analytical choice change the substantive interpretation?
Does new corresponding material support the target relation?
Does the model or measure have support for this intended use?
Do apparently corroborating accounts have independent evidential contributions?
Which difference between the original and receiving context can change the relation?

These questions identify possible contributions, not six mandatory tests. Select the question whose answer can alter the current claim or use. A merely imaginable failure does not establish a defect or a reason to commission research.

Identify the Method and criteria that make a result interpretable. If a domain criterion, source relation or admissible comparison is missing and could change the selection, return that precise input. Do not manufacture a success threshold from a preferred outcome.

RMP.6:4.2 - Reuse the examination that already answers

An existing result can answer the present credibility question when its claim, relevant material, Method, criterion, assumptions and scope support the proposed use. Recover those relations and the result’s qualifications. Repeat only the affected examination when a changed premise, execution defect or new use makes it necessary.

A result’s age alone does not invalidate an unchanged premise. Actual expiry of a required calibration, permission or applicable criterion can matter. The relevant owner supplies that condition; RMP.6 does not infer either validity or invalidity from a date label alone.

If the existing basis is sufficient for this question, return the answer and its needed limitation. The current account may already be complete. An unused test catalogue, comparison table or separate justification for not testing is unnecessary.

RMP.6:4.3 - Match the credibility move to the threat

Move	Comparison to make	What the result cannot establish by itself
Computational reproduction	Use the identified data, analysis and relevant environment to compare the output actually obtained with the reported result under a justified tolerance.	Agreement does not establish the model’s substantive adequacy, a causal claim or transfer to another setting. Missing access and unsuccessful execution are not discrepant outputs.
New-data replication	Compare a target claim using new material and a justified corresponding design. State what is held comparable, what differs and the agreement criterion.	A matching or non-matching significance label alone does not settle effect magnitude, uncertainty, equivalence or every proposed explanation.
Robustness or sensitivity	Examine plausible analytical, parameter, missingness or model choices that can change the conclusion under the declared question.	Favorable selected variants do not establish robustness over unexamined choices; a changed target quantity is a different claim, not another interchangeable estimate.
Validation for a use	Examine the empirical and conceptual relation between the model, measure or result and its intended use under supplied domain criteria.	Fitting to calibration material or checking implementation does not by itself establish adequacy for the intended use. Use challenging material separate from fitting where the claim requires it.
Triangulation or corroboration	Compare what differently limited Methods or sources contribute, their dependence and the relation expected among their results.	Agreement among copies is not independent corroboration; differing results need not be averaged into a common claim.
Transfer examination	Identify differences between source and receiving populations, settings, periods, materials or practices that can change the claimed relation.	Success in the source context does not supply an unrestricted target-context result.

Use a disciplinary variant with its explicit meaning when that changes the examination. The moves are a repertoire, not stages or a maturity scale. Several can be useful together only when their contributions and the relation between their results justify the combination.

A test must bear on the actual threat. Reproducing a simulation cannot replace a missing mechanism comparison. An inter-coder reliability procedure cannot validate a reflexive thematic interpretation merely because both involve interviews. Replicating a unique historical event is not the test of its source chronology; source criticism or corroboration may be.

RMP.6:4.4 - Compare contribution with the whole obtainable burden

Compare only real alternatives, including adequate existing results and no additional testing. One targeted examination can be enough; a fixed count of alternatives or tests adds no assurance.

For each serious candidate, state which possible result could change the answer, what the attainable material and Method can discriminate, what limits would remain, and the whole arrangement needed to obtain and interpret the result. Include:

access to sources, participants, specimens, data and code;
qualified personnel, instruments, preparation and computation;
actual performance, interpretation and any independent return the claim needs;
participant effort, protection, legitimate restrictions and permissions;
timing, delay, shared capacity, necessary records and displaced work.

Use the precision the choice needs. Do not invent probabilities, monetary values or a universal credibility score when a qualitative comparison or a supplied bound settles it. The value of research includes a defensible contribution to knowledge, later inquiry and several receiving questions; immediate commercial payoff is not its sole measure.

Test resource combinations when candidates share or compete for capacity. Two examinations that can each require four to six non-overlapping investigator-days do not form an assured ten-day programme while six plus six remains admissible. Choose a useful sufficient subset, obtain a genuinely different arrangement or decline the combined programme. A shorter arrangement still has to warrant its burden.

RMP.6:4.5 - Return the current decision and specify only selected work

Choose the result that the current facts warrant:

Present condition	Result
An adequate existing examination answers the selected credibility question	Return that answer, its basis and the qualification needed for use.
An additional examination has a relevant contribution and a worthwhile obtainable arrangement	Select the comparison and state what its possible results would change, its required inputs, criterion and remaining limits.
The present question can finish without additional testing	Return the supported claim, narrowing, refusal or explicit uncertainty with the reason or limitation the receiver needs. Infeasibility or low value of testing does not improve the evidence.
A missing input prevents this particular selection	Name the exact result, owner interface, affected choice and condition for reconsidering it. A missing-input return does not commission obtaining it.

These are current answers, not mandatory documents. A material unknown can remain a limitation of a completed no-new-test answer when no acquisition is selected. Use a hold only for the particular selection it actually prevents.

For selected new work, make the proposed comparison executable and interpretable before relying on it. State the target claim, material and design correspondence, relevant Method, criterion or tolerance, possible interpretation-changing differences, obtaining arrangement and useful stop. A new or changed design may need RMP.2; consequential operational choices may need RMP.3. Actual conduct and deviations need an adequate RMP.4 trace.

The selected examination remains prospective. Its specification and permitted arrangement do not establish that it was performed or that it produced a favorable result.

RMP.6:4.6 - Interpret an actual return without changing its kind

When an examination returns, compare what was performed with what the intended claim and criterion require. Retain justified deviations and their effects. Distinguish the following:

Actual return	Interpretation and continuation
The comparison was completed and supports the target relation under its criterion	State that bounded support and the limits left outside the comparison.
The comparison was completed and yields a consequential discrepancy	Examine whether it bears on the same claim and whether changed material, Method, execution or context explains the discrepancy; revise or return the affected claim.
The comparison was completed but remains inconclusive	State the unresolved discrimination and the support that still obtains; do not convert it into agreement or evidence of absence.
Access, execution or interpretability failed	State what could not be examined and why. An unattemptable or failed computation is not a discrepant computational result.

For replication, preserve the estimate, uncertainty, design correspondence and stated success criterion. A changed criterion may be defensible, but it must be identified and justified rather than silently substituted after seeing the result. A single discrepancy does not automatically falsify every associated explanation; equally, a favorable repeat does not confer unlimited credibility.

Return a changed or defeated claim to RMP.5, a design premise to RMP.2, an operational mismatch to RMP.3, or a missing trace relation to RMP.4. A materially different research question requires its own qualified frame. If the selected credibility question is answered, return the answer directly; no further research cycle is automatic.

RMP.6:4.7 - When a protective requirement itself is disputed

A record demand, independent check, evidence threshold or participant condition may have contested necessity or proportionality. In that case, separate research about its merits from the authority to change its current force.

Make the merits question concrete: who or what is protected, from which unwanted outcome, under which baseline conditions; what incremental protection the requirement is expected to provide; over which horizon and with what uncertainty; and what burden or displaced protective work it creates. Compare realistic retention, revision, replacement or removal options. A required label is not evidence of marginal protection, and a lower cost is not evidence that the protection is unnecessary.

RMP can examine the causal, measurement or evidence basis of that comparison. The responsible receiving practice retains the value judgement, protection duties and competent amendment decision. A binding condition continues to govern actual conduct until a competent change. Ordinary uncontested inquiry need not create a requirement-review dossier.

RMP.6:4.8 - Preserve a useful answer for its receiver

Return the claim, the selected credibility question and the conclusion that can now be used. Carry the source, limitation or reason that changes the receiving decision. Reuse the existing research account; do not copy an adequate test report into a new certificate.

Where the receiver lacks access to decisive material, distinguish its public answer from an examination available only to a permitted critic. A lost record, inaccessible original or unperformed test cannot be recovered by stronger wording. Keep that limit without erasing the bounded contribution that remains.

RMP.6:5 - Archetypal Grounding

The following cases have constructed premises. They illustrate choices and returns, not performed studies or actual permissions.

RMP.6:5.1 - From a reproduced model to a relevant field comparison

The Pump-P4 model produces the reported accelerated-degradation curve from its identified data and code. RMP.5 has retained that computational result and left field relevance unresolved. The question now is whether another rerun would support a longer maintenance interval.

The live threat concerns the mechanism-to-field relation. The existing reproduction already answers its computational question; repeating it cannot supply the missing field comparison. Return the limited model result. If no worthwhile relevant examination is obtainable, finish with the field-use uncertainty intact.

Suppose an independent qualified field dataset then becomes accessible within the available window and can distinguish the competing degradation mechanisms. Select a use-specific validation or mechanism comparison under the supplied domain criteria. Specify the correspondence with the intended field use and the result that would narrow or defeat that use. The decision selects research; only the later actual comparison supplies its result, and the maintenance authority still chooses the interval.

RMP.6:5.2 - A blocked reproduction is not a contrary result

An analyst receives a published model claim and code, but the data needed to execute its reported analysis are not accessible under the available rights. The analyst cannot obtain the target computation.

The result is an access-limited attempt, not a reproduction with a discrepant output and not evidence that the underlying claim is false. State the unavailable comparison and what the supplied code or description can still support. If a permitted data arrangement later becomes available, assess and perform the actual reproduction under its criterion.

If that later computation returns a different value, preserve the distinction: now there is an observed discrepancy to examine. Check the target, inputs, environment and justified tolerance before deciding which part of the reported claim changes.

RMP.6:5.3 - Historical corroboration and a qualitative receiving change

A directive’s date is established, but whether the practice preceded it remains unresolved because an earlier archive series is inaccessible. Reprinting the directive or locating three copies of the same account supplies no independent earlier chronology. The current answer retains that limitation; no further search is automatically commissioned.

An authenticated earlier ledger becomes accessible and can settle the relevant chronology. Source-critical corroboration is now a useful candidate. Its result may defeat the claim that practice began after the directive without establishing the cause of the earlier change.

In a different inquiry, a receiver asks whether a reflexive thematic account proves that most workers share the theme. The live issue is the new prevalence claim. More coder agreement on the same interviews does not supply its population basis. Preserve the situated interpretation and return the new question for an appropriate design. If the actual concern is instead whether the interpretation suppresses a discordant account, examination of that material under the selected qualitative Method can be the relevant contribution.

RMP.6:5.4 - Two useful examinations and ten days

A research team has ten available investigator-days. Two proposed credibility examinations each need four to six days of non-overlapping effort from that same team. Supplied bounds do not exclude six plus six.

Do not select the pair as an obtainable ten-day programme by separately comparing six plus four and four plus six. Determine whether one examination can answer the current credibility question, whether the arrangement can change, or whether the present answer should retain uncertainty without the pair.

A provider then supplies a qualified shared-preparation arrangement that bounds the combined effort to eight days while preserving both discriminating comparisons. The feasibility answer changes. Reassess the contribution, remaining limits and displaced work before selecting the now obtainable programme; eight days is not itself evidence that the tests are worth doing.

RMP.6:5.5 - Two different merits answers, one authority boundary

A research protocol requires an additional independent check of a consequential record transformation. Its marginal protection is questioned.

In the first constructed case, supplied examination records show that the check catches an otherwise missed transformation error capable of changing a safety-relevant exposure classification. The baseline controls miss that error, and the check fits the available window without displacing more protective work. The evidence supports retaining that protection or an alternative that preserves its function.

In the second case, supplied records show that the proposed check repeats the same source and procedure, adds no demonstrated discrimination of the protected failure, and displaces a different examination that can expose a consequential error. The merits comparison supports revising or replacing the duplicate requirement under those premises.

These are different substantive recommendations. Neither case transfers amendment authority to the researcher. Retain the in-force condition while returning the evidence, uncertainty, burden and protected-function comparison to its competent owner. A favorable resource argument alone never grants a waiver.

RMP.6:6 - Bias-Annotation

Recurring bias	Likely drift	Corrective move
Repetition bias	Any repeat is treated as a stronger test of the substantive claim.	Name the precise relation being challenged and the result that could change it.
Favorable-outcome bias	Criteria or tested variants are selected after seeing agreement.	Preserve the intended criterion, justified changes and unexamined range.
Quantity-of-methods bias	More Methods or sources are treated as independent support.	Recover their distinct contributions, dependence and disagreement logic.
Acquisition bias	Every remaining uncertainty becomes another study.	Compare attainable contribution with the full arrangement and allow a bounded completion.
Authority shortcut	A requirement is defended by its label or bypassed because it is costly.	Examine protection merits separately from the competent decision and current force.

RMP.6:7 - Conformance Checklist

Apply these questions to the current credibility decision and any actual return, not to an imagined complete battery of tests.

Direct entry identifies the bounded claim, current basis, proposed use and live threat.
The selected question has a possible result that can change the claim or useful next move.
An adequate existing examination can complete the question without an unused test inventory.
Reproduction, replication, robustness, validation, corroboration and transfer retain their different comparisons and limits.
The Method, relevant correspondence and criterion make the selected examination interpretable.
Real alternatives are compared without a numerical quota or universal credibility score.
Contribution is compared with complete obtainable burden, shared resources, protection and displaced work.
A no-new-test answer preserves insufficient evidence or uncertainty where it remains.
A missing-input return names the dependent selection without commissioning acquisition.
The test decision, specification, actual performance and returned result remain distinct.
Agreement, discrepancy, inconclusiveness and inability to attempt or interpret are not conflated.
A disputed requirement receives a protected-function and burden comparison while current authority remains intact.
The receiving answer retains its material qualification and the relevant exact return.

RMP.6:8 - Common Anti-Patterns and How to Avoid Them

Anti-pattern	Why it fails	Practical correction
“The rerun validates the model”	The computation can agree while the intended-use relation remains unsupported.	Select the examination of that relation under supplied criteria.
“The replication was not significant, so the effect is absent”	The label omits magnitude, uncertainty and the chosen comparison.	Interpret the actual result under its design and criterion.
“Many copies corroborate the source”	Repeated carriers can preserve one dependent account.	Examine origin, dependence and the distinct evidential contribution.
“No data access means the claim failed”	An unattemptable comparison supplies no contrary output.	Return the access limit and the exact examination still unavailable.
“Each test fits, so the programme fits”	Shared demands can exceed the total available arrangement.	Check admitted joint combinations and choose a feasible useful set.
“We selected the test, so credibility increased”	A prospective decision supplies no performed result.	Keep the claim’s current support unchanged until actual evidence warrants revision.
“It is required, therefore it protects”	Current force does not establish incremental protection.	Examine merits and return amendment to its responsible authority.

RMP.6:9 - Consequences

Credibility work becomes a contribution to a stated claim rather than a ritual sequence. Useful existing results can finish the question; newly obtainable comparisons can justify further work. Access failures, discrepancies and uncertainty remain interpretable instead of being converted into blanket verdicts.

The cost is judgement about the comparison, disciplinary criteria and full arrangement. Some tests remain unavailable, and some attractive claims remain unsupported. A qualified completion makes that boundary usable without removing an authority’s legitimate requirement.

RMP.6:10 - Rationale

A credibility test is informative through the relation between its possible result and the claim. That relation explains both why one well-targeted comparison can matter and why a battery of mismatched tests can add little.

Evidence adequacy and acquisition choice are different questions. Preserving both allows a researcher to finish the present allocation honestly while keeping uncertainty visible. Separating prospective decisions from actual results prevents an approved study plan from being mistaken for support.

Protective requirements also make distinct claims about value, causal or measurement effects, practical burden and authority. Examining one does not automatically establish or cancel the others.

RMP.6:11 - SoTA-Echoing

The working question is which examination can improve or revise confidence in a bounded research claim, under what criterion, and at what obtainable burden.

The National Academies’ 2019 synthesis supplies the useful distinction between computational reproduction and new-data replication. It remains an anchor rather than a universal ladder.

The 2026 SCORE accounts refine different questions in social and behavioural research. The reproducibility study distinguishes availability of usable data from computational reproduction. The replicability study makes differing success criteria consequential. The analytical-robustness study examines defensible analytical alternatives. RMP.6 retains those distinctions without importing the studies’ sampled fields, rates or tolerances as universal criteria.

Braun and Clarke’s qualitative guidance and 2024 RTARG account preserve methodological coherence for reflexive thematic analysis. The American Historical Association’s standards retain source integrity and criticism for historical claims. Neither implies that all credibility questions should use the same repetition or agreement procedure.

The 2021 MRC framework supports question-dependent perspectives, context and research’s resource consequences in complex-intervention research. RMP.6 carries that contrast into the local choice without imposing intervention phases on every domain.

These sources do not supply the actual data, domain threshold, empirical result or authority. Reopen the affected comparison when a stronger methodological account, changed source, criterion, obtainable arrangement or receiving use changes what the test could establish.

RMP.6:12 - Relations

RMP.5: supplies a bounded claim and criticism or receives a claim whose support changed; compatible direct inputs also allow entry.
RMP.2: selects a new or revised research design when the credibility question requires one. Its no-new-study answer does not imply a selected test or design.
RMP.3 and RMP.4: supply the consequential operational specification and actual trace needed for a selected examination; neither creates its result in advance.
RMP.1: qualifies a materially changed research question or supplies an exact ordinary-practice or source return.
FPF A.10 and A.10.1: govern claim-bound evidence use and revalidation of affected uses when a relied-on source changes. C.11.DUA and A.15.9: govern decision-useful evidence demands and obtaining a bounded result from another practice, including attainable contribution and whole burden. The research-specific comparison remains here.
Domain and receiving practices: supply Methods, criteria, protection and permissions, and retain the value, action and amendment decisions.
RMP.7 and RMP.8: an actual credibility result can supply compatible corpus appraisal or exact claim/model revision and evidence return. A selected test alone supplies neither result. An adequate credibility answer can still finish directly.