CHK.1:11 - SoTA-Echoing

How should questions be selected when expert attention, description-based extraction and generated refinement are all available? Adapt situated selection from R7/R10 together with the proposal-and-challenge line in AutoChecklist: use the least costly way to expose the consequential difference, then judge the proposed question against the work and its grounds. The selected answer is conditional. An explicit, usable description favors extraction; a familiar local omission can favor direct expert selection; uncertain coverage can justify generated contrasts and refinement.

These are serious alternatives with different costs. Extraction can preserve a description’s blind spot; direct expertise can miss an unfamiliar case; generation can add duplicates or plausible but unsupported criteria. SPECA supplies a concrete specification-derived audit alternative and counterevidence about mistaken scope assumptions. Its single case and correct-specification premise do not establish general coverage. AutoChecklist supplies evaluated generation/refinement methods in its stated tasks, not evidence that generation wins for this workshop or every domain. R7/R10 supply the practice connection to subjects and work; they do not compare model performance.

Section 4.2 therefore selects the proposal method by the missing distinction and includes adjudication effort. For the three explicit importer properties in §5, extraction plus a practitioner’s usefulness challenge supplies the stated questions without a generation loop. If a plausible imported result satisfies all three yet defeats the intended use, comparing that result can expose an additional question. Reject automatic iteration merely because more criteria can be produced. Accept the added examination cost when it can reveal such an unresolved consequential difference; retain direct selection when it already provides adequate coverage.

The sources do not rank these alternatives at equal effort across domains. Where that ranking changes a substantial investment, use a representative local comparison through ME.11 and ME.14. Reopen the local choice when a missed difference defeats coverage, the description’s usefulness is in doubt, or another proposal method produces needed distinctions at lower total effort.