fpf.tools
Library / Knowledge-Corpus Access Engineering Principles Framework
Jump to passage
In this reading
Source · lines 1274–1288

Link to current text

Published source confirmed at last check

Source changed 2026-10-03 10:39:28 UTC · snapshot created 2026-10-03 10:40:04 UTC · last check 2026-10-03 11:10:20 UTC

KCAE.EVAL:4.4 - Isolate the failed contribution before repairing

Use controlled substitutions after the whole comparison identifies a consequential question:

  • Supply adequate source candidates manually to the same assessor and delivery path. If the result recovers, finding was a limiting factor; if not, downstream work remains.
  • Hold the candidate pool and source context fixed while comparing assessors. This separates candidate availability from ranking and condition judgement.
  • Hold a qualified contribution fixed while varying delivery. A failed recipient can then expose omitted context, excessive compression, unavailable tools or missing preparation.
  • Hold the source snapshot and queries fixed while varying representation or route. Inspect not only aggregate retrieval scores but unique useful discoveries and unique harmful omissions.
  • Replay a controlled source change through build, persistence, reload and use. Compare the affected result with unchanged controls.
  • Hold inquiry content fixed while changing the encounter occasion. Observe interruption and uptake, not merely whether a notification was sent.

These substitutions diagnose an operation under controlled inputs. They do not prove that the repaired whole will produce those inputs naturally. Re-run the relevant complete use after repair. Components can interact: a larger pool can improve recall and overwhelm an assessor, while a more selective delivery can save context and remove a decisive exception. Do not add separate component gains as if they were independent benefits.

For encounter recovery, interrupt the actual implementation after claiming work, during execution and after committing the result. Present a concurrent duplicate and a later completed duplicate. Observe whether unfinished work has an actual continuation, whether a stale owner can publish, and whether an external effect can be repeated or remains unknown. A constructed state trace can expose a missing branch; it does not establish a real database’s atomicity, scheduler behavior or recipient idempotency. Keep those implementation questions separate from a classifier’s inquiry quality.