C.16:8.3.2 - Interpreting an assessment of independent performance

The sought quantity p is the fraction of a population able to perform a specified action independently under stated conditions. An applicable assessment calibration supplies a, the probability of a positive test when the capability is present, and b, the probability when it is absent. Partitioning the population by that capability gives the expected positive fraction q=ap+b(1-p).

With a=0.9 and b=0.1, the relation is q=0.1+0.8p. An observed positive fraction 0.7 gives the estimate p=0.75 under this model. Sampling uncertainty, uncertainty in the calibrated rates and their applicability determine how precisely that estimate can be used. When a=b, the expected positive fraction is independent of p, so this test supplies no such distinction.

Now allow hints during the assessment. The earlier a and b may no longer describe the procedure. With the changed rates unknown, the positive fraction alone no longer determines p. If the independent-performance claim is still needed, return to that performance condition or obtain a calibration applicable to the changed procedure. The model explains which inference is available; it uses the subject’s account of the capability and its assessment.