MMP.14:5.1 - A correct overall mean conceals failed conditional predictions

Two message routes, A and B, are used under ordinary load. The target is the chance of timely delivery for each known route. In this small example every message has a complete binary record, the deadline is unchanged, and outcomes are assumed independent with a stable probability within each route and load regime.

The supplied records are partitioned before fitting. Only the fitting and diagnostic portions are opened during model construction; the assessment portion remains withheld until the revised forecasts are fixed.

The original model (M_0) ignores route: (Y_i\sim\mathrm{Bernoulli}(p)). Its maximum-likelihood estimate from the fitting portion is (\hat p=1/2). Its expected diagnostic total equals the observed total: 10 timely deliveries out of 20. That agreement does not answer the route-specific question.

Choose (D=|K_A/10-K_B/10|), where (K_A,K_B) are the timely counts in the diagnostic portion. Observed (D=0.8). Under the common-probability model, condition on the observed total (K_A+K_B=10). Then [ P(K_A=k\mid K_A+K_B=10,M_0) =\frac{\binom{10}{k}\binom{10}{10-k}}{\binom{20}{10}}. ] This reference retains the two sample sizes and removes the unknown common (p). The exact two-sided tail for (D\ge0.8) is [ \frac{2(1+100)}{184756} =\frac{101}{92378}\approx0.001093. ] It exposes a discrepancy in the common-probability account under its independence and stability assumptions. It does not identify a causal route effect. A shared disturbance confounded with route could demand a different repair.

Suppose the subject account permits route-specific response probabilities. Construct (M_1): (Y_i\mid g_i\sim\mathrm{Bernoulli}(p_{g_i})). Using the same fitting records gives (\hat p_A=0.8,\hat p_B=0.2). The changed component is the relation between the known route and the response probability, not the binary recording rule.

Recalculate the original discrepancy under the fitted (M_1), retaining the same total of 10. Conditional replicate counts have weights [ w_k=\binom{10}{k}^{{2}16^k,\qquad P(K_A=k\mid K_A+K_B=10,\hat M_1)=w_k/\sum_{j=0}}{10}w_j, ] where (16=(0.8/0.2)/(0.2/0.8)) is the fitted odds ratio. Summing (k=0,1,9,10) gives (P(D\ge0.8)=0.37348). The revised point model accommodates the diagnostic contrast. This calculation is conditional on its fitted probabilities; it neither calibrates a test of the estimated family nor independently confirms the repair.

Fix these point-probability forecasts and open the assessment portion. The sum of log probabilities of its 20 individual outcomes, using natural logarithms, is [ L_0=20\log(0.5)=-13.86294,\qquad L_1=16\log(0.8)+4\log(0.2)=-10.00805. ] Thus (M_1)’s fixed forecasts gain (3.85490) on this portion. This is an observed paired comparison, not a guaranteed future gain or a parameter-uncertainty interval. The diagnostic portion was used to propose the repair; it was not counted as untouched assessment.

For three future independent A messages under the same regime, the point forecast of at least one late delivery changes from (1-0.5^3=0.875) to (1-0.8^3=0.488). That receiving calculation must change. If uncertainty about the probabilities matters, MMP.13 must propagate it; the point calculation does not already do so.

Changed condition. The supplied operating condition now specifies high load. Before any refitting, a supplied high-load batch has 5/10 timely outcomes on each route. The ordinary-load forecasts give [ L_1^{H}=10\log(0.8)+10\log(0.2)=-18.32581, ] whereas the common (0.5) forecast still gives (-13.86294). Carrying over the repaired forecast loses (4.46287) on this batch. The earlier assessment concerned ordinary load and does not establish transfer. Keep that use boundary and examine invariance if a high-load forecast is needed. A high-load common-rate fit of (0.5) is a possible new model; its fit here is not an untouched assessment. Its three-message late-delivery forecast would again be (0.875), conditional on that rate and independence.

For the narrower question of expected timely deliveries with equal numbers of A and B under ordinary load, both fitted models give one half of the total. If only that expectation is needed and its basis suffices, this case does not require adopting the richer model or obtaining new observations. It does not make the two models’ conditional or joint predictions equivalent.