FIN.18:4.5 - Evaluate the action implied by the prediction

Run the proposed forecast through the actual funding or protection rule, including lead time, capacity and cost. A lower error is not enough if both forecasts trigger action after the bank’s deadline. A signal that correctly predicts a shortage still needs an obtainable amount of finance and a repayment path. Conversely, a conservative signal can avoid a shortage while creating frequent unnecessary draws, collateral calls or idle cash.

Separate a missed adverse event from a false signal, and identify their consequences. The cost of missing payroll can differ greatly from the cost of reserving an unused facility. Those costs and governing constraints determine the useful trade-off; a general accuracy percentage cannot supply it. Where consequences cannot responsibly be reduced to money, keep the relevant failure criterion alongside financial costs.

Use the same starting position and attainable action set in each comparison. Include the consequences of the chosen intervention in later cash. If historical actual cash already contains emergency borrowing triggered by the old forecast, comparing it directly with an unacted-on new forecast can misidentify both the forecast error and avoided loss. Recover the underlying cash before that intervention, or explicitly qualify what can be learned from the available data.

Test whether a simpler change to the decision rule closes the problem. A different trigger, a prepared response to a named large receipt or a more appropriate reserve can sometimes improve action without a new estimator. Any changed reserve or authority still requires its actual decision. Compare the cost of that alternative rather than treating every missed shortage as evidence for a more complex model.