CMP.7:4.5 - Establish the learning result at the strength available
Distinguish three results:
- Obtaining: the algorithm returns the stated rule or collection under its inputs and computational limits.
- Fit or update performance: the returned rule achieves the stated empirical criterion, or the online procedure has the stated behavior on feedback.
- Further-use performance: a bound, comparison, assumption or observation supports the response on the receiving cases.
Prove or check only the result needed for the use, with additional work selected through C.11.DUA. For example, a finite fixed H with a realizable target and independent examples from the receiving distribution permits a sample-based generalization argument. A finite online candidate family can instead support a mistake bound for an arbitrary sequence under a stable realizable target; independent sampling is then unnecessary for that bound.
In a statistical argument, retain the class, loss range, dependence and selection conditions used by the theorem. Repeatedly choosing rules against the same assessment examples changes what that assessment supports. In a changed environment, all historical observations need not have the same relevance to the future query.
When available examples leave candidates disagreeing at a consequential input, return that ambiguity or choose a rule under an explicit selection basis. Obtaining an additional response is one possible move, not a default obligation. Compare its likely effect with the cost of asking, delaying or acting under the remaining uncertainty.