Controlling Confusion via Generalisation Bounds

Reuben Adams, John Shawe-Taylor, Benjamin Guedj

Keywords: PAC-Bayes, Generalisation bounds, Multiclass classification

Abstract: We establish new generalisation bounds for multiclass classification by abstracting to a more general setting of discretised error types. Extending the PAC-Bayes theory, we are hence able to provide fine-grained bounds on performance for multiclass classification, as well as applications to other learning problems including discretisation of regression losses. Tractable training objectives are derived from the bounds. The bounds are uniform over all weightings of the discretised error types and thus can be used to bound weightings not foreseen at training, including the full confusion matrix in the multiclass classification case.

TL;DR: A new type of generalisation bound providing more informative measures of performance.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2202.05560/code)