ChePAN: Constrained Black-Box Uncertainty Modelling with Quantile Regression

Axel Brando, Joan Gimeno, Jose Antonio Rodriguez-Serrano, Jordi Vitria

Keywords: Uncertainty modelling, Deep Learning, Black Box, Aleatoric, Quantile Regression, Chebyshev Polynomial, Neural networks

Abstract: Most predictive systems currently in use do not report any useful information for auditing their associated uncertainty and evaluating the corresponding risk. Taking it for granted that their replacement may not be advisable in the short term, in this paper we propose a novel approach to modelling confidence in such systems while preserving their predictions. The method is based on the Chebyshev Polynomial Approximation Network (the ChePAN), a new way of modelling aleatoric uncertainty in a regression scenario. In the case addressed here, uncertainty is modelled by building conditional quantiles on top of the original pointwise forecasting system considered as a black box, i.e. without making assumptions about its internal structure. Furthermore, the ChePAN allows users to consistently choose how to constrain any predicted quantile with respect to the original forecaster. Experiments show that the proposed method scales to large size data sets and transfers the advantages of quantile regression to estimating black-box uncertainty.

One-sentence Summary: ChePAN allows us to model the aleatoric uncertainty constrained to an existing pointwise preditictive system considered as a black box, i.e. without making assumptions about its internal structure.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=IP9e3bHQLb