Calibration-Guided Quantile Regression

ICLR 2026 Conference Submission20525 Authors

Keywords: Deep learning, quantile regression, frequentist statistics, calibration, pareto front

TL;DR: Calibration-guided quantile regression generates the best approximation of the calibration-sharpness Pareto front.

Abstract: Obtaining finite-sample guarantees for predictive models is crucial for settings where decisions have to be made under uncertainty. This has motivated works where models are trained, then recalibrated to yield coverage guarantees. However, doing so often significantly increases model entropy, i.e., it becomes less sharp, making the model less useful. To mitigate this, recent works have increasingly attempted to achieve a balance between good calibration and sharpness. However, these methods often involve deriving completely new, poorly understood loss functions or employing a complex and computationally intensive training pipeline. Moreover, the trade-off between sharpness and calibration is frequently unclear for these methods. In this work, we argue for making the trade-off explicit by choosing the sharpest model subject to some pre-set miscalibration tolerance. To achieve this, we present a simple yet effective approach that combines two established metrics in a novel fashion: we minimize the pinball loss while controlling for calibration using a held-out dataset. Coincidentally, our method motivates a hitherto unexplored analysis: we explicitly compute the Pareto front achieved across methods in terms of sharpness and calibration, and compare performance against this Pareto front. Our approach consistently outperforms various state-of-the-art methods in terms of various Pareto front-related metrics, even though the competing methods are more complex and computationally expensive.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 20525