Temperature Scaling for Quantile Calibration

Saiteja Utpala, Piyush Rai

Keywords: Calibration, Reliable Uncertainty Quantification, Probabilistic Deep Learning

TL;DR: We investigated the performance of Temperature Scaling for regression calibration in context of quantile calibration and found that it doesn't perform as well as it does for classification calibration.

Abstract: Deep learning models are often poorly calibrated, i.e., they may produce overconfident predictions that are wrong, implying that their uncertainty estimates are unreliable. While a number of approaches have been proposed recently to calibrate classification models, relatively little work exists on calibrating regression models. Temperature Scaling is one of the most popular methods for \emph{classification calibration}. It performs better than or equal to more sophisticated methods. We investigate the use of Temperature Scaling for \emph{regression calibration} under notion of quantile calibration.