On the Calibration of Isotonic Distributional Regression

Tobias Biegert, Johannes Resin, Alexander I. Jordan, Sebastian Lerch

Keywords: calibration, probabilistic forecasting, conditional distribution estimation

TL;DR: We identify and explain a Thomae's function–like spiking artifact in IDR PIT values and discuss mitigation via smoothing.

Abstract: Isotonic distributional regression (IDR) has recently been proposed as a non-parametric technique for probabilistic forecasting via the estimation of conditional distributions under order restrictions, based on continuous outcomes from deterministic model outputs. IDR has been widely used and multiple studies report promising performance and well-calibrated forecasts across application domains. We document a peculiar out-of-sample artifact of IDR, where probability integral transform (PIT) values are rational fractions with empirical frequencies resembling Thomae's function, an effect which thus far seems to have been overlooked. This phenomenon is demonstrated empirically using meteorological data and a simulation study. We further provide a theoretical explanation linking the spiking behavior to the discrete structure of the isotonic fits, and discuss how this artifact can be mitigated via smoothing approaches.

Submission Number: 18