Multivariate Calibration is Performative: A Perspective on Pitfalls and Progress

Sofian Zalouk, Charles Marx, Syrine Belakaria, Christopher De Sa, Stefano Ermon

Keywords: calibration, time-series, multivariate, performative prediction, probabilistic forecasting, post-hoc recalibration, uncertainty quantification

TL;DR: We identify key pitfalls in achieving multivariate calibration and outline future directions

Abstract: Reliable structured prediction requires forecasts that are calibrated both marginally and jointly. We identify two fundamental challenges in post-hoc multivariate calibration: performativity, where recalibration alters the very statistics used to assess calibration, and proportionality, the need to preserve a model’s learned dependence structure when correcting miscalibration. We show that two trivial forecasters can satisfy any notion of multivariate calibration, motivating a search for minimal corrections. By framing recalibration as a constrained optimization problem, we outline principled directions for feedback-aware algorithms that deliver reliable multivariate forecasts.

Submission Number: 99