On Volume Minimization in Conformal Regression

Batiste Le bars, Pierre Humbert

TL;DR: We study the question of volume minimization in Conformal Regression, we present new algorithms and derive finite sample bounds on the excess volume loss.

Abstract:

We study the question of volume optimality in split conformal regression, a topic still poorly understood in comparison to coverage control. Using the fact that the calibration step can be seen as an empirical volume minimization problem, we first derive a finite-sample upper-bound on the excess volume loss of the interval returned by the classical split method. This important quantity measures the difference in length between the interval obtained with the split method and the shortest oracle prediction interval.
Then, we introduce *EffOrt*, a methodology that modifies the learning step so that the base prediction function is selected in order to minimize the length of the returned intervals.
In particular, our theoretical analysis of the excess volume loss of the prediction sets produced by *EffOrt* reveals the links between the learning and calibration steps, and notably the impact of the choice of the function class of the base predictor. We also introduce *Ad-EffOrt*, an extension of the previous method, which produces intervals whose size adapts to the value of the covariate. Finally, we evaluate the empirical performance and the robustness of our methodologies.

Lay Summary:

Machine learning models often make predictions without saying how confident they are. Conformal Prediction (CP) is a technique that fixes this by giving not just one answer, but a range (or interval) where the true answer is likely to fall, like saying “we think the house price will be between \$200K and 250K.”

Most existing work on conformal prediction focuses on making sure these ranges are reliable, that is, the true answer really is inside the range most of the time. But less is known about how to make these ranges as short and useful as possible, without losing that reliability.

In our work, we explore how to shrink these intervals. First, we analyze the most used CP method and show how close it gets to the shortest possible range. Then, we propose a new method called EffOrt that changes how the model is trained so that it naturally produces shorter, more informative intervals. We also introduce Ad-EffOrt, a version that makes the range bigger or smaller depending on how uncertain the model is — for example, giving narrower ranges for well-understood data and wider ones when it’s less sure.

We test these methods and show they perform well in practice, giving reliable predictions that are also more precise.

Link To Code: https://github.com/pierreHmbt/AdEffOrt

Primary Area: General Machine Learning->Everything Else

Keywords: Conformal Prediction; Minimum Volume Sets

Submission Number: 6560