A Pitfall in Conformal Prediction: When Shorter Intervals Are Not Better

Yizhou Min, Lanqi Li, Yizhou Lu, Zhen Zhang, Jiaye Teng

Keywords: Conformal Prediction, Length Hacking

TL;DR: We find that the interval length might be improved through a invaild conformal prediction approach while introducing instability and unfairness

Abstract:

Conformal prediction has become a cornerstone of distribution-free uncertainty
quantification, conventionally evaluated by its coverage and interval length. This
work critically examines the sufficiency of these standard metrics. We demon-
strate that the interval length might be deceptively improved through a counter-
intuitive approach termed Prejudicial Trick (PT), while the coverage remains
valid. Specifically, for any given test sample, PT probabilistically returns an inter-
val, which is either null or constructed using an adjusted confidence level, thereby
preserving marginal coverage. While PT potentially yields a deceptively lower
interval length, it introduces practical vulnerabilities: the same input can yield
completely different prediction intervals across repeated runs of the algorithm.
We formally derive the conditions under which PT achieves these misleading improvements and provide extensive empirical evidence across various regression
and classification tasks. Furthermore, we introduce a new metric interval stability which helps detect whether a new conformal prediction method implicitly
improves the length based on such PT-like techniques.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 7272