Optimal transport-based conformal prediction

Gauthier Thurin, Kimia Nadjahi, Claire Boyer

Abstract: Conformal Prediction (CP) is a principled framework for quantifying uncertainty in black-box learning models, by constructing prediction sets with finite-sample coverage guarantees. Traditional approaches rely on scalar nonconformity scores, which fail to fully exploit the geometric structure of multivariate outputs, such as in multi-output regression or multiclass classification. Recent methods addressing this limitation impose predefined convex shapes for the prediction sets, potentially misaligning with the intrinsic data geometry. We introduce a novel CP procedure handling multivariate score functions through the lens of optimal transport. Specifically, we leverage Monge-Kantorovich vector ranks and quantiles to construct prediction region with flexible, potentially non-convex shapes, better suited to the complex uncertainty patterns encountered in multivariate learning tasks. We prove that our approach ensures finite-sample, distribution-free coverage properties, similar to typical CP methods. We then adapt our method for multi-output regression and multiclass classification, and also propose simple adjustments to generate adaptive prediction regions with asymptotic conditional coverage guarantees. Finally, we evaluate our method on practical regression and classification problems, illustrating its advantages in terms of (conditional) coverage and efficiency.

Lay Summary: Most of machine learning models used on a daily basis are black boxes, meaning that it is difficult to control their uncertainty. Conformal prediction gathers techniques used for quantifying this uncertainty. However, the traditional paradigm is mostly used to target a single variable. We use recent statistical tools (optimal-transport based quantiles) to extend the usual methodology towards multiple variables.

Link To Code: https://github.com/gauthierthurin/OTCP

Primary Area: General Machine Learning

Keywords: conformal prediction, optimal transport, generalized quantiles, uncertainty quantification

Submission Number: 9985