Conformal Prediction for Deep Classifier via Truncating
Keywords: Conformal Prediction, Uncertainty Quantification
Abstract: Conformal Prediction is a distribution-free statistical framework that outputs a set of possible labels to capture the predictive uncertainty. In this work, we show that existing conformal prediction methods may generate inefficient sets arising from the inclusion of redundant labels. To mitigate this issue, we propose a novel conformal prediction algorithm, $\textit{Post-Calibration Truncated Conformal Prediction}$ (PoT-CP), which limits the size of the prediction sets generated by existing conformal prediction methods through a maximum rank cutoff. Specifically, PoT-CP determines this cutoff by minimizing a truncation rank that preserves the marginal coverage of the calibration dataset. The key idea is to eliminate classes with high predictive uncertainty in the prediction sets, allowing PoT-CP to further shorten the prediction sets. Theoretically, we provide the asymptotic validity of marginal coverage for PoT-CP and demonstrate the asymptotic conditional coverage equivalence between PoT-CP and the standard conformal prediction algorithm. Extensive experiments demonstrate that PoT-CP can effectively reduce prediction set sizes while maintaining the stable conditional coverage of various conformal prediction algorithms across different classification tasks.
Supplementary Material: zip
Primary Area: interpretability and explainable AI
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1876
Loading