Conformalized Credal Set Predictors
Keywords: Conformal Prediction, Credal Sets, Imprecise Probabilities, Uncertainty Representation, Uncertainty Quantification
TL;DR: A novel conformal prediction method is introduced to construct credal sets that are able to represent both the aleatoric and epistemic uncertainty in a prediction.
Abstract: Credal sets are sets of probability distributions that are considered as candidates for an imprecisely known ground-truth distribution. In machine learning, they have recently attracted attention as an appealing formalism for uncertainty representation, in particular, due to their ability to represent both the aleatoric and epistemic uncertainty in a prediction. However, the design of methods for learning credal set predictors remains a challenging problem. In this paper, we make use of conformal prediction for this purpose. More specifically, we propose a method for predicting credal sets in the classification task, given training data labeled by probability distributions. Since our method inherits the coverage guarantees of conformal prediction, our conformal credal sets are guaranteed to be valid with high probability (without any assumptions on model or distribution). We demonstrate the applicability of our method on ambiguous classification tasks for uncertainty quantification.
Primary Area: Probabilistic methods (for example: variational inference, Gaussian processes)
Submission Number: 20230
Loading