Credal Prediction based on Relative Likelihood

Timo Löhr, Paul Hofman, Felix Mohr, Eyke Hüllermeier

Keywords: Uncertainty Representation, Credal Set, Relative Likelihood

TL;DR: This paper proposes a principled method to construct credal sets using relative likelihood and ensemble learning, achieving improved uncertainty representation without sacrificing predictive accuracy.

Abstract:

Predictions in the form of sets of probability distributions, so-called credal sets, provide a suitable means to represent a learner's epistemic uncertainty. In this paper, we propose a theoretically grounded approach to credal prediction based on the statistical notion of relative likelihood: The target of prediction is the set of all (conditional) probability distributions produced by the collection of plausible models, namely those models whose relative likelihood exceeds a specified threshold. This threshold has an intuitive interpretation and allows for controlling the trade-off between correctness and precision of credal predictions. We tackle the problem of approximating credal sets defined in this way by means of suitably modified ensemble learning techniques.
To validate our approach, we illustrate its effectiveness by experiments on benchmark datasets demonstrating superior uncertainty representation without compromising predictive performance. We also compare our method against several state-of-the-art baselines in credal prediction.

Primary Area: Probabilistic methods (e.g., variational inference, causal inference, Gaussian processes)

Submission Number: 23162