Label-Wise uncertainty decomposition for Multi-label Classification by Maximizing Type II Likelihood

Minghao Li, Junjie Qiu, Weishi Shi

Keywords: uncertainty estimation, multilable classification, Bayesian inference, Evidential learning, OOD

Abstract:

Currently, the way deep learning models recognize uncertainty remains inconsistent with human perception.
In multi-label classification, quantifying uncertainty at the label level presents challenges, as each label may exhibit distinct model confidence levels. Understanding and decomposing label-specific uncertainty is essential for interpreting model behavior and ensuring reliable predictions.
We build a hierarchical Bayesian methodology for multi-label classification that leverages a Type II likelihood and Empirical Bayes. And then we estimate and decompose label-wise uncertainties by the law of the total variance. Our approaches offer four main contributions: (1) it is data centric, as Type II likelihood maximization ensures higher data likelihood; (2) it decomposes label-wise uncertainty into the model variance, the model bias and noise; (3) our uncertainty is intuitively interpretable when combined with observational data; and (4) when applied to out-of-distribution (OOD) detection task, it achieves a \(~6.88\%\) lower FPR95 score than the second-best method on NUS-WIDE.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 23287