Uncertainty Quantification Revisited: Conditional Cross-Entropy for Neural Classifiers

Yuanzhao Niu, Runhua Wang, Ping Xu, Zhi Tian

Keywords: Model Reliability, Uncertainty Quantification, Cross-Entropy, Bayesian Decision

Abstract: We propose correctness-aligned uncertainty (CAU) scores, a new family of uncertainty quantification (UQ) measures that explicitly account for whether predictions are correct or incorrect. Unlike conventional UQ, which relies on a single averaged score that conflates confidence across outcomes, CAU separates uncertainty contributions by correctness, providing richer interpretability and actionable insights. Our approach leverages conditional cross-entropy (CCE) to quantify the dissimilarity between estimated and target uncertainty distributions, yielding two complementary CAU metrics for correct and incorrect decisions. This correctness alignment naturally bridges multi-class UQ with binary decision theory, enabling principled formulations grounded in Neyman–Pearson and Bayesian detection principles. Beyond interpretability, CAU scores serve as effective tools for model refinement, misclassification detection, and performance improvement. We further highlight challenges in neural classifiers, where class imbalance between correct and incorrect predictions causes conventional UQ-based formulations to collapse to accuracy-dominated measures, thereby exposing the limitations of standard UQ approaches. We empirically demonstrate that CAU maintains robust uncertainty assessment for models with varying accuracy levels, where existing metrics often fail.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 22958