Making Reliable and Flexible Decisions in Long-tailed Classification
Abstract: Long-tailed classification is challenging due to its heavy imbalance in class probabilities. While existing methods often focus on overall accuracy or accuracy for tail classes, they overlook a critical aspect: certain types of errors can carry greater risks than others in real-world long-tailed problems. For example, misclassifying patients (a tail class) as healthy individuals (a head class) entails far more serious consequences than the reverse scenario. To address this critical issue, we introduce Making Reliable and Flexible Decisions in Long-tailed Classification (RF-DLC), a novel method aimed at ensuring reliable predictions in long-tailed problems. Leveraging Bayesian Decision Theory, we introduce an integrated gain to seamlessly combine long-tailed data distribution and the decision-making procedure. We further propose an efficient variational optimization strategy for the decision risk objective. Our method adapts readily to diverse utility matrices, which can be designed for specific tasks, ensuring its flexibility for different problem settings. In empirical evaluation, we design a new metric, False Head Rate, to quantify tail-sensitivity risk, and conduct comprehensive experiments on multiple real-world tasks, including classification, uncertainty estimations, and ablation studies, to demonstrate the reliability and flexibility of our method.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Revising the introduction based on reviewers' suggestions
Assigned Action Editor: ~Krishnamurthy_Dvijotham2
Submission Number: 1733
Loading