Trusting the Untrustworthy: A Cautionary Tale on the Pitfalls of Training-based Rejection Option
Abstract: We consider the problem of selective classification, also known as rejection option. We first analyze state-of-the-art methods that involve a training phase to produce a selective classifier capable of determining when it should abstain from making a decision. Although only some of these frameworks require changes to the basic architecture of the classifier, by adding a module for selection, all methods necessitate implementing modifications to the standard training procedure and loss function for classification. Crucially, we observe two types of limitations affecting these methods: on the one side, these methods exhibit poor performance in terms of selective risk and coverage over some classes, which are not necessarily the hardest to classify; and surprisingly, on the other side, the classes for which they attain low performance vary with the model initialization. Additionally, some of these methods also decrease the accuracy of the final classification. We discuss the limitations of each framework, demonstrating that these shortcomings occur for a wide range of models and datasets. Moreover, we establish a formal connection between the trade-off of detecting misclassification errors and the minimization of risks in selective classification. This connection enables the development of a testing framework that requires no training and can be seamlessly applied to any pre-trained classifier, thereby enabling a rejection option.
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=DvUEstfH0c&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)
Changes Since Last Submission: # Summary of Changes for New Submission
We have made several significant modifications to the paper compared to the previous version (<https://openreview.net/notes/edits/attachment?id=Q8f8Izrcoq&name=pdf>) for this new submission, which are summarized as follows:
## New Proposition and Proof
- We have formalized our theoretical contribution by introducing a Proposition and including its proof in the appendix. The proposition establishes that the "optimal misclassification detector" achieves the most favorable trade-off ($P_I$, $P_{II}$) while simultaneously minimizing the selective classification risk. This formalizes a connection between the findings of Granese et al. (2021) and the rejection option problem. To the best of our knowledge, this formulation has not been previously addressed in the literature, making it a valuable theoretical contribution of our paper.
- We have updated Section 4, Section 8.1, and Section 8.2 to incorporate these new developments.
## New Experiments
- We have conducted additional experiments to compare our approach with ODIN [1] and MC Dropout [2]. The results of these experiments have been integrated into Tables 1, 3, and 4.
- Furthermore, we have included a new set of experiments that investigate the impact of calibration in rejection option methods. The corresponding results are presented in Table 2 of Section 6.3.
## Improved Clarity
- We have expanded our discussion on the relationships with previous work, particularly emphasizing the connections with Granese et al. (2021) and Chow (1970). The changes are mainly concentrated in the introductory paragraph of section 4.
- To enhance clarity, conciseness, and accuracy, we have meticulously reworded our contributions in the main contributions subsection of the introduction to align them more appropriately with the manuscript's contents.
- Furthermore, we have added additional details regarding Figure 4 and overall enhanced the quality of the writing throughout the paper.
References:
[1] Liang, S., et al. Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. ICLR 2018.
[2] Gal, Y., and Ghahramani, Z. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. ICML 2016.
Assigned Action Editor: ~Makoto_Yamada3
Submission Number: 1310
Loading