On the Evaluation of Neural Selective Prediction Methods for Natural Language Processing

Published: 01 Jan 2023, Last Modified: 20 May 2025ACL (1) 2023EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: We provide a survey and empirical comparison of the state-of-the-art in neural selective classification for NLP tasks. We also provide a methodological blueprint, including a novel metric called refinement that provides a calibrated evaluation of confidence functions for selective prediction. Finally, we supply documented, open-source code to support the future development of selective prediction techniques.
Loading