Reliable AI through Out-of-Distribution Detection: A Benchmark Study

José Lucas De Melo Costa, Eliott Barbot

Abstract: This article delves into the issue of detecting out-of-distribution (OOD) examples in machine learning models, with a focus on natural language processing (NLP) applications. It is crucial to identify OOD data to build reliable AI, as our models are not trained to handle such examples and may perform poorly on them. To tackle this problem, we implement and compare various OOD detection methods from literature, and evaluate their effectiveness across different datasets, similarity scores between data points, expected distribution, and computational constraints. Furthermore, this article offers open-source code for the imple mentation of these methods.