Exploring Textual Out-Of-Distribution Detection: from simplistic supervised to advanced self-supervised techniques

Camille Langlois, Joé Vincent-Galtié

Abstract:

This paper delves into the issue of Textual Out-Of-Distribution (OOD) detection, which refers to the capability of machine learning models to recognize data samples that significantly deviate from their training data distribution. In Natural Language Processing (NLP) applications, Textual OOD detection is critical to ensuring the robustness and dependability of production systems. This study investigates the effectiveness of various methods for OOD detection in NLP, utilizing a transformer-based language model and different datasets with varying degrees of similarity to the training data. Our findings demonstrate that both the Mahalanobis-based score utilizing the last layer representation and the Cosine Projection score utilizing the average latent representation outperform the other scores in terms of AUROC. However, the supervised approach did not perform as well.
Code is available on github : https://github.com/joevincentgaltie/OOD_Detection_ENSAE.git