A study of OOD detectors with Text Classifier

Sami Sabah

Abstract: The effectiveness of machine learning methods relies on the assumption that both the training and testing data conform to the same distribution $p_{xy}$. However, once a model is developed, it can be utilized with any data that meets the requisite format. In practice, models often fail when presented with data from a distribution that differs from the one on which they were trained. Even more concerning, models can produce incorrect predictions with high confidence, without notifying users of the error. This represents a significant safety issue, particularly in sensitive fields such as medicine or finance, prompting the machine learning community to recognize the need for methods to identify changes in data distribution. The active area of research that tackles this issue is known as Out-of-Distribution (OOD) detection. In this article, we address this problem following the approach of Colombo et al., 2022. Our study is divided into three parts: we first frame the problem, then benchmark OOD detectors, and finally, we investigate the effect of different aggregation methods by applying the same OOD detector with varying numbers of layers.