AP-OOD: Attention Pooling for Out-of- Distribution Detection

ICLR 2026 Conference Submission18724 Authors

Keywords: out-of-distribution detection, attention pooling, nlp, language models

Abstract:

Out-of-distribution (OOD) detection, which maps high-dimensional data into
a scalar OOD score, is critical for the reliable deployment of machine learning
models. A key challenge in recent research is how to effectively leverage
and aggregate token embeddings from language models to obtain the OOD
score. In this work, we propose AP-OOD, a novel OOD detection method
for natural language that goes beyond simple average-based aggregation by
exploiting token-level information. AP-OOD is a semi-supervised approach
that flexibly interpolates between unsupervised and supervised settings,
enabling the use of limited auxiliary outlier data. Empirically, AP-OOD
sets a new state of the art in OOD detection for text: in the unsupervised
setting, it reduces the FPR95 (false positive rate at 95% true positives) from
27.77% to 5.91% on XSUM summarization, and from 75.19% to 68.13% on
WMT15 En–Fr translation.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 18724