OODEEL: A Holistic Library for Unified Post-Hoc OOD Detection Research And Application

Paul Novello, Yannick Prudent, Corentin Friedrich, Joseba Dalmau

Keywords: Out-of-distribution detection, Benchmark, Open-source software

Abstract:

We present OODEEL, an open-source Post-hoc Out-of-Distribution (OOD) detection library. OODEEL is designed as a highly customizable tool that supports a wide range of OOD detectors and can be applied to any model classifier architectures from both PyTorch and TensorFlow. It implements unified abstractions so that every building block, such as activation shaping and layer-wise aggregation, can be used seamlessly by any detector. It also provides a user-friendly API that allows for easy integration of new OOD detectors, which can then benefit from all these building blocks, and native compatibility with most TensorFlow and PyTorch models.

OODEEL seamlessly handles standard OOD evaluation settings for benchmarking, including multiple ID/OOD datasets (both near- and far-OOD). Hence, we leverage its holistic implementation to address several critical aspects of OOD evaluation that are often overlooked in current benchmarks: robustness to model variability, effect of aggregation of layer-wise scores, effect of activation shaping, and link between in-distribution accuracy and OOD detection performances.

Primary Area: infrastructure, software libraries, hardware, systems, etc.

Submission Number: 11755