Provably Robust Detection of Out-of-distribution Data (almost) for free

Alexander Meinke, Julian Bitterwolf, Matthias Hein

Keywords: out-of-distribution detection, adversarial noise, provable robustness, guarantees

Abstract: The application of machine learning in safety-critical systems requires a reliable assessment of uncertainy. However, deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. Even if trained to be non-confident on OOD data one can still adversarially manipulate OOD data so that the classifier again assigns high confidence to the manipulated samples. In this paper we propose a novel method that combines a certifiable OOD detector with a standard classifier from first principles into an OOD aware classifier. This way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in either prediction accuracy or detection performance for non-manipulated OOD data. Moreover, due to the particular construction our classifier provably avoids the asymptotic overconfidence problem of standard neural networks.

One-sentence Summary: We combine a classifier and a provably robust OOD detector in order to obtain provable robustness around OOD data and asymptotic guarantees without sacrificing performance.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2106.04260/code)