On Validity of Extreme Value Theory-Based Parametric Models for Out-of-Distribution Detection

Tomasz Walkowiak, Kamil Szyc, Henryk Maciejewski

Abstract: Open-set classifiers need to be able to recognize inputs that are unlike the training or known data. As this problem, known as out-of-distribution (OoD) detection, is non-trivial, a number of methods to do this have been proposed. These methods are mostly heuristic, with no clear consensus in the literature as to which should be used in specific OoD detection tasks. In this work, we focus on a recently proposed, yet popular, Extreme Value Machine (EVM) algorithm. The method is unique as it uses parametric models of class inclusion, justified by the Extreme Value Theory, and as such is deemed superior to heuristic methods. However, we demonstrated a number of open-set text and image recognition tasks, in which the EVM was outperformed by simple heuristics. We explain this by showing that the parametric (Weibull) model in EVM is not appropriate in many real datasets, which is due to unsatisfied assumptions of the Extreme Value Theorem. Hence we argue that the EVM should be considered another heuristic method.

External IDs:dblp:conf/iccS/WalkowiakSM21