An Empirical Exploration of Open-Set Recognition via Lightweight Statistical Pipelines

Shu Kong, Deva Ramanan

Keywords: open-set recognition, anomaly detection, statistical models, Gaussian Mixture Models, open-world image classification, open-world semantic segmentation

Abstract: Machine-learned safety-critical systems need to be self-aware and reliably know their unknowns in the open-world. This is often explored through the lens of anomaly/outlier detection or out-of-distribution modeling. One popular formulation is that of open-set classification, where an image classifier trained for 1-of-$K$ classes should also recognize images belonging to a $(K+1)^{th}$ "other" class, not present in the training set. Recent work has shown that, somewhat surprisingly, most if not all existing open-world methods do not work well on high-dimensional open-world images (Shafaei et al. 2019). In this paper, we carry out an empirical exploration of open-set classification, and find that combining classic statistical methods with carefully computed features can dramatically outperform prior work. We extract features from off-the-shelf (OTS) state-of-the-art networks for the underlying $K$-way closed-world task. We leverage insights from the retrieval community for computing feature descriptors that are low-dimensional (via pooling and PCA) and normalized (via L2-normalization), enabling the modeling of training data densities via classic statistical tools such as kmeans and Gaussian Mixture Models (GMMs).

One-sentence Summary: The paper proposes an empirical pipeline for open-world recognition based on classic statistical models, which are built on properly processed deep off-the-shelf features and achieve state-of-the-art performance under various setups.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=43qhatRyw6