OpenPatch: a 3D patchwork for Out-Of-Distribution detection

Paolo Rabino, Antonio Alliegro, Francesco Cappio Borlino, Tatiana Tommasi

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: 3D, point clouds, open set, semantic novelty detection, out of distribution, OOD

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose a novel method for 3D out of distribution detection that works, in realistic scenarios, out of the box without additional training

Abstract:

Moving deep learning models from the laboratory setting to the open world en-
tails preparing them to handle unforeseen conditions. In several applications the
occurrence of novel classes during deployment poses a significant threat, thus it is
crucial to effectively detect them. Ideally, this skill should be used when needed
without requiring any further computational training effort at every new task.
Out-of-distribution detection has attracted significant attention in the last years,
however the majority of the studies deal with 2D images ignoring the inherent 3D
nature of the real-world and often confusing between domain and semantic novelty.
In this work, we focus on the latter, considering the objects’ geometric structure
captured by 3D point clouds regardless of the specific domain.
We advance the field by introducing OpenPatch that builds on a large pre-trained
model and simply extracts from its intermediate features a set of patch represen-
tations that describe each known class. For any new sample, we obtain a novelty
score by evaluating whether it can be recomposed mainly by patches of a single
known class or rather via the contribution of multiple classes. We present an
extensive experimental evaluation of our approach for the task of semantic novelty
detection on real-world point cloud samples when the reference known data are
synthetic. We demonstrate that OpenPatch excels in both the full and few-shot
known sample scenarios, showcasing its robustness across varying pre-training
objectives and network backbones. The inherent training-free nature of our method
allows for its immediate application to a wide array of real-world tasks, offering a
compelling advantage over approaches that need expensive retraining efforts.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5546