Leaving No OOD Instance Behind: Instance-Level OOD Fine-Tuning for Anomaly Segmentation

YUXUAN ZHANG, Zhenbo Shi, Han ye, Shuchang Wang, Zhidong Yu, Shaowei Wang, Wei Yang

Keywords: Computer Vision, Semantic Segmentation, Out-of-distribution Detection, Outlier Exposure

Abstract: Out-of-distribution (OOD) fine-tuning has emerged as a promising approach for anomaly segmentation. Current OOD fine-tuning strategies typically employ global-level objectives, aiming to guide segmentation models to accurately predict a large number of anomaly pixels. However, these strategies often perform poorly on small anomalies. To address this issue, we propose an instance-level OOD fine-tuning framework, dubbed LNOIB (Leaving No OOD Instance Behind). We start by theoretically analyzing why global-level objectives fail to segment small anomalies. Building on this analysis, we introduce a simple yet effective instance-level objective. Moreover, we propose a feature separation objective to explicitly constrain the representations of anomalies, which are prone to be smoothed by their in-distribution (ID) surroundings. LNOIB integrates these objectives to enhance the segmentation of small anomalies and serves as a paradigm adaptable to existing OOD fine-tuning strategies, without introducing additional inference cost. Experimental results show that integrating LNOIB into various OOD fine-tuning strategies yields significant improvements, particularly in component-level results, highlighting its strength in comprehensive anomaly segmentation.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 15418