ShiftedBronzes: Benchmarking and Analysis of Domain Fine-grained Out-of-Distribution Detection in Gradual Shifts

Rixin Zhou, Honglin Pang, Qian Zhang, Ruihua Qi, Chuntao Li, Xi Yang

Keywords: Fine-grained Multi-level OOD detection; Benchmarking and Analysis

TL;DR: Our work introduces ShiftedBronzes, the first benchmark for multi-level, fine-grained OOD detection, along with a novel distribution-aware degree metric (DAD) and comprehensive evaluation across various OOD methods.

Abstract: Out-of-distribution (OOD) detection with fine-grained classes remains a significant challenge due to subtle distribution shifts that can substantially degrade model performance. Existing benchmarks either (i) evaluate multiple, coarse-grained shift levels on general-purpose datasets (e.g., near-OOD \& far-OOD), or (ii) examine limited shift level in fine-grained datasets (e.g., open-set recognition on fine-grained dataset), but none comprehensive address multi-level fine-grained shift. To bridge this gap, we introduce ShiftedBronzes, a benchmark designed for fine-grained OOD detection that systematically covers multiple, nuanced shift levels. We evaluate representative post-hoc detectors and vision-language model (VLM)-based methods on ShiftedBronzes and five general OOD benchmarks, uncovering two key limitations: (i) most post-hoc detectors fail to distinguish between varying degrees of distributional shifts, and (ii) although VLMs with prompt tuning outperform most post-hoc methods, they suffer from overfitting when fine-tuned on data with a single background context. To address these challenges, we propose a distribution-aware sensitivity metric that quantifies model robustness across shift levels, and a background-bias mitigation strategy tailored to VLMs. Together, ShiftedBronzes and our sensitivity metric form a comprehensive framework for evaluating and advancing robust multi-level OOD detection in fine-grained domains.

Primary Area: datasets and benchmarks

Submission Number: 15380