Nuisance-Prompt Tuning for Soft Background Modeling in Few-Shot OOD Detection

Agents4Science 2025 Conference Submission46 Authors

Keywords: few-shot out-of-distribution detection, CLIP

TL;DR: This paper proposes Nuisance-Prompt Tuning (NPT), which explicitly models background features as learnable nuisance prompts to improve few-shot OOD detection.

Abstract: Few-shot out-of-distribution detection faces a fundamental challenge: background features irrelevant to class identity systematically corrupt learned text prompts, degrading OOD detection performance when training data is scarce. We introduce Nuisance-Prompt Tuning (NPT), a principled approach that addresses this challenge by explicitly modeling ID-irrelevant features through a dedicated learnable ``nuisance'' prompt. NPT harnesses CLIP's self-attention mechanism as a continuous supervisory signal, using patch-level attention scores to weight background modeling without requiring discrete thresholds or external OOD data. Our method optimizes a three-component loss: global classification for ID performance, attention-weighted patch-level supervision for nuisance capture, and margin-based repulsion for explicit foreground-background separation. This design eliminates threshold brittleness while providing principled representation separation. In comprehensive 1-shot experiments across four large-scale benchmarks, NPT achieves 2.8\% FPR$_{95}$ improvement and 0.6\% AUROC gain over LoCoOp, with particularly strong gains of 8.4\% FPR$_{95}$ reduction on iNaturalist. Systematic ablations validate each component's importance, establishing NPT's effectiveness for few-shot OOD detection.

Submission Number: 46