Learning Unknowns from Unknowns: Diversified Negative Prototypes Generator for Few-shot Open-Set Recognition

Zhenyu Zhang, Guangyao Chen, Yixiong Zou, Yuhua Li, Ruixuan Li

Abstract: Few-shot open-set recognition (FSOR) is a challenging task that requires a model to recognize known classes and identify unknown classes with limited labeled data. Existing approaches, particularly Negative-Prototype-Based methods, generate negative prototypes based solely on known class data. However, as the unknown space is infinite while the known-space is limited, these methods suffer from limited representation capability. To address this limitation, we propose a novel approach, termed Diversified Negative Prototypes Generator (DNPG), which adopts the principle of "learning unknowns from unknowns." Our method leverages the unknown space information learned from base classes to generate more representative negative prototypes for novel classes. During the pre-training phase, we learn the unknown space representation of the base classes. This representation, along with inter-class relationships, is then utilized in the meta-learning process to construct negative prototypes for novel classes. To prevent prototype collapse and ensure adaptability to varying data compositions, we introduce the Swap Alignment (SA) module. Our DNPG model, by learning from the unknown space, generates negative prototypes that cover a broader unknown space, thereby achieving state-of-the-art performance on three standard FSOR datasets. We provide the source code in the supplementary materials for reproducibility.

Primary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: This paper makes a contribution to the field of multimedia/multimodal processing by addressing the challenging task of Few-Shot Open-Set Recognition (FSOR). FSOR is a critical task in multimedia scenarios where new content frequently emerges, requiring the model to recognize known classes and identify unknown classes with limited labeled data. The proposed Diversified Negative Prototypes Generator (DNPG) model innovatively adopts the principle of "learning unknowns from unknowns," utilizing the inverse representation of base classes to generate more representative negative prototypes for novel classes. This approach enables the model to cover a broader unknown space, thereby enhancing its capability to handle diverse multimedia data. The introduction of the Swap Alignment (SA) module further ensures the diversity and adaptability of the negative prototypes to variations in data composition. By achieving state-of-the-art performance on standard FSOR datasets, this work significantly advances the field of multimedia/multimodal processing, particularly in improving the robustness and adaptability of models to new and unseen content.

Submission Number: 1366