Query-Guided Prototype Generation for Few-Class Classification

yoav shalev, Bashar Elnashef, Lior Wolf

Keywords: few-class classification, prototype learning, query-guided retrieval, transformer-based classification, frozen encoder, few-shot learning, contextual prototypes, ELMES embeddings, support set selection, Few-Class Arena

TL;DR: Boosting few‑class classification with architecture‑agnostic module for prototype generation, via query-guided support retrieval and fusion.

Abstract: Recent studies in few-class regime show that the performance of standard image classifiers, especially those trained on many classes, can be significantly degraded when applied to tasks with only a few target categories. In this setting, larger backbones do not necessarily yield better results, and traditional scaling laws often break down, leading to high variance and unpredictable behavior. To address these challenges, we propose a simple yet effective classification module, including prototype generation via query-guided support retrieval and fusion, which can be attached to any frozen image encoder. For each query, a small class-wise support set is retrieved from the training data based on feature similarity to the query. Each retrieved support set is then fused with the query using a transformer module to produce contextual prototypes, which are subsequently processed by a second transformer-based classifier, in which the query attends to the contextual prototypes to produce the final prediction. This approach addresses run-time and memory constraints by restricting attention to a compact set of query-specific prototypes, rather than processing full support sets jointly. It requires no fine-tuning or retraining of the backbone encoder and is compatible with a wide range of architectures. Evaluated across diverse datasets and models from the Few-Class Arena benchmark, our method consistently improves performance over strong baselines and outperforms recent meta-learning methods tailored to this setting. By transforming frozen encoders into query-guided prototype matchers, our approach provides a practical, scalable, and state-of-the-art solution for few-class classification.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 11399