Task-specific Meta-feature Selection for Few-shot Segmentation

Yuan Wang, Zhaoyang Li, Guoxin Xiong, Rui Sun, Tianzhu Zhang

Keywords: Computer Vision, Semantic Segmentation, Few-Shot Segmentation

TL;DR: We propose a task-specific gradient-based few-shot segmentation algorithm that can serve as a plug-and-play module embedded in multiple few-shot segmentation methods, consistently enhancing performance.

Abstract:

Few-shot segmentation (FSS) aims to segment new category images given only a few labeled samples. Most previous works concentrate on the design of intricate query decoders to perform feature matching or aggregation between the support and query. In this paper, we revisit a widely overlooked aspect of existing FSS methods, i.e., the exploration of pretrained backbone features. We find that treating all feature channels equally is suboptimal and propose a Task-specific Channel-wise Modulation Network (TCMNet) to focus more attention on task-aware channels, facilitating more effective utilization of pre-trained features. The proposed TCMNet enjoys several merits. First, we design a self-modulation block that injects the gradient information into channel-wise attention layers, thereby enhancing the discriminability between target and background features. Second, a cross-calibration block is introduced to align the support features toward the query according to the target gradient and representations, which mitigates the impact of intra-class diversity. Extensive experimental results on COCO-20i and Pascal-5i benchmarks demonstrate that the TCMNet, as a general plugin, consistently achieves significant improvements over different query decoders and also achieves state-of-the-art results. In addition, the decent performance achieved by exploring the backbone features may inspire another direction for developing more comprehensive FSS models.

Primary Area: transfer learning, meta learning, and lifelong learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5807