Prior Guided Feature Enrichment Network for Few-Shot Segmentation

Zhuotao Tian, Hengshuang Zhao, Michelle Shu, Ruiyu Li, Jiaya Jia

Abstract:

State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results and hard work on
unseen classes without fine-tuning. Few-shot segmentation is thus proposed to tackle this problem by learning a model that quickly
adapts to new classes with a few labeled support samples. These frameworks still face the challenge of generalization ability
reduction on unseen classes due to inappropriate use of high-level semantic information of training classes and spatial inconsistency
between query and support targets. To alleviate these issues, we propose the Prior Guided Feature Enrichment Network (PFENet). It
consists of novel designs of (1) a training-free prior mask generation method that not only retains generalization power but also
improves model performance and (2) Feature Enrichment Module (FEM) that overcomes spatial inconsistency by adaptively enriching
query features with support features and prior masks. Extensive experiments on PASCAL-5i and COCO prove that the proposed prior
generation method and FEM both improve the baseline method significantly. Our PFENet also outperforms state-of-the-art methods by
a large margin without efficiency loss. It is surprising that our model even generalizes to cases without labeled support samples. Our
code is available at https://github.com/Jia-Research-Lab/PFENet/.