Decoupled Feature Matching for Few-shot Counting and Localization

Chao Zhai, Fan Zhang, Wenyu Chen, Malu Zhang, Fan Li, Xuanting Xie

Abstract: Few-shot counting (FSC) aims to train a generalized visual counting model that can count any novel category given a small number of support samples. Current prevalent approaches treat FSC as a feature-matching task, leveraging attention to aggregate information from all other query patches or supports for each query patch. However, we notice that this operation blends target features with non-target features, making it difficult for the model to differentiate between targets and non-targets, thereby impacting counting accuracy. To tackle this issue, we develop a Decoupled Feature Matching Module (DFMM), which decouples target and non-target regions and conducts self-aggregation within respective regions. Furthermore, we design a Consistency Alignment Loss (CAL) to facilitate discriminative ability between targets and non-targets across multiple scales. Besides, we adopt a localization paradigm for counting and propose an Anchor-based Assignment Strategy to stabilize the optimization process and improve counting accuracy. Experiments on FSC147 and CARPK demonstrate that our method can achieve performance on par with state-of-the-art methods. Qualitative and quantitative experiments both confirm the efficacy of our proposed components.