Semantic-Aware Feature Aggregation for Few-Shot Image Classification

Fusheng Hao, Fuxiang Wu, Fengxiang He, Qieshi Zhang, Chengqun Song, Jun Cheng

Abstract: Generating features from the most relevant image regions has shown great potential in solving the challenging few-shot image classification problem. Most of existing methods aggregate image regions weighted with attention maps to obtain category-specific features. Instead of using attention maps to indicate the relevance of image regions, we directly model the interdependencies between prototype features and image regions, resulting in a novel Semantic-Aware Feature Aggregation (SAFA) framework that can place more weights on category-relevant image regions. Specifically, we first design a “reduce and expand” block to extract category-relevant prototype features for each image. Then, we introduce an additive attention mechanism to highlight category-relevant image regions while suppressing the others. Finally, the weighted image regions are aggregated and used for classification. Extensive experiments show that our SAFA places more weights on category-relevant image regions and achieves state-of-the-art performance.