Exploring Category-Shared and Category-Specific Features for Fine-Grained Image Classification

Haoyu Wang, Dongliang Chang, Weidong Liu, Bo Xiao, Zhanyu Ma, Jun Guo, Yaning Chang

Abstract: The attention mechanism is one of the most vital branches to solve fine-grained image classification (FGIC) tasks, while most existing attention-based methods only focus on inter-class variance and barely model the intra-class similarity. They perform the classification tasks by enhancing inter-class variance, which narrows down the intra-class similarity indirectly. In this paper, we intend to utilize the intra-class similarity as assistance to improve the classification performance of the obtained attention feature maps. To obtain and utilize the intra-class information, a novel attention mechanism, named category-shared and category-specific feature extraction module (CSS-FEM) is proposed in this paper. CSS-FEM firstly extracts the category-shared features based on the intra-class semantic relationship, then focuses on the discriminative parts. CSS-FEM is assembled by two parts: 1) The category-shared feature extraction module extracts category-shared features that contain high intra-class semantic similarity, to reduce the large intra-class variances. 2) The category-specific feature extraction module performs spatial-attention mechanism in category-shared features to find the discriminative information as category-specific features to decrease the high inter-class similarity. Compared with the state-of-the-art methods, the experimental results on three commonly used FGIC datasets show that the effectiveness and competitiveness of the proposed CSS-FEM. Ablation experiments and visualizations are also provided for further demonstrations.