Class-specific Feature Learning through Mitigating Spurious Correlation in Context-Poor Classes for Long-Tailed Classification

Ming Liu, Biao Hou, Bo REN, Hanyuan Ge

Keywords: Long-tailed Classification; Imbalanced Learning

Abstract: The inevitably long-tailed data in real-world presents a challenge for deep learning methods. Imbalanced data causes the context in each class to be unevenly distributed, prompting models to prioritize the accurate classification of context-rich classes while largely disregarding context-poor classes. Firstly, from both theoretical and practical perspectives, we explain why the model tends to favor the context-rich classes from both training and test sets, and breaking the spurious correlation between class-specific and context-poor features will further improve the effectiveness and robustness. Then, we propose a framework termed spurious correlation of context-poor, which aims to focus on class-specific features by progressively breaking the spurious correlation of limited data. Specifically, Grad-CAM is utilized to segment contextual regions and foregrounds within the samples coarsely. The high-confidence masks are retained and used to generate samples for the context-poor classes. Subsequently, to further mitigate spurious correlation, more reasonable class centers are incorporated into the contrastive loss to minimize the distance between semantically similar samples while maximizing the separation between samples from different classes in the feature space. To better preserve discriminative features, supervision is also performed on the generated samples. Finally, the experiments conducted on CIFAR10/100-LT, iNaturalist 2018, and ImageNet-LT demonstrate the effectiveness of our model. The code is available in the supplementary material.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 11564