Decoupled and Patch-based Contrastive Learning for Long-tailed Visual Recognition

shiyu xuan, Shiliang Zhang

Keywords: long-tailed, self distillation

Abstract:

The imbalance of the dataset leads to the trained model being biased towards head classes and under-represent the tail classes, making the long-tailed recognition challenging.
To address those issues, this paper proposes the decoupled and patch-based contrastive learning. Given an anchor image, the supervised contrastive learning pulls two kinds of positives together in the embedding space: the same image with different data augmentation and other images from the same classes. The weights of two kinds of positives can be influenced by the cardinality of different classes, leading to biased feature space. The decoupled supervised contrastive loss decouples the two kinds of positives, removing the influence of the imbalanced dataset. To improve the discriminative of the learned model on the tail classes, patch-based self distillation crops the small patches from the global view of an image. These small patches can encode the shared visual patterns between different images, and thus can be used to transfer similarity relationship knowledge. Experiments on several long-tailed classification benchmarks demonstrate the superiority of our method. For instance, it achieves 57.7% top-1 accuracy on the ImageNet-LT dataset. Combined with the ensemble-based method, the performance can be further boosted to 59.7%. Our code will be released.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)