Go to ICLR 2024 Conference homepageContinual Learners are Viable Long-Tailed Recognizers
We propose a series of theorems which demonstrate that using Continual Learning (CL) to sequentially learn the majority and minority class subsets in a highly imbalanced dataset, is an effective solution for Long-Tailed Recognition (LTR). First, we theoretically prove that under the assumption of strong convexity of the loss function, the weights of a learner trained on a long-tailed dataset are bounded to reside within a neighborhood of the weights of the same learner trained strictly on the largest subset of that dataset. As a result, we present a novel perspective that CL methods, which are designed to optimize the weights in a way that the model performs well on multiple sets, are viable solutions for LTR. To validate our proposed perspective, we first verify the predicted upper bound of the neighborhood radius using the MNIST-LT toy dataset. Next, we evaluate the efficacy of several CL strategies on multiple standard LTR benchmarks (CIFAR100-LT, CIFAR10-LT, and ImageNet-LT), and show that standard CL methods achieve strong performance gains compared to baseline models and tailor-made approaches for LTR. Finally, we assess the applicability of CL techniques on real-world data by exploring CL on the naturally imbalanced Caltech256 dataset and demonstrate its superiority over state-of-the-art models. Our work not only unifies LTR and CL but also paves the way for leveraging advances in CL methods to tackle the LTR challenge more effectively.