How Does Fine-Tuned Foundation Models Help for Long-Tailed Data

ICLR 2026 Conference Submission18349 Authors

Keywords: long-tail learning, class-imbalanced learning, fine-tune

TL;DR: We provide a scientific guideline for the accurate use of long-tail learning methods in fine-tuning foundation models.

Abstract:

Deep long-tail learning is a challenging visual recognition problem that trains models on long-tailed distributed datasets.
In the last decade, a large number of methods have been proposed to solve the problems caused by imbalanced data.
Many methods have been proven useful in learning a deep model from scratch, such as ResNet or ResNeXt, but they have not been validated as effective in fine-tuning the pre-trained foundation models, such as CLIP or ViT. If users inappropriately apply these long-tail learning methods, it may result in worse accuracy than expected.
However, there is no scientific guideline for these methods in the existing literature.
In this paper, we first collect the widely used methods of existing long-tail learning and then conduct extensive and systematic experiments to provide a guideline for the accurate use of these methods in fine-tuning foundation models.
Furthermore, we observe that the current comparison protocol ignores the influence of training cost and hyperparameter selection, which may potentially lead to unfair comparisons and biased results.
Motivated by our empirical studies, we propose a unified fine-tuning framework for long-tailed recognition.
Experimental results demonstrate that the proposed framework outperforms existing methods on multiple long-tailed datasets, including ImageNet-LT, Places-LT, CIFAR100-LT, and iNaturalist 2018.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 18349