Revive Re-weighting in Imbalanced Learning by Density Ratio Estimation

Jiaan Luo, Feng Hong, Jiangchao Yao, Bo Han, Ya Zhang, Yanfeng Wang

Keywords: machine learning, density ratio estimation, optimization

Abstract: In deep learning, model performance often deteriorates when trained on highly imbalanced datasets, especially when evaluation metrics require robust generalization across underrepresented classes. To address the challenges posed by imbalanced data distributions, this study introduces a novel method utilizing density ratio estimation for dynamic class weight adjustment, termed as Re-weighting with Density Ratio (RDR). Our method adaptively adjusts the importance of each class during training, mitigates overfitting on dominant classes and enhances model adaptability across diverse datasets. Extensive experiments conducted on various large scale benchmark datasets validate the effectiveness of our method. Results demonstrate substantial improvements in generalization capabilities, particularly under severely imbalanced conditions.

Primary Area: Optimization (convex and non-convex, discrete, stochastic, robust)

Submission Number: 4464