Learning to Re-weight Examples with Optimal Transport for Imbalanced Classification

Dan dan Guo, Zhuo Li, meixi zheng, He Zhao, Mingyuan Zhou, Hongyuan Zha

Keywords: imbalanced classification optimal transport reweighting

Abstract:

Imbalanced data pose challenges for deep learning based classification models. One of the most widely-used approaches for tackling imbalanced data is re-weighting, where training samples are associated with different weights in the loss function. Most of existing re-weighting approaches treat the example weights as the learnable parameter and optimize the weights on the meta set, entailing expensive bilevel optimization. In this paper, we propose a novel re-weighting method based on optimal transport (OT) from a distributional point of view. Specifically, we view the training set as an imbalanced distribution over its samples, which is transported by OT to a balanced distribution obtained from the meta set. The weights of the training samples are the probability mass of the imbalanced distribution and
learned by minimizing the OT distance between the two distributions. Compared with existing methods, our proposed one disengages the dependence of the weight learning on the concerned classifier at each iteration. Experiments on image, text and point cloud datasets demonstrate that our proposed re-weighting method has excellent performance, achieving state-of-the-art results in many cases and
providing a promising tool for addressing the imbalanced classification issue. The code has been made available at
https://github.com/DandanGuo1993/reweight-imbalance-classification-with-OT.

TL;DR: The weights of the training samples are the probability mass of the imbalanced distribution and learned by minimizing the OT distance between the two distributions.

Community Implementations: [ 4 code implementations](https://www.catalyzex.com/paper/learning-to-re-weight-examples-with-optimal/code)