When resampling/reweighting improves feature learning in imbalanced classification? A toy-model study
Abstract: A toy model of binary classification is studied with the aim of clarifying the class-wise resampling/reweighting effect on the feature learning performance under the presence of class imbalance. In the analysis, a high-dimensional limit of the input space is taken while keeping the ratio of the dataset size against the input dimension finite and the non-rigorous replica method from statistical mechanics is employed. The result shows that there exists a case in which the no resampling/reweighting situation gives the best feature learning performance irrespectively of the choice of losses or classifiers, supporting recent findings in~\citet{kang2019decoupling,cao2019learning}. It is also revealed that the key of the result is the symmetry of the loss and the problem setting. Inspired by this, we propose a further simplified model exhibiting the same property in the multiclass setting. These clarify when the class-wise resampling/reweighting becomes effective in imbalanced classification.
Submission Length: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=0mULkj0Lyp&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)
Changes Since Last Submission: The following modifications have been made to the manuscript for the camera-ready version.
- Date is added
- Openreview URL is added
- Deanonymized
- Acknowledgement section is added
- Colored texts are uncolored
Also, some very minor changes in expressions are conducted as follows:
- On page 2: for long decades -> over many decades
- On page 3: The sentence about Gaussian universality at the end of Section 2.1 has been slightly revised to make it more precise.
- On page 6: generated again from -> generated from
- On page 6: the effects of sampling on classification performance -> the effects of (under/over)sampling on classification performance
- On page 6: This gives an important contribution -> This is an important contribution
- On page 20: Inserted a phrase "but not its details,"
- On page 26: is used -> is used in the last equality
- On page 26: the $\Lambda^*$ dependence -> the $\Lambda^*$-dependence
- On page 27: Inserted a phrase "from Eq. (61)".
- On page 28: Inserted a phrase "which corresponds to Eq. (38a)." below Eqaution (68).
- Overall: eqs. -> Eqs.
Assigned Action Editor: ~Bruno_Loureiro1
Submission Number: 4023
Loading