Label Noise-Robust Learning using a Confidence-Based Sieving Strategy
Abstract: In learning tasks with label noise, boosting model robustness against overfitting is a pivotal challenge because the model eventually memorizes labels including the noisy ones. Identifying the samples with corrupted labels and preventing the model from learning them is a promising approach to address this challenge. Per-sample training loss is a previously studied metric that considers samples with small loss as clean samples on which the model should be trained. In this work, we first demonstrate that this small-loss trick is not efficient by itself. Then, we propose a novel discriminator metric called confidence error and a sieving strategy called CONFES to effectively differentiate between the clean and noisy samples. We experimentally illustrate the superior performance of our proposed approach compared to recent studies on various settings such as synthetic and real-world label noise. Moreover, we show CONFES can be combined with other approaches such as Co-teaching and DivideMix to further improve the model performance.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: We would like to thank all reviewers for their valuable feedback. We revised the manuscript accordingly and submitted the revised draft (additions are highlighted).
Here is the summary of the main modifications:
1. We explained the effective role of model confidence in label noise settings from previous studies, the reasons behind more efficient performance of the proposed confidence error metric (which is based on model confidence) and CONFES sieving technique (please see last paragraphs of section 3).
2. We added the results for three additional baseline methods: LRT(Reviewer jcnm), MentorMix (Reviewer S2G3), and PTD (Reviewer xj1Z).
3. We provided sensitivity analysis of the CONFES hyper-parameters (requested by Reviewer S2G3).
4. We added the results for different noise levels (requested by Reviewer S2G3).
5. We discussed the advantages of our confidence error metric over likelihood ratio (Reviewer jcnm) (please see the last paragraphs of section 3).
Assigned Action Editor: ~Matthew_Blaschko1
Submission Number: 679
Loading