L2B: Learning to Bootstrap Robust Models for Combating Label Noise

Yuyin Zhou, Xianhang Li, Fengze Liu, Qingyue Wei, Xuxi Chen, Lequan Yu, Cihang Xie, Matthew P. Lungren, Lei Xing

Keywords: noisy label, meta-learning, image classification

Abstract: Deep neural networks have shown great success in representation learning. However, when learning with noisy labels (LNL), they can easily overfit and fail to generalize to new data. To address this challenge, in this paper, we propose a novel machine learning method called Learning to Bootstrap (L2B) that leverages a joint reweighting mechanism to train models using their own predictions to bootstrap themselves without being adversely affected by erroneous pseudo-labels. Unlike conventional approaches, L2B dynamically adjusts the importance weight between real observed labels and pseudo-labels, as well as between different samples, to determine the appropriate weighting. Additionally, L2B conducts implicit relabeling concurrently, leading to significant improvements without incurring additional costs. L2B offers several benefits over the baseline methods. It yields more robust models that are less susceptible to the impact of noisy labels by guiding the bootstrapping procedure more effectively. It better exploits the valuable information contained in corrupted instances by adapting the weights of both instances and labels. Furthermore, L2B is compatible with existing noisy label learning methods and delivers competitive results on several benchmark datasets, including CIFAR-10, CIFAR-100, ISIC2019, and Clothing 1M datasets. Extensive experiments demonstrate that our method effectively mitigates the challenges of noisy labels, often necessitating few to no validation samples, and be well generalized to other tasks such as image segmentation. This not only positions it as a robust complement to existing LNL techniques but also underscores its practical applicability. The code and models are available at https://anonymous.4open.science/r/L2B-6006.

Primary Area: transfer learning, meta learning, and lifelong learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4871