A Simple Approach to the Noisy Label Problem Through the Gambler's Loss

Liu Ziyin, Ru Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda

TL;DR: We propose to a simple loss function and an analytical early stopping criterion to deal with the label noise problem.

Abstract: Learning in the presence of label noise is a challenging yet important task. It is crucial to design models that are robust to noisy labels. In this paper, we discover that a new class of loss functions called the gambler's loss provides strong robustness to label noise across various levels of corruption. Training with this modified loss function reduces memorization of data points with noisy labels and is a simple yet effective method to improve robustness and generalization. Moreover, using this loss function allows us to derive an analytical early stopping criterion that accurately estimates when memorization of noisy labels begins to occur. Our overall approach achieves strong results and outperforming existing baselines.

Code: https://github.com/codesubmiter/b3124134

Keywords: noisy labels, robust learning, early stopping, generalization