S3: Supervised Self-supervised Learning under Label Noise

Chen Feng, Georgios Tzimiropoulos, Ioannis Patras

Keywords: Learning under label noise, Supervised learning, Self-supervised learning

Abstract: Despite the large progress in supervised learning with Neural Networks, there are significant challenges in obtaining high-quality, large-scale and accurately labeled datasets. In this context, in this paper we address the problem of classification in the presence of noisy labels and more specifically, both close-set and open-set label noise, that is when the true label of a sample may, or may not belong to the set of the given labels. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space, a relabeling mechanism that relies on the confidence of the classifier across subsequent iterations and a training strategy that trains the encoder both with a self-consistency loss and the classifier-encoder with cross-entropy loss on the selected samples alone. Without bells and whistles, such as co-training so as to reduce the self-confirmation bias, our method significantly surpasses previous methods on both CIFAR10/CIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.

One-sentence Summary: An effective and robust method SOTA performance for learning with complex noisy dataset

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2111.11288/code)