DivideMix: Learning with Noisy Labels as Semi-supervised LearningDownload PDF

Published: 20 Dec 2019, Last Modified: 22 Oct 2023ICLR 2020 Conference Blind SubmissionReaders: Everyone
Keywords: label noise, semi-supervised learning
TL;DR: We propose a novel semi-supervised learning approach with SOTA performance on combating learning with noisy labels.
Abstract: Deep neural networks are known to be annotation-hungry. Numerous efforts have been devoted to reducing the annotation cost when learning with deep networks. Two prominent directions include learning with noisy labels and semi-supervised learning by exploiting unlabeled data. In this work, we propose DivideMix, a novel framework for learning with noisy labels by leveraging semi-supervised learning techniques. In particular, DivideMix models the per-sample loss distribution with a mixture model to dynamically divide the training data into a labeled set with clean samples and an unlabeled set with noisy samples, and trains the model on both the labeled and unlabeled data in a semi-supervised manner. To avoid confirmation bias, we simultaneously train two diverged networks where each network uses the dataset division from the other network. During the semi-supervised training phase, we improve the MixMatch strategy by performing label co-refinement and label co-guessing on labeled and unlabeled samples, respectively. Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods. Code is available at https://github.com/LiJunnan1992/DivideMix .
Code: [![github](/images/github_icon.svg) LiJunnan1992/DivideMix](https://github.com/LiJunnan1992/DivideMix)
Data: [CIFAR-10](https://paperswithcode.com/dataset/cifar-10), [CIFAR-100](https://paperswithcode.com/dataset/cifar-100), [CIFAR-100N](https://paperswithcode.com/dataset/cifar-100n), [CIFAR-10N](https://paperswithcode.com/dataset/cifar-10n), [Clothing1M](https://paperswithcode.com/dataset/clothing1m), [WebVision](https://paperswithcode.com/dataset/webvision-database)
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 1 code implementation](https://www.catalyzex.com/paper/arxiv:2002.07394/code)
Original Pdf: pdf
14 Replies