SelfClean: A Self-Supervised Data Cleaning Strategy

Fabian Gröger, Simone Lionetti, Philippe Gottfrois, Alvaro Gonzalez-Jimenez, Ludovic Amruthalingam, Matthew Groh, Alexander A. Navarini, Marc Pouly

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Data Cleaning, Data-centric AI, Self-Supervised Learning

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: A holistic self-supervised data cleaning strategy to detect irrelevant samples, near duplicates and label errors.

Abstract:

Most benchmark datasets for computer vision contain irrelevant images, near duplicates, and label errors.
Consequently, model performance on these benchmarks may not be an accurate estimate of generalization capabilities.
This is a particularly acute concern in computer vision for medicine where datasets are typically small, stakes are high, and annotation processes are expensive and error-prone.
In this paper we propose SelfClean, a general procedure to clean up image datasets exploiting a latent space learned with self-supervision.
By relying on self-supervised learning, our approach focuses on intrinsic properties of the data and avoids annotation biases.
We formulate dataset cleaning as either a set of ranking problems, which significantly reduce human annotation effort, or a set of scoring problems, which enable fully automated decisions based on score distributions.
We demonstrate that SelfClean achieves state-of-the-art performance in detecting irrelevant images, near duplicates, and label errors within popular computer vision benchmarks,
retrieving both injected synthetic noise and natural contamination.
In addition, we apply our method to multiple image datasets and confirm an improvement in evaluation reliability.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5292