Data Brittleness Estimation with Self-Supervised Features

Vasu Singla, Pedro Sandoval-Segura, Micah Goldblum, Jonas Geiping, Tom Goldstein

Keywords: data attribution, data brittleness estimation

Abstract: To what extent are model predictions sensitive to modifications in training data? Data attribution approaches have served to answer this question. These approaches can be used for estimating data brittleness i.e., identifying which subset of training samples had the highest positive influence on a test sample. However, these methods come at a high computational cost, are memory intensive, and are hard to scale to large models or datasets. Current state-of-the-art approaches require an ensemble of as many as \textbf{300,000 models}. In this work, we focus on a computationally efficient baseline centered on estimating two types of data brittleness metrics. Our baseline approach uses the image features from a \textbf{single} pretrained self-supervised backbone. In contrast to data attribution approaches, our method is model-agnostic based on the intuition that different models leverage data in similar ways. Our results show this simple assumption works well in practice, achieving competitive performance with state-of-the-art attribution approaches on CIFAR-10 and ImageNet, under limited computational and memory requirements. Our work serves as a simple baseline showing that effective data brittleness estimates can be achieved based solely using knowledge of the training data.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 12102