Blind Unlearning: Unlearning Without a Forget Set

ICLR 2025 Conference Submission12339 Authors

Keywords: machine unlearning, data privacy

TL;DR: Our work tackles the question: is it possible to forget data, without knowing explicitly what that data is? and introduces a new unlearning method, Reload, which does not require explicit access to the forget dataset.

Abstract:

Machine unlearning is the study of methods to efficiently remove the influence
of some subset of the training data from the parameters of a previously-trained
model. Existing methods typically require direct access to the “forget set” – the
subset of training data to be forgotten by the model. This limitation impedes privacy, as organizations need to retain user data for the sake of unlearning when a
request for deletion is made, rather than being able to delete it immediately. We
first introduce the setting of blind unlearning – unlearning without explicit access
to the forget set. Then, we propose a method for approximate unlearning called
RELOAD, that leverages ideas from gradient-based unlearning and neural network
sparsity to achieve blind unlearning. The method serially applies an ascent step
with targeted parameter re-initialization and fine-tuning, and on empirical unlearning tasks, RELOAD often approximates the behaviour of a from-scratch retrained
model better than approaches that leverage the forget set. Finally, we extend the
blind unlearning setting to blind remedial learning, the task of efficiently updating
a previously-trained model to an amended dataset.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 12339