Certified Unlearning for Neural Networks

Anastasia Koloskova, Youssef Allouah, Animesh Jha, Rachid Guerraoui, Sanmi Koyejo

Abstract:

We address the problem of machine unlearning, where the goal is to remove the influence of specific training data from a model upon request, motivated by privacy concerns and regulatory requirements such as the “right to be forgotten.” Unfortunately, existing methods rely on restrictive assumptions or lack formal guarantees.
To this end, we propose a novel method for certified machine unlearning, leveraging the connection between unlearning and privacy amplification by stochastic post-processing. Our method uses noisy fine-tuning on the retain data, i.e., data that does not need to be removed, to ensure provable unlearning guarantees. This approach requires no assumptions about the underlying loss function, making it broadly applicable across diverse settings. We analyze the theoretical trade-offs in efficiency and accuracy and demonstrate empirically that our method not only achieves formal unlearning guarantees but also performs effectively in practice, outperforming existing baselines.

Lay Summary:

Imagine you've shared personal information with a company, only to later decide you want it deleted. While companies can remove your data from their records, the artificial intelligence (AI) models trained on that data might still retain its influence. This poses a challenge: how can we ensure that AI models "forget" specific data upon request?

Our research tackles this issue by introducing a method that allows AI models to unlearn particular pieces of information when asked. Unlike previous approaches that often come with limitations or lack formal assurances, our technique provides certified guarantees that the specified data no longer affects the model's behavior.

We achieve this by retraining the AI model using only the data meant to be retained, adding a layer of randomness to ensure the unwanted information is effectively removed. This approach doesn't rely on specific assumptions about the model's design, making it versatile across various applications.

Through theoretical analysis and practical experiments, we demonstrate that our method not only ensures the desired unlearning but also maintains the model's performance. This advancement is a step forward in respecting individual privacy rights in the age of AI.

Link To Code: https://github.com/stair-lab/certified-unlearning-neural-networks-icml-2025

Primary Area: Social Aspects->Privacy

Keywords: machine unlearning, privacy amplification by stochastic post processing, privacy amplification by iteration

Submission Number: 1539