Efficient Source-free Unlearning via Energy-Guided Data Synthesis and Discrimination-Aware Multitask Optimization

Xiuyuan Wang, Chaochao Chen, Weiming Liu, Xinting Liao, Fan Wang, Xiaolin Zheng

Abstract: With growing privacy concerns and the enforcement of data protection regulations, machine unlearning has emerged as a promising approach for removing the influence of forget data while maintaining model performance on retain data. However, most existing unlearning methods require access to the original training data, which is often impractical due to privacy policies, storage constraints, and other limitations. This gives rise to the challenging task of source-free unlearning, where unlearning must be accomplished without accessing the original training data. Few existing source-free unlearning methods rely on knowledge distillation and model retraining, which impose substantial computational costs. In this work, we propose the Data Synthesis-based Discrimination-Aware (DSDA) unlearning framework, which enables efficient source-free unlearning in two stages: (1) Accelerated Energy-Guided Data Synthesis (AEGDS), which employs Langevin dynamics to model the training data distribution while integrating Runge–Kutta methods and momentum to enhance efficiency. (2) Discrimination-Aware Multitask Optimization (DAMO), which refines the feature distribution of retain data and mitigates the gradient conflicts among multiple unlearning objectives. Extensive experiments on three benchmark datasets demonstrate that DSDA outperforms existing unlearning methods, validating its effectiveness and efficiency in source-free unlearning.

Lay Summary:

Growing privacy laws (like GDPR) let people demand their data be deleted from AI systems. To comply, researchers propose "machine unlearning", techniques to remove specific data from trained models. However, most existing unlearning methods need the original training data, which is often unavailable due to privacy rules or storage limits. Even the few workarounds are too computationally expensive for real-world use.
We propose DSDA, a new framework for efficient "source-free unlearning" (no original data needed). First, it synthesizes artificial data mimicking the original training distribution using accelerated energy-guided sampling. Second, it introduces a discrimination-aware optimizer that precisely removes the influence of the forget data while protecting retained knowledge and resolving conflicts between unlearning tasks.
We conduct extensive experiments on three benchmark datasets. Results demonstrate that DSDA outperforms existing unlearning methods, validating its effectiveness and efficiency in source-free unlearning.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Primary Area: Social Aspects->Privacy

Keywords: source-free unlearning, machine unlearning, unlearn

Submission Number: 5707