Towards Aligned Data Forgetting via Twin Machine Unlearning
Keywords: machine unlearning
Abstract: Modern privacy regulations have spurred the evolu-
tion of machine unlearning, a technique enabling a trained model
to efficiently forget specific training data. In prior unlearning
methods, the concept of “data forgetting” is often interpreted
and implemented as achieving zero classification accuracy on
such data. Nevertheless, the authentic aim of machine unlearn-
ing is to achieve alignment between the unlearned model and
the gold model (i.e., the model derived from re-training from
scratch without the data to be forgotten). Here, “alignment”
signifies the encouragement for both models to achieve identical
classification accuracy. Owing to its generalization ability, the
gold model can correctly classify a portion of the forgotten data,
resulting in a non-zero classification accuracy. To better align
the unlearned model with the gold model, we propose a Twin
Machine Unlearning (TMU) approach, where a twin unlearning
problem is defined corresponding to the original unlearning
problem. Consequently, the generalization-label predictor trained
on the twin problem can be transferred to the original problem,
facilitating aligned data forgetting. Additionally, we introduce a
noise-perturbed fine-tuning scheme to balance the trade-off be-
tween retaining the model’s generalization ability and enhancing
its resilience to Membership Inference Attacks. Comprehensive
empirical experiments illustrate that our approach significantly
enhances the alignment between the unlearned model and the
gold model. Meanwhile, our method allows data forgetting
without compromising the model’s accuracy.
Primary Area: other topics in machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5677
Loading