Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

Hang Zhang, Kaifeng Zhang, Yixiao Ma, Ye Zhu, Kai Ming Ting

Keywords: machine unlearning, privacy protection, kernel mean embedding

Abstract:

Various machine unlearning techniques have been developed in response to privacy legislation requirements. These techniques enable individuals to exercise their legal right to have their data $D_f$ removed from a machine learning model. This process, commonly referred to as machine unlearning, is accomplished via the use of an unlearning function denoted as $U$.
Existing methods design an intricate $U$ to unlearn $D_f\subset D$ from a previous model $A(D)$ learned on $D$, so that the unlearned model performs as closely as possible to the retrained model $A(D\setminus D_f)$. However, these methods often take a long time due to the complex structures of $U$.

Inspired by Learning to Optimize, in this paper, we introduce the first learning-based model-agnostic approach named Learning-to-UnLearn (or L2UL) based on a distribution perspective, which acquires a simple $U$ via learning.
Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining, while also exhibiting impressive efficiency.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 7279