Minimum Invasive Machine Unlearning via Optimal Neural Weight Masking

Sujay Menon, Matthew Young, Mahtab Sarvmaili, Cheng Chang, Himanshu Rai, Maksims Volkovs, Ga Wu

Keywords: Machine Unlearning, Optimal Probabilistic Masking, Sparse Parameter Selection

Abstract: Existing Machine Unlearning (MU) approaches typically update all model parameters to achieve the goal of forgetting, which inevitably degrades model’s generalization capability. While recent research shows that unlearning can be achieved by manipulating a limited number of neural activations, these approaches are typically either agnostic to target data instances or computationally intensive due to the need to search for specific parameter subsets for each unlearning request. To mitigate these computational challenges, this paper presents Optimal Probabilistic Masking, a framework that localizes unlearning to a minimal subset of model parameters through a constrained KL divergence objective. Specifically, inspired by the probabilistic control theory, we introduce a probabilistic objective based on the KL divergence between a proposed masking distribution conditioned on the data and the optimal masking distribution, with a constraint on the pre-defined sparsity of the masking. The proposed masking distribution is represented as a parameterized neural network that, given a training sample and the model parameters, produces a unique binary mask highlighting the relevant parameters. Empirical results across multiple benchmarks demonstrate that our method efficiently identifies the model weights that are critical for target data prediction; manipulating the identified weights can effectively unlearn the target data while maintaining strong generalization performance, achieving a desirable balance between unlearning fidelity and model utility.

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: true

Submission Guidelines: true

Anonymous Url: true

No Acknowledgement Section: true

Submission Number: 9963