One-Shot Machine Unlearning with Mnemonic Code

Tomoya Yamashita, Masanori Yamada, Takashi Shibata

Keywords: Deep Learning, Machine Unlearning, Mnemonic Code

Verify Author List: I have double-checked the author list and understand that additions and removals will not be allowed after the submission deadline.

TL;DR: We propose a lightweight and fast Machine Unlearning method using Mnemonic Code

Abstract:

Ethical and privacy issues inherent in artificial intelligence (AI) applications have been a growing concern with the rapid spread of deep learning.
Machine unlearning (MU) is the research area that addresses these issues by making a trained AI model forget about undesirable training data.
Unfortunately, most existing MU methods incur significant time and computational costs for forgetting.
Therefore, it is often difficult to apply these methods to practical datasets and sophisticated architectures, e.g., ImageNet and Transformer.
To tackle this problem, we propose a lightweight and effective MU method.
Our method identifies the model parameters sensitive to the forgetting targets and adds perturbation to such model parameters.
We identify the sensitive parameters by calculating the Fisher Information Matrix (FIM).
This approach does not require time-consuming additional training for forgetting.
In addition, we introduce class-specific random signals called mnemonic code to reduce the cost of FIM calculation, which generally requires the entire training data and incurs significant computational costs.
In our method, we train the model with mnemonic code; when forgetting, we use a small number of mnemonic codes to calculate the FIM and get the effective perturbation for forgetting.
Comprehensive experiments demonstrate that our method is faster and better at forgetting than existing MU methods.
Furthermore, we show that our method can scale to more practical datasets and sophisticated architectures.

Url Link To Your Supplementary Code: https://github.com/tomyamkum/OneShotMU-with-MNCode

Primary Area: Trustworthy Machine Learning (accountability, explainability, transparency, causality, fairness, privacy, robustness, autoML, etc.)

Paper Checklist Guidelines: I certify that all co-authors of this work have read and commit to adhering to the guidelines in Call for Papers.

Student Author: No

Submission Number: 92