UGradSL: Machine Unlearning Using Gradient-based Smoothed Label

Zonglin Di, Jinghan Jia, Jiancheng Liu, Zhaowei Zhu, Sijia Liu, Yang Liu

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Machine Unlearning, Negative Label Smoothing, Influence Function

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Machine Unlearning Using Gradient-based Smoothed Label

Abstract: The objective of machine unlearning (MU) is to eliminate previously learned data from a model. However, it is challenging to strike a balance between computation cost and performance when using existing MU techniques. Taking inspiration from the influence of label smoothing on model confidence, we consider MU as decreasing confidence in the forgotten data and increasing it in the remaining. This observation suggests a simple gradient-based MU approach that uses an inverse process of label smoothing. This work introduces UGradSL, a simple, plug-and-play MU approach that uses smoothed labels. We provide theoretical analyses demonstrating why properly introducing label smoothing improves MU performance. We conducted extensive experiments on six datasets of various sizes and different modalities, demonstrating the effectiveness and robustness of our proposed method. The consistent improvement in MU performance is only at a marginal cost of additional computations. For instance, UGradSL improves over the gradient ascent MU baseline by 66\% unlearning accuracy without sacrificing unlearning efficiency. This work also introduces a more practical MU paradigm, known as group-forgetting, which involves forgetting a subgroup of a superclass.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4130