Low Compute Unlearning via Sparse Representations

TMLR Paper4828 Authors

Abstract:

Machine unlearning, which involves erasing knowledge about a \emph{forget set} from a trained model, can prove to be
costly and infeasible using existing techniques. We propose a low-compute unlearning technique based on a discrete representational bottleneck. We show that the proposed technique efficiently unlearns the forget set and incurs negligible damage to the model's performance on the rest of the dataset. We evaluate the proposed technique on the problem of class unlearning using four datasets: CIFAR-10, CIFAR-100, LACUNA-100 and ImageNet-1k. We compare the proposed technique to SCRUB, a state-of-the-art approach which uses knowledge distillation for unlearning. Across all four datasets, the proposed technique performs as well as, if not better than SCRUB while incurring almost no computational cost.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Aaron_Klein1

Submission Number: 4828