PULSE: Projection-based Unlearning via Linear Speedy Entropy Maximization

TMLR Paper9342 Authors

Abstract: Machine unlearning enables selective erasure of knowledge associated with specific data points from trained models without retraining from scratch. However, existing retain-data-free methods typically degrade retain accuracy by 13--50\%, require access to retain data to preserve model utility, and incur high computational costs. Moreover, the majority of existing approximate unlearning methods are not designed for the black-box setting, where the unlearner has access only to the last few layers to classifier head and not to the feature extractor which is vendor locked. To address these limitations, we propose \textbf{PULSE} (Projection-based Unlearning via Linear Speedy Entropy Maximization), a retain-data-free unlearning method that performs \emph{knowledge localization} in representation space. PULSE introduces a learnable projection matrix that can be trained jointly with the model (fully retain-data-free during unlearning) or attached post hoc to any pretrained network (requiring only a small subset of training data for efficient initialization). During unlearning, a forget-specific projection is optimized to maximize confidence on the forget set via entropy minimization. Subtracting a scaled copy of this matrix from the original projection induces a targeted entropy increase on forget samples while preserving global model utility through controlled geometric transformations of localized feature subspaces. Extensive experiments on CIFAR-10, CIFAR-100, CIFARSuper20, and ImageNet-1k across MobileNetV2, ResNet18/50, and ViT-B/16 demonstrate that PULSE achieves competitive forgetting performance while preserving model utility. It runs faster than strong baselines thereby, establishing PULSE as a scalable and practical paradigm for efficient, localized machine unlearning in both joint-training and black-box post-hoc settings.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Pin-Yu_Chen1

Submission Number: 9342