PULSE: Projection-based Unlearning via Linear Speedy Entropy Maximization

ICLR 2026 Conference Submission13132 Authors

Keywords: approximate unlearning, projection methods, computational efficiency

TL;DR: We introduce PULSE, which achieves efficient machine unlearning by manipulating representation space, eliminating the need for retain data and dramatically reducing computational costs

Abstract: Machine unlearning enables selective erasure of knowledge associated with specific data points from trained models without retraining from scratch. However, existing unlearning approaches face significant limitations: they typically degrade performance on remaining data, require access to the original retain dataset to maintain the model utility, and incur high computational costs. To address these challenges, we propose PULSE (Projection-based Unlearning via Linear Speedy Entropy Maximization), a novel retain-data-free unlearning method. PULSE jointly learns a projection matrix alongside the model backbone during training. During the unlearning phase, PULSE freezes the model backbone and trains a forget-specific projection matrix that maximizes confidence on the data to be forgotten. By subtracting this forget-specific matrix from the original projection, PULSE transforms confident predictions into targeted uncertainty, effectively achieving forgetting. Unlike existing methods that modify model outputs to enforce forgetting, PULSE operates directly on representation space. Extensive experiments on standard benchmarks show that PULSE achieves near-perfect forget accuracy while preserving retain accuracy and runs 10–20× faster while being memory efficient thus, establishing a new paradigm for efficient machine unlearning.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 13132