FF-Erase : Machine Unlearning and Verification for Forward-Forward Models

WENSiyuan, Meng Zhang, Ningning Ding

Keywords: forward-forward, machine unlearning, machine unlearning verification, backpropagation-free

TL;DR: We propose the first machine unlearning algorithm for forward-forward models, and we also propose a MIA specific to FF models for accurate and practical unlearning verification.

Abstract: The Forward-Forward (FF) algorithms present promising and biologically plausible alternatives to backpropagation (BP), enabling efficient model training through layer-wise greedy optimization. However, the critical task of machine unlearning for FF models, which involves efficiently removing specific training data's influence without full retraining, remains a foundational yet unexplored problem. The inherent characteristics of FF models, such as their sensitivity to parameter tuning and layer-wise independent training, pose unique challenges, often causing catastrophic model collapse when applying conventional unlearning methods. To fill this gap, we introduce a novel unlearning framework specifically for FF models, which employs a goodness-guided strategy. This method proposes a stable guidance model to generate target goodness distributions, steering the original model to unlearn forgetting data by shifting its layer-wise goodness scores, thereby effectively adapting gradient-based unlearning for the FF architecture. To enable robust verification on unlearning performance, we also propose a novel goodness-based membership inference attack (G-MIA), a powerful and lightweight black-box attack that leverages the unique properties of FF models' goodness scores. Our experiments demonstrate that our proposed method effectively removes the influence of target forgetting data on FF models while preserving model utility on the remaining data. Critically, our approach accomplishes 1.9 to 3.1$\times$ faster than retraining from scratch, establishing an efficient foundation for FF unlearning.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 4821