ECO: Efficient Computational Optimization for Exact Machine Unlearning in Deep Neural Networks

Yu-Ting Huang, Pei-Yuan Wu, Chuan-Ju Wang

Keywords: Machine Unlearning, Parametric Programming, CP Algorithm, KKT Condition, Dual Support Vector Machine

Abstract: This paper introduces ECO, an efficient computational optimization framework that adapts the CP algorithm—originally proposed by Cauwenberghs & Poggio (2000)—for exact unlearning within deep neural network (DNN) models. ECO utilizes a single model architecture that integrates a DNN-based feature transformation function with the CP algorithm, facilitating precise data removal without necessitating full model retraining. We demonstrate that ECO not only boosts efficiency but also maintains the performance of the original base DNN model, and surprisingly, it even surpasses naive retraining in effectiveness. Crucially, we are the first to adapt the CP algorithm’s decremental learning for leave-one-out evaluation to achieve exact unlearning in DNN models by fully removing a specific data instance's influence. We plan to open-source our implementation to promote further research in this field.

Submission Number: 11