Factor Decorrelation Enhanced Data Removal from Deep Predictive Models

Wenhao Yang, Lin Li, Xiaohui Tao, Kaize Shi

Keywords: Factor decorrelation, Data removal, Out-of-distribution performance, Deep predictive models, Optimization

TL;DR: We propose a data removal method using factor decorrelation and loss perturbation to boost model robustness and accuracy, achieving strong performance on five benchmarks even under significant distribution shifts.

Abstract: The imperative of user privacy protection and regulatory compliance necessitates sensitive data removal in model training, yet this process often induces distributional shifts that undermine model performance-particularly in out-of-distribution (OOD) scenarios. We propose a novel data removal approach that enhances deep predictive models through factor decorrelation and loss perturbation. Our approach introduces: (1) a discriminative-preserving factor decorrelation module employing dynamic adaptive weight adjustment and iterative representation updating to reduce feature redundancy and minimize inter-feature correlations. (2) a smoothed data removal mechanism with loss perturbation that creates information-theoretic safeguards against data leakage during removal operations. Extensive experiments on five benchmark datasets show that our approach outperforms other baselines and consistently achieves high predictive accuracy and robustness even under significant distribution shifts. The results highlight its superior efficiency and adaptability in both in-distribution and out-of-distribution scenarios.

Primary Area: Optimization (e.g., convex and non-convex, stochastic, robust)

Submission Number: 22039