On the Path Dependence of Gradient Ascent-Based Unlearning

Varun Sampath Kumar, Esmaeil S. Nadimi, Vinay Chakravarthi Gogineni

Abstract: Machine unlearning enables selective erasure of knowledge associated with specific data points from trained models. In this work, we show that gradient ascent-based machine unlearning is fundamentally path-dependent, i.e, under identical optimization budgets, different orderings of the same forget samples can induce qualitatively different forgetting vs retention trade-offs. Through experiments on CIFAR-10 and TinyImageNet, we demonstrate that structured sample orderings systematically steer unlearning toward distinct extremal regimes, whereas random ordering explores these outcomes only stochastically and with high variance. We further show that this behaviour is robust across multiple ordering proxies, including epistemic uncertainty, loss, and gradient norm, indicating that the ordering itself, rather than a specific heuristic are the primary driver of trajectory divergence. These findings reveal sample ordering as a previously overlooked but practically significant degree of freedom in machine unlearning and motivate trajectory-aware design and evaluation of optimization-based unlearning methods.

Submission Number: 15