TruVRF: Toward Triple-Granularity Verification on Machine Unlearning

Chunyi Zhou, Yansong Gao, Anmin Fu, Kai Chen, Zhi Zhang, Minhui Xue, Zhiyang Dai, Shouling Ji, Yuqing Zhang

Abstract: The right to be forgotten has incentivized machine unlearning, but a key challenge persists: the lack of reliable methods to verify unlearning conducted by model providers. This gap facilitates dishonest model providers to deceive data contributors. Current approaches often rely on invasive methods like backdoor injection. However, it poses security concerns and is also inapplicable to legacy data—already released data. To tackle this challenge, this work initializes the first non-invasive unlearning verification framework which operates at triple-granularity (class-, volume-, sample-level) to assess the data facticity and volume integrity of machine unlearning. In this paper, we propose a framework, named TruVRF, encompasses three Unlearning-Metrics, each tailored to counter different types of dishonest model providers or servers (Neglecting Server, Lazy Server, Deceiving Server). TruVRF leverages non-invasive model sensitivity to enable multi-granularity verification of unlearning. Specifically, Unlearning-Metric-I checks if the removed class matches the data contributor’s unlearning request, Unlearning-Metric-II measures the amount of unlearned data, and Unlearning-Metric-III validates the correspondence of a specific unlearned sample with the requested deletion. We conducted extensive evaluations of TruVRF efficacy across three datasets, and notably, we also evaluated the effectiveness and computational overhead of TruVRF in real-world applications for the face recognition dataset. Our experimental results demonstrate that TruVRF achieves robust verification performance: Unlearning-Metric-I and -III achieve over 90% verification accuracy on average against dishonest servers, while Unlearning-Metric-II maintains an inference deviation within 4.8% to 8.2%. Additionally, TruVRF demonstrates generalizability across diverse conditions, including varying numbers of unlearned classes and sample volumes. Significantly, TruVRF is applied to two state-of-the-art unlearning frameworks: SISA (presented at Oakland’21) and Amnesiac Unlearning, representing exact and approximate unlearning methods, respectively, which affirm TruVRF’s practicality. In addition, we conducted extensive evaluations around TruVRF, including ablation experiments, trade-offs in computational overhead, and the robustness of model sensitivity, among others.