Towards Privacy-Guaranteed Label Unlearning in Vertical Federated Learning: Few-Shot Forgetting Without Disclosure

ICLR 2026 Conference Submission15004 Authors

Keywords: Federated Learning, Machine Unlearning, Privacy-Preserving

TL;DR: We propose the first method for label unlearning in Vertical Federated Learning (VFL), addressing privacy risks with limited labeled data using manifold mixup and gradient-based forgetting, followed by recovery optimization.

Abstract: This paper addresses the critical challenge of unlearning in Vertical Federated Learning (VFL), a setting that has received far less attention than its horizontal counterpart. Specifically, we propose the first method tailored to *label unlearning* in VFL, where labels play a dual role as both essential inputs and sensitive information. To this end, we repurpose manifold mixup traditionally used as an augmentation technique into a privacy-preserving transformation that disguises label information in the shared embeddings. These augmented embeddings are then subjected to gradient-based label forgetting, effectively removing the associated label information from the model. To recover performance on the retained data, we introduce a recovery-phase optimization step that refines the remaining embeddings. This design achieves effective label unlearning while preserving privacy and maintaining computational efficiency. We validate our method through extensive experiments on diverse datasets, including MNIST, CIFAR-10, CIFAR-100, ModelNet, Brain Tumor MRI, COVID-19 Radiography, and Yahoo Answers demonstrate strong efficacy and scalability. Overall, this work establishes a new direction for unlearning in VFL, showing that re-imagining mixup as a privacy mechanism can unlock practical, privacy-preserving, and utility-preserving unlearning. Our code will be released publicly.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 15004