Efficient Privacy-Preserving Federated Learning With Selective Parameter Encryption
Keywords: Federated learning, privacy, homomorphic encryption, inversion attack
TL;DR: Efficient privacy-preserving federated learning based on selective homomorphic encryption with quantifiable privacy guarantee
Abstract: Federated learning trains machine learning models on distributed devices by aggregating local model updates instead of local data. However, privacy concerns arise as aggregating local model updates on the server may reveal sensitive personal information by inversion attacks. Privacy-preserving methods, such as homomorphic encryption (HE), then become necessary for FL training. Despite HE's privacy advantages, its applications suffer from impractical overheads, especially for foundation models. In this paper, we present the first practical privacy-preserving federated learning work with efficient HE-based secure model aggregation. Our approach proposes to selectively encrypt sensitive parameters, significantly reducing both computation and communication overheads during training while providing quantifiable privacy guarantee. Our optimization shows considerable overhead reduction, particularly for large foundation models (e.g.
100x reduction for GPT-2), demonstrating the potential for scalable HE-based FL deployment.
Supplementary Material: zip
Primary Area: alignment, fairness, safety, privacy, and societal considerations
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4889
Loading