Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs

ICLR 2025 Workshop BuildingTrust Submission34 Authors

Track: Long Paper Track (up to 9 pages)

Keywords: Machine Learning, Federated Learning, Large Language Models, Privacy, Memorization

TL;DR: LoRA fine-tuning significantly reduces memorization in federated learning, mitigating privacy risks from adversarial prompting while maintaining model performance.

Abstract:

Federated learning (FL) is a popular paradigm for collaborative training which avoids direct data exposure between clients. However, data privacy issues still remain: FL-trained large language models are capable of memorizing and completing phrases and sentences contained in training data when given with their prefixes. Thus, it is possible for adversarial and honest-but-curious clients to recover training data of other participants simply through targeted prompting. In this work, we demonstrate that a popular and simple fine-tuning strategy, low-rank adaptation (LoRA), reduces memorization during FL up to a factor of 10. We study this effect by performing a medical question-answering fine-tuning task and injecting multiple replicas of out-of-distribution sensitive sequences drawn from an external clinical dataset. We observe a reduction in memorization for a wide variety of Llama 2 and 3 models, and find that LoRA can reduce memorization in centralized learning as well. Furthermore, we show that LoRA can be combined with other privacy-preserving techniques such as gradient clipping and Gaussian noising, secure aggregation, and Goldfish loss to further improve record-level privacy while maintaining performance.

Submission Number: 34