Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs

Thierry Bossy, Julien Tuấn Tú Vignoud, Tahseen Rabbani, Juan R. Troncoso Pastoriza, Martin Jaggi

Keywords: Machine Learning, Federated Learning, Large Language Models, Memorization, Privacy

TL;DR: LoRA fine-tuning significantly reduces memorization in federated learning, mitigating privacy risks from adversarial prompting while maintaining model performance.

Abstract: Federated learning (FL) is a popular paradigm for collaborative training which avoids direct data exposure between clients. However, data privacy issues still remain: FL-trained large language models are capable of memorizing and completing phrases and sentences contained in training data when given with their prefixes. Thus, it is possible for adversarial and honest-but-curious clients to recover training data of other participants simply through targeted prompting. In this work, we demonstrate that a popular and simple fine-tuning strategy, low-rank adaptation (LoRA), reduces memorization during FL by up to a factor of 10. We study this effect by performing fine-tuning tasks in high-risk domains such as medicine, law, and finance. We observe a reduction in memorization for a wide variety of Llama 2 and 3 models, and find that LoRA can reduce memorization in centralized learning as well. Furthermore, we show that LoRA can be combined with other privacy-preserving techniques such as gradient clipping and Gaussian noising, secure aggregation, and Goldfish loss to further improve record-level privacy while maintaining performance.

Submission Number: 43