From Aggregation to Guidance: Strategies for Personalized Federated Fine-Tuning of Foundation Models

Mikołaj Piórczyński, Wojciech Łapacz, Xinyu Li, Chang Liu, Abby Turner, Artur Dubrawski

Track: Extended Abstract Track

Keywords: Model Merging, Foundation Models, LLMs, Fine-tuning, Personalized Federated Learning

TL;DR: We study how server-side model merging and client-side learning strategies can improve the effectiveness of personalized federated fine-tuning of foundation models.

Abstract:

Federated learning (FL) enables collaborative fine-tuning of pretrained foundation models in privacy-sensitive settings without directly sharing raw data. Personalized federated learning (PFL) further addresses client-side heterogeneity by learning models tailored to each client's local tasks while still benefiting from cross-client collaboration. In this work, we study strategies to improve the effectiveness of personalized federated fine-tuning of Large Language Models (LLMs) using LoRA. On the server side, we explore advanced aggregation strategies that go beyond simple parameter averaging, drawing on model merging techniques to construct a robust global model.
On the client side, we investigate different ways to leverage the global model to guide local learning, including hard initialization, parameter regularization, and function-space guidance via knowledge distillation, and propose a similarity-based adjustment strategy to further improve local learning. Empirical results on the Super NaturalInstructions dataset demonstrate that careful design of both server- and client-side strategies has the potential to improve PFL performance, providing insights for developing more effective PFL learning frameworks for fine-tuning LLMs for heterogeneous client tasks.

Submission Number: 105