FedSFT: Resource-Constrained Federated Black-Box Adaptation of Large Language Models

Zhenxiao Zhang, Yu Zhang, Zhidong Gao, Yuanxiong Guo, Yanmin Gong

Keywords: Large language model, federated learning, fine-tuning, resource efficiency

Abstract: Federated fine-tuning enables privacy-preserving adaptation of large language models (LLMs) by allowing decentralized training without sharing raw data. However, its real-world deployment is often hindered by restricted access to model parameters and substantial computation, communication, and memory overhead. To address these challenges, we propose $\textbf{Fed}$erated $\textbf{S}$urrogate $\textbf{F}$ine-$\textbf{T}$uning (FedSFT), a novel framework for federated black-box fine-tuning of LLMs that requires access only to the token probabilities of output sequences and significantly reduces resource demands on clients. In each communication round of FedSFT, clients fine-tune a small model that serves as a surrogate for the large model hosted on the server. The server then leverages the logit offsets between the tuned and untuned small models to adjust the output of the untuned large model and distills the knowledge to update the small model for the next training round. Experimental results show that FedSFT significantly reduces client-side computation, communication, and memory overhead while maintaining competitive performance compared to direct federated fine-tuning of large models. FedSFT offers a promising solution for efficient and privacy-preserving black-box fine-tuning of large models on resource-constrained clients, broadening the accessibility and applicability of state-of-the-art LLMs.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 13984