From continuous pre-training to alignment: A comprehensive toolkit for large language models in federated learning

Zhuo Zhang, Yukun Zhang, Guanzhong Chen, Lizhen Qu, Xun Zhou, Hui Wang, Zenglin Xu

Abstract: The rapid success of Large Language Models (LLMs) has unlocked vast potential for AI applications in privacy-sensitive domains. However, the traditional centralized training of LLMs poses significant challenges due to privacy concerns regarding collecting sensitive data from diverse sources. This paper offers a promising and privacy-enhancing solution for LLMs: collaboratively training LLMs via Federated Learning (FL) across multiple clients, eliminating the need for raw data transmission. To this end, we present F4LLM, a new and comprehensive toolbox that supports the entire Federated LLM pipeline, from Continuous pre-training to alignment and LLM evaluation. F4LLM employs gRPC as the communication protocol to support various widely-used FL algorithms, ensuring efficient development and benchmarking in geo-distributed FL environments. Moreover, F4LLM offers both open-form and closed-form evaluation options via the efficient inference tool vLLM. The source code and documentation are at here.