Federated Training, Fine-Tuning, RAG, and Inference using Flower

Dimitris Stripelis

Keywords: federated learning, federated fine-tuning & benchmarking, federated retrieval-augmented generation (RAG), on-device inference

TL;DR: A hands-on session on federated fine-tuning, pre-training, and deployment of LLMs using Flower, with real-world benchmarks across diverse domains.

Abstract:

The rise of large language models (LLMs) has accelerated AI capabilities across domains, but it has also increased reliance on centralized infrastructures, massive public datasets, and workflows that often compromise user privacy. In this presentation, we introduce a comprehensive approach to decentralized LLM fine-tuning using federated learning (FL), answering retrieval-augmented generation queries in federated environments (FedRAG), and performing privacy-preserving inference, powered by the open-source Flower framework.

We begin by exploring federated fine-tuning of LLMs, demonstrating how models can be collaboratively adapted across distributed, private data silos without exposing raw data. Next, we present the FlowerTune LLM Leaderboard, a community-driven benchmarking suite that evaluates federated fine-tuning across four domains—NLP, finance, medicine, and coding—offering the first large-scale comparison of 26 pre-trained LLMs across aggregation strategies and resource constraints.

We then discuss federated RAG pipelines, enabling contextual generation in sensitive data environments, and conclude by covering on-device and remote inference, showcasing deployment techniques that preserve privacy through secure handoffs to the cloud, through Flower Intelligence.

Overall, this presentation aims to deliver hands-on guidance and deep technical insights for researchers, engineers, and developers seeking to deploy and tune LLMs in a secure, decentralized, and privacy-preserving manner.

Submission Number: 103