Go to ICLR 2026 Conference homepageWeaving in the Clouds: Achieving Synergistic Collaboration among LLM Agents via Federated Learning
Keywords: Federated Learning, Multi-Agent Collaboration, LLMs, Privacy-Preserving Machine Learning
TL;DR: We introduce FedWave, a novel federated learning framework that enables LLM agents to effectively collaborate on workflow tasks without compromising data privacy.
Abstract: Multi-Agent Systems (MAS) powered by Large Language Models (LLMs) have shown immense potential in solving complex, sequential tasks by simulating expert collaboration. However, their reliance on centralized data clashes with real-world privacy constraints and data silos. Conversely, existing privacy-preserving paradigms like Federated Learning (FL) typically ignore the inherent sequential dependencies present in collaborative workflows, leading to suboptimal performance. To bridge this critical gap, we introduce FedWave, a novel framework for federated multi-agent collaboration. FedWave empowers LLM-based agents to collaboratively solve complex sequential tasks under strict privacy constraints by employing three core mechanisms: (1) a collaborative Value Chain Layer to model sequential dependencies, enabling efficient local fine-tuning through Federated Learning with LoRA adapters; (2) an intelligent Mixture of Experts (MoE) router at the server level for dynamic, task-aware aggregation of expert knowledge, moving beyond simple averaging; and (3) a final Direct Preference Optimization (DPO) stage to align the model's collaborative outputs with human preferences. Extensive experiments demonstrate that FedWave significantly outperforms both traditional federated learning and centralized multi-agent baselines, effectively achieving synergistic collaboration without compromising data privacy. The codes are available at https://anonymous.4open.science/r/FedWave-111A.
Primary Area: other topics in machine learning (i.e., none of the above)
Submission Number: 15412
Loading