Graph-Localized Offline Federated Multi-Agent Reinforcement Learning for Wireless Networks

O. Tansel Baydas, Ayse Sila Okcu, Ozgur B. Akan

Keywords: offline reinforcement learning, multi-agent reinforcement learning, federated learning, wireless networks

TL;DR: We exploit the interaction graph in offline federated MARL to replace global coverage with local neighborhood coverage, and match a centralized raw-pooling oracle on multi-AP user association.

Abstract: Wireless networks motivate offline multi-agent reinforcement learning (MARL) as online exploration degrades service, and operator logs are rarely poolable across deployments. Existing offline MARL relies on global coverage that scales exponentially in the number of agents. We exploit the interaction graph induced by interference and contention to replace global coverage with $\kappa$-hop neighborhood coverage: a client contributes to agent~$i$'s local estimator only if it observes $\mathcal{N}_i^\kappa$. We prove a localized offline policy guarantee whose error decomposes into a locality bias decaying exponentially in $\kappa$, a near-neighbor shift bias, and a federated estimation term shrinking with the pooled observability-valid sample size. Our algorithm, F-GLOFF, matches a centralized raw-pooled oracle on multi-AP user association without sharing transitions and outperforms the engineered baselines by 12%.

Submission Number: 35