Correlated Policy Optimization in Multi-Agent Subteams

ICLR 2026 Conference Submission15273 Authors

Keywords: multi-agent reinforcement learning, multi-agent coordination, Bayesian network, subteam

Abstract: In cooperative multi-agent reinforcement learning, agents often face scalability challenges due to the exponential growth of the joint action and observation spaces. Inspired by the structure of human teams, we explore subteam-based coordination, where agents are partitioned into fully correlated subgroups with limited inter-group interaction. We formalize this structure using Bayesian networks and propose a class of correlated joint policies induced by directed acyclic graphs . Theoretically, we prove that regularized policy gradient ascent converges to near-optimal policies under a decomposability condition of the environment. Empirically, we introduce a heuristic for dynamically constructing context-aware subteams with limited dependency budgets, and demonstrate that our method outperforms standard baselines across multiple benchmark environments.

Primary Area: reinforcement learning

Submission Number: 15273