Loc-FACMAC: Locality Based Factorized Multi-Agent Actor- Critic Algorithm for Cooperative Tasks

TMLR Paper1893 Authors

Abstract:

In this work, we present a novel cooperative multi-agent reinforcement learning method
called Locality based Factorized Multi-Agent Actor-Critic (Loc-FACMAC). Existing stateof-the-art algorithms, such as FACMAC, rely on global reward information, which may
not accurately reflect individual agents’ actions’ influences in decentralized systems. We
integrate the concept of locality into critic learning, where strongly related agents form
partitions during training. Agents within the same partition have a greater impact on each
other, leading to more precise policy evaluation. Additionally, we construct a dependency
graph to capture the relationships between agents, facilitating the partitioning process. This
approach mitigates the curse of dimensionality and prevents agents from using irrelevant
information. Our method improves upon existing algorithms by focusing on local rewards
and leveraging partition-based learning to enhance training efficiency and performance. We
evaluate the performance of Loc-FACMAC in two environments: Multi-cartpole and BoundedCooperative-Navigation. We explore the impact of partition sizes on the performance and
compare the result with baseline MARL algorithms such as LOMAQ, FACMAC, and QMIX.
The experiments reveal that, if the locality structure is defined properly, Loc-FACMAC
outperforms these baseline algorithms up to 45% , indicating that exploiting the locality
structure in the actor-critic framework improves the MARL performance.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Marc_Lanctot1

Submission Number: 1893