Constructing Informative Subtask Representations for Multi-Agent Coordination
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Multi-Agent Reinforcement Learning, Coordination, Subtask Representation, Vector Quantised Variational Autoencoder
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: The introduction of subtasks holds the promise of promoting coordination in scenarios without communication. Instead of manually defined subtasks, recent studies attempt to decompose the overall task and allocate subtasks to agents automatically, but it remains unclear how to acquire a set of proficient subtask representations. In essence, the subtasks serve as auxiliary signals that assist agents in deducing the broader context from limited observations. To embed maximal information into subtask representations, we propose to first learn a vector quantization variational autoencoder which takes individual observations of agents as inputs and reconstructs the global state based on their assigned subtasks as latent variables. Next, the informative representations can be readily integrated into various classic multi-agent reinforcement learning frameworks to facilitate insightful decisions of agents. Experiments on StarCraft II micro-war challenges and Google Research Football have demonstrated that our method learns reasonable and informative subtask representations, which facilitate the decision-making of agents and significantly improve the overall performance.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
Supplementary Material: zip
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2668
Loading