CoDreamer: Communication-Based Decentralised World Models

Edan Toledo, Amanda Prorok

Keywords: MARL, model-based RL, Communication, GNN

Abstract: Sample efficiency is a critical challenge in Reinforcement Learning. Model-based RL has emerged as a solution, but its application has largely been confined to single-agent scenarios. In this work, we introduce CoDreamer, an extension of the Dreamer algorithm for multi-agent environments. CoDreamer leverages Graph Neural Networks for a two-level communication system to tackle challenges such as partial observability and inter-agent cooperation. Communication is separately utilized within the learned world models and within the learned policies of each agent to enhance modelling and task-solving. We show that CoDreamer offers greater expressive power than a naive application of Dreamer, and we demonstrate its superiority over baseline methods across various multi-agent environments.

Submission Number: 10