Past to Present: Assessing Evaluation in Multi-agent Reinforcement Learning

Siddarth Singh, Omayma Mahjoub, Ruan John de Kock, Arnu Pretorius, Wiem Khlifi, Abidine Vall, Kale-ab Tessera

Keywords: Multi-agent reinforcement learning, MARL, Evaluation

TL;DR: We condust an analysis comparing historical and recent trends MARL evaluation methodology and provide insights in areas that need to be improved to maintain trust in MARL as a developing field.

Abstract: Establishing sound experimental standards and rigour is important in any growing field of research. Deep Multi-Agent Reinforcement Learning (MARL) is one such nascent field. Although exciting progress has been made, MARL has recently come under scrutiny for replicability issues and a lack of standardised evalution methodology, specifically in the cooperative setting. Although protocols have been proposed to help alleviate the issue, it remains important to actively monitor the health of the field. In this work, we extend the database of evaluation methodology previously published by \cite{gorsane2022standardised} containing meta-data on MARL publications from top-rated conferences and compare the findings extracted from this updated database to the trends identified in their work. Our analysis shows that many of the worrying trends in performance reporting remain. This includes the omission of uncertainty quantification, not reporting all relevant evaluation details and a narrowing of algorithmic development classes. Promisingly, we do observe a trend towards more difficult scenarios in SMAC-v1, which if continued into SMAC-v2 will encourage novel algorithmic development. Our data indicate that replicability needs to be approached more proactively by the MARL community to ensure trust in the field as we move towards exciting new frontiers.

Submission Category: Machine learning algorithms

Submission Number: 17