Population-based Evaluation in Repeated Rock-Paper-Scissors as a Benchmark for Multiagent Reinforcement Learning

Marc Lanctot, John Schultz, Neil Burch, Max Olan Smith, Daniel Hennes, Thomas Anthony, Julien Perolat

Keywords: multiagent, reinforcement learning, benchmark, environment, evaluation, rock, paper, scissors, roshambo

TL;DR: A paper that introduces a new benchmark for multiagent RL based on Rock, Paper, Scissors and an evaluation suite based on performance against a population of bot strategies.

Abstract: Progress in fields of machine learning and adversarial planning has benefited significantly from benchmark domains, from checkers and the classic UCI data sets to Go and Diplomacy. In sequential decision-making, agent evaluation has largely been restricted to few interactions against experts, with the aim to reach some desired level of performance (e.g. beating a human professional player). We propose a benchmark for multiagent learning based on repeated play of the simple game Rock, Paper, Scissors along with a population of forty-three tournament entries, some of which are intentionally sub-optimal. We describe metrics to measure the quality of agents based both on average returns and exploitability. We then show that several RL, online learning, and language model approaches can learn good counter strategies and generalize well, but ultimately lose to the top-performing bots, creating an opportunity for research in multiagent learning.

Type Of Paper: Abstract of recently published journal papers (max page 2)

Anonymous Submission: Anonymized submission.

Submission Number: 5