The First International Maritime Capture the Flag Competition: Lessons Learned and Future Directions

Prithviraj Dasgupta, John Kliem, Zachary Serlin, Michael Novitzky, Jordan Beason, Tyler Errico, Michael Benjamin, Tyler Paine, Peter Crowley, Simon Mark Lucas, James Chao, Wiktor Mateusz Piotrowski, Gautham Dixit, Matthew Leprell, Jeffrey Richley, Christopher Tucker, Abdullah Alamar, Katsuki Ohto, John Meo

Keywords: adversarial game, multi-agent, competition, reinforcement learning, sim-to-real

TL;DR: Algorithms and lessons learned from a public challenge on learning to play a multi-player, adversarial game

Abstract: Maritime Capture the Flag (MCTF) is a 3-vs-3 multi-agent real-time strategy game that utilizes a marine robotics simulator with support for hardware deployment. The game presents several research challenges in the areas of coordination and communication of multi-agent teams in adversarial environments with sparse rewards, and safe autonomy. In this paper, we report our experiences and challenges in deploying the MCTF game as an open, public challenge as part of the competition track at the 2024 Autonomous Agents and Multi-agent Systems (AAMAS) conference. The top performing teams were also evaluated on unmanned surface vehicles playing a 3-vs-3 MCTF game in a physical marine environment. We summarize the techniques used by the top eight competition entries that featured control algorithms ranging from multi-agent deep reinforcement learning to heuristic approaches for path planning and search algorithms. Our analysis of the competition results reveals a trade off between winning versus safety, as a key factor differentiating teams' performance was their agents' handling of safety behaviors like collisions with other players. We conclude by highlighting the key research gaps in deploying multi-player game-like encounters in the real world scenarios.

Submission Number: 16