Discovery of Adversarial Endgame Chess Positions

Jonathan Rosenthal, Ananth Shreekumar, Ruoyu Song, Mohammad Areeb, Zengxiang Lei, Raymond Muller, Z. Berkay Celik

Keywords: Chess, Sampling, Adversarial Search and Robustness, Probabilistic Methods

TL;DR: We devise a way to find chess endgame positions, where modern chess engines make mistakes.

Abstract:

Chess engines have become an essential component of today's lucrative online chess market, and many players treat their recommendations as the ground truth.
However, these engines are not perfect and can make mistakes when faced with certain endgame positions. The occurrence of such positions within an engine's search could lead to errors cascading to the root. Despite this, the systematic generation and analysis of positions that expose such weaknesses remains an underexplored area of research.
To fill this gap, we develop AdvChess, a novel framework to automatically generate adversarial chess positions. These are positions where state-of-the-art engines deviate from theoretically optimal play.
Our approach focuses on identifying fair and legal positions where engine failures result in significant outcome changes, particularly in the context of endgame play, where ground-truth labels can be extracted from specialized endgame tablebases.
We design state and action encodings as well as a reward function for the foundation of the generative modeling problem.
We find that adversarial positions generated for Stockfish are least transferable across different computational settings
and that transferability does not correlate directly with engine strength.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 22752