Re-evaluating metamorphic testing of chess engines: A replication study

Axel Martin, Djamel Eddine Khelladi, Théo Matricon, Mathieu Acher

Abstract: Highlights•We replicated the original study varying depth and dataset (real games).•Metamorphic relations (MRs) are less effective on real positions and higher depths.•MRs must consider depth; effectiveness drops past threshold.•Stockfish code review explains discrepancies in transformed and low-depth positions.•Replication matters: MRs must be causally related to domain’s specificities.