Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play

ACL ARR 2026 January Submission3649 Authors

Keywords: Reasoning Transfer, Self-Play Reinforcement Learning, Game-Based Learning, Large Language Models, Mathematical Reasoning

Abstract: Games offer a compelling paradigm for developing general reasoning capabilities in language models, as they naturally demand strategic planning, probabilistic inference, and adaptive decision-making. However, existing self-play approaches rely solely on terminal game outcomes, providing no mechanism to distinguish transferable reasoning patterns from game-specific heuristics. We present STRATAGEM, which addresses two fundamental barriers to reasoning transfer: domain specificity, where learned patterns remain anchored in game semantics, and contextual stasis, where static game contexts fail to cultivate progressive reasoning. STRATAGEM selectively reinforces trajectories exhibiting abstract, domain-agnostic reasoning through a Reasoning Transferability Coefficient, while incentivizing adaptive reasoning development via a Reasoning Evolution Reward. Experiments across mathematical reasoning, general reasoning, and code generation benchmarks demonstrate substantial improvements, with particularly strong gains on competition-level mathematics where multi-step reasoning is critical. Ablation studies and human evaluation confirm that both components contribute to transferable reasoning.

Paper Type: Long

Research Area: Generalizability and Transfer

Research Area Keywords: Language Modeling

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 3649