R2PS: Worst-Case Robust Real-Time Pursuit Strategies under Partial Observability

ICLR 2026 Conference Submission16663 Authors

Keywords: pursuit-evasion game, partial observability, dynamic programming, belief preservation, reinforcement learning, real-time pursuit strategy, worst-case robustness

Abstract: Computing worst-case robust strategies in pursuit-evasion games (PEGs) is time-consuming, especially when real-world factors like partial observability are considered. While important for general security purposes, real-time applicable pursuit strategies for graph-based PEGs are currently missing when the pursuers only have imperfect information about the evader's position. Although state-of-the-art reinforcement learning (RL) methods like Equilibrium Policy Generalization (EPG) and Grasper provide guidelines for learning graph neural network (GNN) policies robust to different game dynamics, they are restricted to the scenario of perfect information and do not take into account the possible case where the evader can predict the pursuers' actions. This paper introduces the first approach to worst-case robust real-time pursuit strategies (R2PS) under partial observability. We first prove that a traditional dynamic programming (DP) algorithm for solving Markov PEGs maintains optimality under the asynchronous moves by the evader. Then, we extend the DP pursuit strategies to a partially observable setting through preserving beliefs about the evader's possible positions. Finally, we embed the belief preservation mechanism into the state-of-the-art EPG framework to finish our R2PS learning scheme, which leads to a real-time pursuer policy through cross-graph reinforcement learning against the asynchronous-move DP evasion strategies. Experiments under different training sets reveal that using the extended DP pursuers as guidance improves the training efficiency under partial observability. After reinforcement learning, our policy achieves robust zero-shot generalization to unseen real-world graph structures and consistently outperforms the policy directly trained on the test graphs by the existing game RL approach, even against a best-responding evader.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 16663