Rethinking Priority Scheduling for Sequential Multi-Agent Decision Making in Stackelberg Games

Xiangyu Liu, Liang Zhang, Bo Jin, Ziqi Wei

Keywords: Stackelberg Game；Multi-agent system

Abstract: Current research applying N-level Stackelberg Game to multi-agent systems often use the default decision order of agents provided by the environment. However, this raises the question: Does the order of agents necessarily affect the ffnal equilibrium point of the game? To address this, we formally analyze the N-level Stackelberg Game in which changing the order where the agents make decisions typically leads to an overdetermined system. As a result, the equilibrium point is shifted unless special structural conditions are met. Based on this, we propose the Hierarchical Priority Adjustment(HPA) method, which adjust and select the agents’ decision order. For the upper level, an upper policy dynamically selects the optimal decision order of agents based on the current game state; for the lower level, agents execute the strategy in the Spatio-Temporal Sequential Markov Game(STMG) based on the selected order. To coordinate learning across time scales, we employ a slow-fast update scheme with shared intrinsic rewards derived from the upper policy advantage function. Experimental results on high-precision control tasks such as multi-agent MuJoCo show that HPA outperforms the benchmark algorithms and robustly adapts to changing environments. These results highlight the crucial role of optimizing the decision order of agents in N-level Stackelberg Game.

Area: Engineering and Analysis of Multiagent Systems (EMAS)

Generative A I: I acknowledge that I have read and will follow this policy.

Submission Number: 317