Situational-Constrained Multi-Agent Coordination through Correlated Equilibria

Libo Zhang, Zhirui Zeng, Yang Chen, Jiamou Liu

Keywords: Situational Constraints, Multi-agent coordination, Correlated Equilibria

Abstract:

Multi-agent coordination requires strategies that align individual benefits with collective goals. While Correlated Equilibria (CE) offer a solution for balancing incentives under coordination, real-world systems often impose additional situational constraints, i.e., context-dependent requirements triggered only under specific conditions, such as fairness during overload or critical-load protection during emergencies. These constraints have not been studied yet, and challenge standard solutions to multi-agent coordination tasks.

We consider a multi-agent coordination task as a Markov Game, and propose Situational-Constrained Density-Based Correlated Equilibria (SC-DBCE), a novel solution concept that formalizes situational constraints as logic implications over single constraints. To solve for SC-DBCE, we introduce Situational-Constrained Correlated Policy Iteration (SC-CPI), a reinforcement learning algorithm with a smooth Log-Sum-Exp–based mechanism for optimizing over situational constraints. We evaluate our SC-CPI algorithm on two benchmarks: one Multi-agent game benchmark with three toy scenarios, and one coordination benchmark that models a smart grid scenario and warehouse robot coordination. Experiments demonstrate SC-CPI consistently outperforms baselines in both equilibrium quality and constraint adherence. To our knowledge, this is the first method that learns Correlated Equilibria to Markov Games under situational constraints.

Area: Coordination, Organisations, Institutions, Norms and Ethics (COINE)

Generative A I: I acknowledge that I have read and will follow this policy.

Submission Number: 594