Beyond Scalar Welfare: Enforcing Identity-Aware Equity in Multi-Agent Reinforcement Learning

Nhat-Hoang P. Nguyen, Thinh Pham, Trung-Hoang Le, sameer alam, Vu N. Duong

Keywords: equity, fairness, cooperative, multi-agent reinforcement learning

TL;DR: We formalize an identity-aware equity metric and an Equity-Lagrangian primal–dual update that enforces near-equal outcomes in cooperative MARL; analyses of experiments show better equity than SWF/dispersion baselines at comparable outcomes.

Abstract: Equity in cooperative multi-agent reinforcement learning (MARL) is imposed through a single scalar, such as an inequality-averse social welfare function or a dispersion index (e.g., Gini, coefficient of variation, or Jain). These proxies conflate two dimensions of inequity, including who and how much, offering no guarantee of identity-level equity. We introduce an identity-aware notion of equity for otherwise heterogeneous agents, requiring each agent's outcome to remain within a relative tolerance band around the mean. From this premise, we propose the Minimum Reallocation of Excess (MRE), separating incidence (who fall outside the band) from displacement (how much outside-band mass they carry), measuring the least reassignment needed to restore equity. We prove convexity, piecewise linearity, Lipschitz continuity, and Pigou-Dalton compliance, yielding a stable and aligned metric to optimize equity-improving transfers. We further design the Equity-Lagrangian Dual (ELD) update, enforcing an expected-MRE constraint while maximizing return. The update is backbone-agnostic and preserves decentralized execution. Formal case analyses show that dispersion and welfare proxies can misrank allocations or fail to enforce identity-level equity, whereas ELD attains near-equal outcomes at comparable efficiency. Illustrative experiments in two cooperative games with state-of-the-art MARL frameworks show that ELD reduces MRE relative to proxy-based constraints while maintaining aggregate returns, proving that identity-aware equity can be enforced without architectural changes. This framework replaces proxy objectives with a measurable equity target and a plug-in primal-dual rule, enabling identity-level equity during MARL training.

Area: Coordination, Organisations, Institutions, Norms and Ethics (COINE)

Generative A I: I acknowledge that I have read and will follow this policy.

Submission Number: 1098