DALMO: A Dynamic Adjustment Lexicographic Multi-objective Approach to Mitigate Negative Side Effects in Multi-agent Systems

Shuli Ai, Wenjun Ma, Jieyu Zhan

Abstract: In multi-agent systems, the inappropriate behavior of agents may lead to a series of negative side effects (NSE) on the environment or other agents, thereby affecting the overall performance of the system. Existing methods for mitigating negative side effects predominantly adopt homogeneous cooperation paradigms, where agents are assumed to have aligned objectives and behave cooperatively. However, these methods critically fail to account for multi-agent scenarios involving heterogeneous objective conflicts, where agents pursue divergent or even conflicting objectives that complicate coordination and side effect mitigation. To address this issue, we present a Dynamic Adjustment Lexicographic Multi-Objective (DALMO) reinforcement learning approach that dynamically adjusts the degree of side effects avoidance based on the strategies of other agents. This approach estimates the side effects of other agents through belief modeling to adjust its own policy and uses a multi-objective integration to balance individual rewards and overall system performance. To evaluate the effectiveness of our approach, we designed a multi-agent grid world experiment in which agents have conflicting objectives. We compared our approach with existing approaches and the experimental results show that our approach achieves higher overall rewards in different scenarios with objective conflicts. This demonstrates that our approach enhances the overall performance of multi-agent systems with objective conflicts by employing corresponding side effect mitigation strategies.

External IDs:dblp:conf/adma/AiMZ25