Attention is Advantage: How the Weaker Defeats the Stronger Through Cooperation

Bowei Cao, Kangyao Huang, Yaorui Yin, Di Guo, Huaping Liu

Keywords: interpretability and explainable AI, multiagent learning, reinforcement learning, adversarial learning

Abstract: The phenomenon of weaker groups overcoming stronger opponents through cooperation in nature has inspired our exploration of cooperative-competitive mechanisms in multi-agent systems. In this work, we investigate emergent coordination policies in asymmetrical confrontations, focusing on how weaker agents collectively counter stronger opponents. The challenge of modeling such intricate interplay with a multilayer perceptron led us to adopt a Transformer architecture, which excels at capturing the complex, dynamic relationships between agents. We develop a two-phase curriculum training, with an attention-based strategy that effectively addresses policy training challenges in high-dimensional state spaces, and construct a scalable arena task that validates their effectiveness. Motivated by the Transformer's cooperative advantage, we utilize integrated gradients to attribute the contribution of attentions for each action dimension—thereby bridging attentions and behaviors and revealing how attentional dynamics scaffold collective superiority. This research provides a paradigm and an analytic approach for distributed collaboration in mixed adversarial systems.

Primary Area: interpretability and explainable AI

Submission Number: 12344