Achieving Multiagent Coordination Through CALA-rFMQ Learning in Continuous Action Space

Wanshu Liu, Chengwei Zhang, Tianpei Yang, Jianye Hao, Xiaohong Li, Zhijie Bao

Abstract: In cooperative multiagent systems, an agent often needs to coordinate with other agents to optimize both individual and system-level payoffs. A lot of multiagent learning approaches have been proposed to address coordination problems in discrete-action cooperative environments. However, it becomes more challenging when faced with continuous action spaces, e.g., slow convergence rate and convergence to suboptimal policy. In this paper, we propose a novel algorithm called CALA-rFMQ (Continuous Action Learning Automata with recursive Frequency Maximum Q-Value) that ensures robust and efficient coordination among multiple agents in continuous action spaces. Experimental results show that CALA-rFMQ facilitates efficient coordination, and outperforms previous works.