Multi-agent Reinforcement Learning with Hybrid Action Space for Free Gait Motion Planning of Hexapod Robots

Huiqiao Fu, Kaiqiang Tang, Peng Li, Guizhou Deng, Chunlin Chen

Keywords: Free Gait, Hexapod Robot, Hybrid Action Space, Multi-agent Reinforcement Learning

TL;DR: We tackle the problem of free gait motion planning for hexapod robots walking in randomly generated plum blossom pile environments.

Abstract: Legged robots are able to overcome challenging terrains through diverse gaits formed by contact sequences. However, environments characterized by discrete footholds present significant challenges. In this paper, we tackle the problem of free gait motion planning for hexapod robots walking in randomly generated plum blossom pile environments. Specifically, we first address the complexity of multi-leg coordination in discrete environments by treating each leg of the hexapod robot as an individual agent. Then, we propose the Hybrid action space Multi-Agent Soft Actor Critic (Hybrid-MASAC) algorithm capable of handling both discrete and continuous actions. Finally, we present an integrated free gait motion planning method based on Hybrid-MASAC, streamlining gait, Center of Mass (COM), and foothold sequences planning into a single model. Comparative and ablation experiments in both of the simulated and real plum blossom pile environments demonstrate the feasibility and efficiency of our method.

Student Paper: yes

Submission Number: 291