Morphology generalizable reinforcement learning via multi-level graph features

Yansong Pan, Rui Zhang, Jiaming Guo, Shaohui Peng, Fan Wu, Kaizhao Yuan, Yunkai Gao, Siming Lan, Ruizhi Chen, Ling Li, Xing Hu, Zidong Du, Zihao Zhang, Xin Zhang, Wei Li, Qi Guo, Yunji Chen

Abstract: Controlling a group of robots with diverse morphologies using a unified policy, known as morphology generalizable control, is a challenging problem in robotic control. Existing graph neural network-based (GNN-based) methods suffer from inefficient modular communication due to non-adjacent modules having to communicate across multiple hops, while transformer-based methods neglect morphology prior information which is crucial for morphology generalizable control. To overcome these limitations, in this work, we propose MG2(Morphology Generalizable Reinforcement Learning via Multi-level Graph Features) which incorporates multi-level graph features derived from the morphology graph into the transformer architecture. To effectively incorporate morphology information while achieving efficient modular communication, MG2 introduces graph features three-levels, local, global, and relative graph features, and incorporates them into the transformer architecture. By introducing morphology prior information, MG2 improves multi-task training and generalization performance in morphology-generalizable reinforcement learning. The performance enhancements are evaluated primarily on the SMP benchmark and consolidated on several UNIMAL robots.