Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner

Philémon Brakel, Steven Bohez, Leonard Hasenclever, Nicolas Heess, Konstantinos Bousmalis

Abstract: We propose a simple imitation learning procedure for learning locomotion controllers that can walk over very challenging terrains. We use trajectory optimization (TO) to produce a large dataset of trajectories over procedurally generated terrains and use Reinforcement Learning (RL) to imitate these trajectories. We demonstrate with a realistic model of the ANYmal robot that the learned controllers transfer to unseen terrains and provide an effective initialization for fine-tuning on challenging terrains that require exteroception and precise foot placements. Our setup combines TO and RL in a simple fashion that overcomes the computational limitations and need for a robust tracking controller of the former and the exploration and reward-tuning difficulties of the latter.