MeMo: Meaningful, Modular Controllers via Noise Injection

Megan Tjandrasuwita, Jie Xu, Armando Solar-Lezama, Wojciech Matusik

Keywords: modular neural network policy, policy transfer, imitation learning, reinforcement learning

TL;DR: We present a method for pretraining modular controllers from a single robot and environment that significantly speeds up RL training for locomotion and grasping when reused on more complex morphologies.

Abstract: Robots are often built from standardized assemblies, (e.g. arms, legs, or fingers), but each robot must be trained from scratch to control all the actuators of all the parts together. In this paper we demonstrate a new approach that takes a single robot and its controller as input and produces a set of modular controllers for each of these assemblies such that when a new robot is built from the same parts, its control can be quickly learned by reusing the modular controllers. We achieve this with a framework called MeMo which learns (Me)aningful, (Mo)dular controllers. Specifically, we propose a novel modularity objective to learn an appropriate division of labor among the modules. We demonstrate that this objective can be optimized simultaneously with standard behavior cloning loss via noise injection. We benchmark our framework in locomotion and grasping environments on simple to complex robot morphology transfer. We also show that the modules help in task transfer. On both structure and task transfer, MeMo achieves improved training efficiency to graph neural network and Transformer baselines.

Primary Area: Robotics

Submission Number: 3440