Modular Recurrence in Contextual MDPs for Universal Morphology Control

Laurens Engwegen, Daan Brinks, Wendelin Boehmer

Keywords: Deep Reinforcement Learning, Robotic Control, Generalization

TL;DR: Introduction of modular recurrence in the architecture of deep reinforcement learning agents for improved generalization in robotic control.

Abstract: A universal controller for any robot morphology would greatly improve computational and data efficiency. By utilizing contextual information about the properties of individual robots and exploiting their modular structure in the architecture of deep reinforcement learning agents, steps have been made towards multi-robot control. Generalization to new, unseen robots, however, remains a challenge. In this paper we hypothesize that the relevant contextual information is partially observable, but that it can be inferred through interactions for better generalization to contexts that are not seen during training. To this extent, we implement a modular recurrent architecture and evaluate its generalization performance on a large set of MuJoCo robots. The results show a substantial improved performance on robots with unseen dynamics, kinematics, and topologies, in four different environments.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Laurens_Engwegen1

Track: Regular Track: unpublished work

Submission Number: 78