Evaluation of Deep Reinforcement Learning Methods for Modular Robots

Risto Kojcev, Nora Etxezarreta, Alejandro Hernández, Víctor Mayoral

Abstract: We propose a novel framework for Deep Reinforcement Learning (DRL) in modular robotics using traditional robotic tools that extend state-of-the-art DRL implementations and provide an end-to-end approach which trains a robot directly from joint states. Moreover, we present a novel technique to transfer these DLR methods into the real robot, aiming to close the simulation-reality gap. We demonstrate the robustness of the performance of state-of-the-art DRL methods for continuous action spaces in modular robots, with an empirical study both in simulation and in the real robot where we also evaluate how accelerating the simulation time affects the robot's performance. Our results show that extending the modular robot from 3 degrees-of-freedom (DoF), to 4 DoF, does not affect the robot's learning. This paves the way towards training modular robots using DRL techniques.

TL;DR: Introduction of framework for training modular robots in simulation using Reinforcement Learning and simulation to real robot transfer.

Keywords: Robotics, modular robots, simulation, reinforcement learning, simulation framework