SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
Keywords: Machine Learning for Robot Control, Reinforcement Learning, Assembly
Abstract: Recent years have seen the development of many methods for robotic reinforcement learning (RL), some of which can even operate on complex image observations, train in the real world, and incorporate auxiliary data, such as demonstrations and prior experience. However, despite these advances, robotic RL remains hard to use. It is acknowledged among practitioners that the particular implementation details of these algorithms are often just as important (if not moreso) for performance as the choice of algorithm that is actually used. We posit that a significant challenge to widespread adoption of robotic RL, as well as further development of robotic RL methods, is the comparative inaccessibility of such methods. To address this challenge, we developed a carefully implemented library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment, high-quality controllers for a few common robots, and a number of challenging example tasks. We provide this library as a resource for the community, describe its design choices, and present experimental results. Perhaps surprisingly, we find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation in less than an hour of training per policy, matching or improving over state-of-the-art results reported for similar tasks in the literature. We hope that these promising results and our high-quality open-source implementation will provide a tool for the robotics community to study new developments in robotic RL. Our code and videos can be found a https://serl-robot.github.io
Submission Number: 21
Loading