A Massively Parallel Benchmark for Safe Dexterous Manipulation

Yiran Geng, Jiamg Ji, Yuanpei Chen, Long Yang, Yaodong Yang

Keywords: Dexterous Manipulation, Safe Reinforcement Learning, Robot Learning

TL;DR: Safety Dexteroushands is the first large-scale task collection focused on safe dexterous manipulation, offering 10+ manipulators and 100+ task combinations.

Abstract: Safe Reinforcement Learning (Safe RL) aims to maximize expected total rewards and avoids violation of certain constraints at the same time. Many constrained environments have been designed to evaluate Safe RL algorithms, but they are more focused on simple navigation tasks and have tremendous gaps with the real world. Meanwhile, dexterous manipulation is a challenging topic in the field of robotics, and places high demands on safety constraints to ensure reliable manipulation in the real world. Consequently, we propose Safety DexterousHands, a massively parallel physical benchmark to facilitate experimental validation in Safe RL research. Safety DexterousHands is built in the Isaac Gym, a GPU-level parallel simulator that enables highly efficient RL training. We designed a series of challenging dexterous manipulation tasks around the safety constraints. To the best of our knowledge, Safety DexterousHands is the first large-scale benchmark focused on safe dexterous manipulation, offering 10+ manipulators and 100+ task combinations. Our experimental results show that Safe RL algorithms can perfectly solve the safe dexterous manipulation task by exploiting the sparse cost penalty, while unsafe RL algorithms struggle to solve most tasks without causing disruption. We expect that this benchmark can deliver a reliable and comprehensive evaluation for Safe RL algorithms and promote a integration of Safe RL and dexterous manipulation.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Reinforcement Learning (eg, decision and control, planning, hierarchical RL, robotics)