Robot Reinforcement Learning on the Constraint Manifold

Puze Liu, Davide Tateo, Haitham Bou Ammar, Jan Peters

Keywords: Robot Learning, Reinforcement Learning, Constrained Markov Decision Process, Safe Exploration

Abstract: Reinforcement learning in robotics is extremely challenging due to many practical issues, including safety, mechanical constraints, and wear and tear. Typically, these issues are not considered in the machine learning literature. One crucial problem in applying reinforcement learning in the real world is Safe Exploration, which requires physical and safety constraints satisfaction throughout the learning process. To explore in such a safety-critical environment, leveraging known information such as robot models and constraints is beneficial to provide more robust safety guarantees. Exploiting this knowledge, we propose a novel method to learn robotics tasks in simulation efficiently while satisfying the constraints during the learning process.