Constrained Reinforcement Learning for Safety-Critical Tasks via Scenario-Based Programming

Davide Corsi, Raz Yerushalmi, Guy Amir, Alessandro Farinelli, David Harel, Guy Katz

Keywords: Constrained Reinforcement Learning, Scenario Based Programming, Safety, Robotic Navigation

TL;DR: A novel technique for incorporating domain-expert knowledge to train a constrained DRL agent, based on scenario-based programming paradigm, we validated our method on the popular robotic mapless navigation problem, both physically and in simulation.

Abstract: Deep reinforcement learning (DRL) has achieved groundbreaking successes in various applications, including robotics. A natural consequence is the adoption of this paradigm for safety-critical tasks, where human safety and expensive hardware can be involved. In this context, it is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our technique exploits the scenario-based programming paradigm, designed to specify such knowledge in a simple and intuitive way. While our approach can be considered general purpose, we validated our method by performing experiments on a synthetic set of benchmark environments, and the popular robotic mapless navigation problem, in simulation and on the actual platform. Our results demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and performance of the agent.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Reinforcement Learning (eg, decision and control, planning, hierarchical RL, robotics)