Constrained Reinforcement Learning for Safety-Critical Tasks via Scenario-Based ProgrammingDownload PDF


22 Sept 2022, 12:33 (modified: 26 Oct 2022, 14:02)ICLR 2023 Conference Blind SubmissionReaders: Everyone
Keywords: Constrained Reinforcement Learning, Scenario Based Programming, Safety, Robotic Navigation
TL;DR: A novel technique for incorporating domain-expert knowledge to train a constrained DRL agent, based on scenario-based programming paradigm, we validated our method on the popular robotic mapless navigation problem, both physically and in simulation.
Abstract: Deep reinforcement learning (DRL) has achieved groundbreaking successes in various applications, including robotics. A natural consequence is the adoption of this paradigm for safety-critical tasks, where human safety and expensive hardware can be involved. In this context, it is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our technique exploits the scenario-based programming paradigm, designed to specify such knowledge in a simple and intuitive way. While our approach can be considered general purpose, we validated our method by performing experiments on a synthetic set of benchmark environments, and the popular robotic mapless navigation problem, in simulation and on the actual platform. Our results demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and performance of the agent.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Supplementary Material: zip
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
Submission Guidelines: Yes
Please Choose The Closest Area That Your Submission Falls Into: Reinforcement Learning (eg, decision and control, planning, hierarchical RL, robotics)
3 Replies