Guided Safe Shooting: model based reinforcement learning with safety constraints
Abstract: Reinforcement learning (RL) has achieved remarkable results in a variety of complex control tasks and decision-making problems, including the game of Go and autonomous driving.
However, applying RL to real-world scenarios, especially those requiring safety-awareness, poses significant challenges.
Existing approaches that enforce strict safety guarantees can limit exploration and lead to suboptimal policies in settings where some safety violations are tolerated.
Conversely, Quality-Diversity (QD) algorithms maximize exploration and search for high-reward policies, but they require many interactions with the environment, potentially exposing the agent to high-risk situations.
In this paper, we propose Guided Safe Shooting (GuSS), a Model-Based RL (MBRL) approach that leverages a QD algorithm as a planner with a soft safety objective.
GuSS is the first MBRL approach that combines QD and safety objectives in a principled way.
Our experiments, conducted on three OpenAI gym environments with safety constraints, show that GuSS reduces safety violations while achieving higher performances compared to the considered baselines, thanks to its increased exploration.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Yang_Li2
Submission Number: 1903
Loading