Prioritizing safety via curriculum learning

Published: 07 Aug 2024, Last Modified: 07 Aug 2024RLSW 2024 PosterEveryoneRevisionsBibTeXCC BY 4.0
Confirmation: Yes
Keywords: Curriculum learning, constrained reinforcement learning
TL;DR: We propose a safe curriculum generation method that reduces safety constraint violations during training while boosting the learning speed of constrained RL agents.
Abstract: Curriculum learning for reinforcement learning (RL) aims to accelerate learning by generating sequences of tasks of increasing difficulty. Besides its sample-efficiency benefits, curriculum learning has the potential to address safety-critical settings where an RL agent must adhere to safety constraints. However, existing curriculum generation approaches still overlook such constraints and thus propose tasks that cause RL agents to violate safety constraints during training and behave suboptimally after. We propose a safe curriculum generation approach (SCG) that aligns the objectives of constrained RL and curriculum learning: improving safety during training and boosting learning speed. SCG generates sequences of tasks where the RL agent can be both safe and performant by initially preferring tasks with minimum safety violations over high-reward ones. In constrained RL environments, we empirically show that compared to the state-of-the-art curriculum learning approaches and their naively modified safe versions, SCG achieves optimal performance and the lowest amount of constraint violations during training.
Submission Number: 3
Loading