HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents

Tristan Tomilin, Meng Fang, Mykola Pechenizkiy

Keywords: reinforcement learning, AI safety, safe RL, constrained RL, benchmark, vizdoom, 3D, difficulty level

TL;DR: A Safe RL benchmark for vision-based learning in complex navigable 3D environments.

Abstract: Advancing safe autonomous systems through reinforcement learning (RL) requires robust benchmarks to evaluate performance, analyze methods, and assess agent competencies. Humans primarily rely on embodied visual perception to safely navigate and interact with their surroundings, making it a valuable capability for RL agents. However, existing vision-based 3D benchmarks only consider simple navigation tasks. To address this shortcoming, we introduce **HASARD**, a suite of diverse and complex tasks to **HA**rness **SA**fe **R**L with **D**oom, requiring strategic decision-making, comprehending spatial relationships, and predicting the short-term future. HASARD features three difficulty levels and two action spaces. An empirical evaluation of popular baseline methods demonstrates the benchmark's complexity, unique challenges, and reward-cost trade-offs. Visualizing agent navigation during training with top-down heatmaps provides insight into a method's learning process. Incrementally training across difficulty levels offers an implicit learning curriculum. HASARD is the first safe RL benchmark to exclusively target egocentric vision-based learning, offering a cost-effective and insightful way to explore the potential and boundaries of current and future safe RL methods. The environments and baseline implementations are open-sourced.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Tristan_Tomilin1

Track: Fast Track: published work

Publication Link: https://iclr.cc/virtual/2025/poster/30963

Submission Number: 37