Diffusion Models for Constrained Planning with Probabilistic Risk-awareness Guarantees

Albin Larsson Forsberg, Kenneth Lau, Alexandros Nikou, Aneta Vulgarakis Feljan, Jana Tumova

Abstract: Diffusion models have shown great potential in generating trajectory plans for agents in environments with unknown dynamics. However, such models provide no safety guarantees. In this work, we focus on risk-aware planning with respect to safety constraints and introduce a probabilistically risk-aware variant of Diffuser (PRA-Diffuser). The diffusion model initially learns a distribution over trajectories that may or may not be unsafe. We then fine-tune this model to reduce the probability of sampling such unsafe trajectories. We analyze the proposed solution and introduce a provable lower bound on risk of safety violation leveraging concentration inequalities for conditional Value-at-Risk. Our approach can be applied to models that have been pre-trained, potentially from datasets containing unsafe trajectories. Our empirical results demonstrate that our approach significantly reduces unsafe trajectories generated by the diffusion model across multiple environments.