A Formal Approach for Safe Reinforcement Learning: A Rate-Adaptive Pacemaker Case Study

Sai Rohan Harshavardhan Vuppala, Nathan Allen, Srinivas Pinisetty, Partha S. Roop

Abstract: As learning-enabled Cyber-Physical Systems (CPSs) are increasingly used in safety-critical settings, there is a growing need to ensure their safety. For example, to tackle the problem of rate-adaptive pacemakers which correct Sinus Node Dysfunction, a Reinforcement Learning (RL) approach may be used to mimic the natural pacing rhythm of the heart. However, this is currently not done and there are no known approaches to ensure the safety of combining RL with conventional pacing algorithms. While there is growing interest on ensuring the safety of AI-enabled CPS, the issue of safe RL for CPS, using light-weight formal methods, has drawn scant attention.