Temporal-difference learning for online reachability analysis

Anayo K. Akametalu, Claire J. Tomlin

Abstract: Hamilton-Jacobi-Isaacs (HJI) reachability analysis has been employed to guarantee safety in a number of applications including robotics, air traffic control, and control of HVAC systems. The current standard for these methods can result in overly-conservative controllers that degrade system performance with respect to other objectives. There has been interest in incorporating online machine learning techniques to reduce the conservativeness of the controller. However, recent efforts have resulted in methods that are computationally inefficient and scale poorly with the dimension of the state space. We propose a novel online reachability update algorithm based on Temporal-Difference (TD) learning that is computationally more efficient. Our algorithm is demonstrated on a simulation of a quadrotor learning to track a trajectory in a confined space. Our method outperforms standard reachability-based controllers when it comes to other (non-safety) objectives.