Fault Tolerant Control combining Reinforcement Learning and Model-based Control

Luke Bhan, Marcos Quiñones-Grueiro, Gautam Biswas

Abstract: Fault tolerant control (FTC) focuses on developing algorithms to accommodate the impact of system faults while allowing the system to continuously operate in a degraded manner. Additionally, data-driven methods like reinforcement learning (RL) have shown excellent performance for complex continuous control tasks. However, faults affect the system dynamics which disrupt optimal policy guarantees as the Markov Property cannot be satisfied. In this work, we propose a scheme based on a combination of parameter estimation, RL, and model-based control to handle faults in a continuous control environment. We empirically demonstrate our approach on a complex octocopter trajectory-tracking task subject to single and multi motor faults. We show improved performance compared with nominal hierarchical PID control for large magnitude faults. Lastly, we demonstrate our approach’s robustness against noisy parameter estimation.