Model-based Unknown Input Estimation via Partially Observable Markov Decision Processes

Wei Liu, Zhilu Lai, Charikleia Stoura, Kiran Bacsa, Eleni Chatzi

Keywords: unknown input estimation, partially observable markov decision process, model-based reinforcement learning, model predictive control, cross-entropy method, dynamics modeling

Abstract: In the context of condition monitoring for structures and industrial assets, the estimation of unknown inputs, usually referring to acting loads, is of salient importance for guaranteeing safe and performant engineered systems. In this work, we propose a novel method for estimating unknown inputs from measured outputs, for the case of systems with a known or learned model of the underlying dynamics. The objective is to infer those system inputs that will reproduce the actual measured outputs; this can be reformulated as a Partially Observable Markov Decision Process (POMDP) problem and solved with well-established planning algorithms for POMDPs. The cross-entropy method (CEM) is adopted in this paper for solving the POMDP due to its efficiency and robustness. The proposed method is demonstrated using simulated dynamical systems for structures with known dynamics, as well as a real wind turbine with learned dynamics inferred through Neural Extended Kalman Filters (Neural EKF); a deep learning-based method for learning stochastic dynamics, previously proposed by the authors.