Generalized Model Recovery from Sampled Data

Ayan Banerjee, Sandeep K. S. Gupta

Abstract: Model recovery (MR) is the process of extracting governing equations of a system from input output traces with two objectives: a) accurately reconstructing the input output traces, and b) reducing error in model coefficient estimation. Higher sampling rates increase model coefficient related in-formation content in the traces improving the likelihood of MR techniques to optimize both objectives. At sampling rates above the Nyquist rate, the information content saturates. At this point any MR technique that reduces reconstruction error by default also decreases model coefficient estimation error. In practical deployment, due to resource constraints data maybe sampled at or below the Nyquist rate, where full information about model coefficients is not embedded in the traces. To improve MR performance, MR techniques should incorporate external knowledge such as the sparsity structure of the nonlinear dynamics to target reduction in model coefficient estimation error. In this manuscript, we: i) analyze the effect of sampling frequency on Cramér Rao Lower Bound (CRLB), a fundamental lower bound on the best possible model coefficient extraction accuracy for any MR technique (unbiased estimator), ii) formally quantify the effect of the sampling frequency on the generalization performance (on unseen traces) of MR technique, iii) demonstrate that state-of-the-art (SOTA) sparse identification of nonlinear dynamics (SINDY) based MR techniques have poor generalization error at low sampling rates, and iv) compare with EMILY, a liquid time-constant neural network (LTC-NN) based architecture that can improve the generalization performance under low sampling rates. We demonstrate that the automated differentiation property of LTC-NN nodes can maintain model structural constraints in between sample times and provides superior generalized MR performance than SOTA nonlinear MR techniques on several benchmarks.