Neural ODE with Differentiable Hidden State for Irregular Time Series

Yan Fang, Xu Wang, Zeyu Liang, Yudong Zhang, Pengkun Wang, Binwu Wang, Yang Wang

Keywords: time series analysis, irregular time series, neural ordinary differential equations

TL;DR: We propose a NODE method with continuous state space for irregular time series analysis.

Abstract:

Capturing the continuous underlying dynamics of irregular time series is essential for accurately reflecting the ongoing evolution and intricate correlations within the data. The discrete nature of current models, including RNN-based models and transformer variants, poses challenges when it comes to generalizing to the continuous-time data paradigms, which is necessary for capturing ongoing dynamics of irregular time series.
Neural Ordinary Differential Equations (NODEs) assume a continuous latent dynamic and provide an elegant framework for irregular time series analysis. However, integrating new information while maintaining the continuity of latent dynamics remains challenging.
To tackle this problem, we introduce Differentiable Hidden State (DHS) enhanced neural ODE, a data-dependent framework that is capable of effectively capturing temporal dependencies and ensuring the continuity of the hidden process. We leverage the theory of generalized inverses to innovatively compute attention mechanism in reverse and obtain a continuous representation. To capture more accurate temporal relationships, we introduce Hoyer metric and maximize the sparsity of it. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of our model.

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9179