Structural Knowledge Informed Continual Multivariate Time Series Forecasting
Primary Area: transfer learning, meta learning, and lifelong learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Continual Learning, Multivariate Time Series Forecasting, Graph Structure Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Recent studies in multivariate time series (MTS) forecasting reveal that explicitly modeling the hidden dependencies among different time series can yield promising forecasting performance and reliable explanations. However, modeling variable dependencies remains underexplored when a sequence of MTS under different regimes (stages) is continuously accumulated. Due to the potential distribution and dependency disparities, the underlying model may encounter the catastrophic forgetting problem, i.e., it is challenging to memorize and infer different types of variable dependencies across different regimes while maintaining forecasting performance. To this end, we propose a novel Structural Knowledge Informed Continual Learning (SKI-CL) framework to perform MTS forecasting under the continual learning setting, which leverages the structural knowledge to characterize the dynamic variable dependencies within each regime. Specifically, we first develop a deep forecasting model with a graph learner that enables fine-grained dynamic MTS modeling. Next, we impose a regularization scheme to ensure the consistency between the learned variable dependencies and the structure knowledge (e.g., physical constraints, domain knowledge, feature similarity). Finally, we develop a representation-matching memory replay scheme to tackle the catastrophic forgetting problem, which maximizes the temporal coverage of MTS data to efficiently preserve the underlying temporal dynamics and dependency structures of each regime. Thoroughly empirical studies on synthetic and real-world benchmarks validate SKI-CL's efficacy and advantages over the state-of-the-art in tackling continual MTS forecasting tasks. In addition, SKI-CL can accurately infer learned dependencies in the test stage based on MTS data without knowing the identities of different regimes.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4554
Loading