Modeling Temporal Data as Continuous Functions with Process Diffusion

Marin Biloš, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, Stephan Günnemann

Keywords: time series, stochastic process, diffusion, probabilistic forecasting, score-based matching

TL;DR: We modify the diffusion framework to model continuous functions and apply the learned generative model on different time series tasks.

Abstract:

Temporal data like time series are often observed at irregular intervals which is a challenging setting for the existing machine learning methods. To tackle this problem, we view such data as samples from some underlying continuous function. We then define a diffusion-based generative model that adds noise from a predefined stochastic process while preserving the continuity of the resulting underlying function.
A neural network is trained to reverse this process which allows us to sample new realizations from the learned distribution. We define suitable stochastic processes as noise sources and introduce novel denoising and score-matching models on processes. Further, we show how to apply this approach to the multivariate probabilistic forecasting and imputation tasks. Through our extensive experiments, we demonstrate that our method outperforms previous models on synthetic and real-world datasets.

Student Paper: Yes

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2211.02590/code)