Bayesian inference for time-varying applications: Particle-based Gaussian process approaches

Yali Wang, Brahim Chaib-draa

Abstract: Gaussian process (GP) is a popular non-parametric model for Bayesian inference. However, the performance of GP is often limited in temporal applications, where the input–output pairs are sequentially-ordered, and often exhibit time-varying non-stationarity and heteroscedasticity. In this work, we propose two particle-based GP approaches to capture these distinct temporal characteristics. Firstly, we make use of GP to design two novel state space models which take the temporal order of input–output pairs into account. Secondly, we develop two sequential-Monte-Carlo-inspired particle mechanisms to learn the latent function values and model parameters in a recursive Bayesian framework. Since the model parameters are time-varying, our approaches can model non-stationarity and heteroscedasticity of temporal data. Finally, we evaluate our proposed approaches on a number of challenging time-varying data sets to show effectiveness. By comparing with several related GP approaches, we show that our particle-based GP approaches can efficiently and accurately capture temporal characteristics in time-varying applications.