Deep Bayesian Filter for Bayes-Faithful Data Assimilation

Yuta Tarumi, Keisuke Fukuda, Shin-ichi Maeda

TL;DR: proposed a novel data assimilation method that achieves the best performance for nonlinear state space models

Abstract: Data assimilation for nonlinear state space models (SSMs) is inherently challenging due to non-Gaussian posteriors. We propose Deep Bayesian Filtering (DBF), a novel approach to data assimilation in nonlinear SSMs. DBF introduces latent variables $h_t$ in addition to physical variables $z_t$, ensuring Gaussian posteriors by (i) constraining state transitions in the latent space to be linear and (ii) learning a Gaussian inverse observation operator $r(h_t|o_t)$. This structured posterior design enables analytical recursive computation, avoiding the accumulation of Monte Carlo sampling errors over time steps. DBF optimizes these operators and other latent SSM parameters by maximizing the evidence lower bound. Experiments demonstrate that DBF outperforms existing methods in scenarios with highly non-Gaussian posteriors.

Lay Summary:

Weather, ocean, and seismic research all rely on data assimilation, which blends observations with physics-based models to track a system’s physical state. Despite its large impact on prediction accuracy, operational weather-forecasting systems still rely on ensemble Kalman filters, while most machine-learning studies focus on data-driven forecasts that omit explicit physics. Methods such as the ensemble Kalman filter struggle in highly nonlinear regimes because they assume Gaussian posterior distributions even when the true dynamics are non-Gaussian.

We propose the Deep Bayesian Filter (DBF), which learns a latent space in which the dynamics are linear—effectively seeking a Koopman-operator representation—so each Bayesian update can be computed in closed form. This analytical update eliminates the need for Monte-Carlo sampling during inference, preventing the error accumulation seen in dynamical VAE algorithms. DBF also scales gracefully to high-dimensional problems thanks to the block-diagonal structure of its dynamics matrix. On the Lorenz-96 benchmark with a nonlinear observation operator, it outperforms existing assimilation methods while running faster.

More accurate atmospheric states could lead to earlier and more reliable warnings of extreme weather events. The same framework also applies to any sequential, nonlinear state-filtering problem, broadening the reach of physics-guided Bayesian inference across the sciences.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https://github.com/pfnet-research/deep-bayesian-filter

Primary Area: Probabilistic Methods->Variational Inference

Keywords: Data Assimilation, Variational Inference, State-Space Models, Koopman Operator

Submission Number: 8378