Bayesian Inference for Sequence Mixture Density Networks using Bézier Curve Gaussian Processes
Abstract: Probabilistic models for sequential data are the basis for a variety of applications concerned with processing timely ordered information. The predominant approach in this domain is given by recurrent neural networks, implementing either an approximate Bayesian approach (e.g. Variational Autoencoders or Generative Adversarial Networks) or a regression-based approach, i.e. variations of Mixture Density networks (MDN). In this paper, we focus on the \emph{$\mathcal{N}$-MDN} variant, which parameterizes (mixtures of) probabilistic B\'ezier curves (\emph{$\mathcal{N}$-Curves}) for modeling stochastic processes. While MDNs are favorable in terms of computational cost and stability, they generally fall behind approximate Bayesian approaches in terms of expressiveness. Towards this end, we present an approach for closing this gap by enabling full Bayesian inference on top of $\mathcal{N}$-MDNs. For this, we show that $\mathcal{N}$-Curves are a special case of non-stationary Gaussian processes (denoted as $\mathcal{N}$-GP) and then derive corresponding mean and kernel functions for different modalities. Following this, we propose the use of the $\mathcal{N}$-MDN as a data-dependent generator for $\mathcal{N}$-GP prior distributions. We show the advantages granted by this combined model in an application context, using human trajectory prediction as an example.
Submission Length: Long submission (more than 12 pages of main content)
Assigned Action Editor: ~Vincent_Fortuin1
Submission Number: 1475
Loading