Poisson-Randomized Gamma Dynamical Systems

Aaron Schein, Scott Linderman, Mingyuan Zhou, David Blei, Hanna Wallach

Abstract: This paper presents the Poisson-randomized gamma dynamical system (PrGDS), a model for sequentially-observed count tensors that encodes a strong inductive bias towards sparsity and burstiness. The PrGDS is based on a new motif in Bayesian latent variable modeling---an alternating chain of discrete Poisson and continuous gamma latent states---that is analytically convenient and computationally tractable, yielding closed-form complete conditionals for all variables by way of the Bessel distribution and a novel discrete distribution that we call the shifted confluent hypergeometric distribution. We draw connections to closely-related models and compare the PrGDS to them in studies of real-world count data of text, international events, and neural spike trains. We find that a sparse variant of the PrGDS---which allows continuous latent states to take values of exactly zero---often obtains the lowest smoothing and forecasting perplexity of all models and is uniquely capable of inferring latent structure that is highly localized in time.

CMT Num: 395

Code Link: https://github.com/aschein/prgds