Streaming Model Selection via Online Factorized Asymptotic Bayesian Inference

Chunchen LIU, Lu Feng, Ryohei Fujimaki

Abstract:

Recent growing needs for real time data analytics have increased importance of streaming model selection. Real world
streaming observations are often obtained by dynamicallychanging or heterogeneous data sources, and learning machines
must identify the complexities of the data generation processes on the fly without prior knowledge. This paper proposes online
FAB (OFAB) inference as a general framework for streaming model selection of latent variable models. The key idea in OFAB
inference is degeneration, i.e. it intentionally considers a “redundant” latent space and dynamically derives a “non-redundant”
latent sub-space using a FAB-unique shrinkage mechanism on demand. By integrating the idea of stochastic variational inference,
OFAB automatically and dynamically selects the best dimensionality of latent variables in a streaming and Bayesian
principled manner. Empirical results on two applications, density estimation and abnormal detection, show that online FAB (OFAB)
outperformed the state-of-the-art online inference methods.