Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Inference and Introspection in Deep Generative Models of Sparse Data
Rahul G. Krishnan, Matthew Hoffman
Nov 04, 2016 (modified: Jan 14, 2017)ICLR 2017 conference submissionreaders: everyone
Abstract:Deep generative models such as deep latent Gaussian models (DLGMs) are powerful and popular density estimators. However, they have been applied almost exclusively to dense data such as images; DLGMs are rarely applied to sparse, high-dimensional integer data such as word counts or product ratings. One reason is that the standard training procedures find poor local optima when applied to such data. We propose two techniques that alleviate this problem, significantly improving our ability to fit DLGMs to sparse, high-dimensional data. Having fit these models, we are faced with another challenge: how to use and interpret the representation that we have learned? To that end, we propose a method that extracts distributed representations of features via a simple linearization of the model.
TL;DR:We study two techniques to improve learning in deep generative models on sparse, high-dimensional text data. We also propose an algorithmic tool to visualize and introspect arbitrarily deep learned models.
Keywords:Unsupervised Learning, Deep learning
Conflicts:cs.nyu.edu, adobe.com, columbia.edu
Enter your feedback below and we'll get back to you as soon as possible.