Mixed Membership Recurrent Neural Networks

Ghazal Fazelnia

Abstract:

Models for sequential data such as the recurrent neural network (RNN) often
implicitly model a sequence as having a fixed time interval between observations and do
not account for group-level effects when multiple sequences are observed. We propose a
model for grouped sequential data based on the RNN that accounts for varying time
intervals between observations in a sequence by learning a group-level base parameter to
which each sequence can revert. Our approach is motivated by the mixed membership
framework, and we show how it can be used for dynamic topic modeling in which the
distribution on topics (not the topics themselves) are evolving in time. We demonstrate
our approach on a dataset of 3.4 million online grocery shopping orders made by 206K
customers.