Topic-Aware Variational Auto-Encoders for Controllable Text Generation

Anonymous

Paper Link: https://openreview.net/forum?id=l_-4qR9XlYl

Paper Type: Long paper (up to eight pages of content + unlimited references and appendices)

Abstract: In this paper, we propose Topic-Aware Variational Auto-Encoders for Controllable Text Generation (TA-VAE). Distinct from existing VAE based approaches, we explicitly model document topic and sequence apart: a text variational auto-encoder (VAE) is utilized for sequence modeling, whose posterior is remolded by a Householder flow to be compatible with the non-isotropic allocation of texts (with diverse topics) in latent space; a variational topic model with its prior conditioned on well-crafted sequential posterior to take advantage from acquired text sequential information. Besides, an explicit discriminator (based on the topic encoder) as well as a mutual information maximization term (on topic latent code and observed data) are additionally added to enhance the utterance of topic behalf. Encouraging experimental results on real-world datasets demonstrate that the proposed model not only learns interpretable topic representations, but is fully capable of generating high-quality paragraphs that are grammatically reasonable and semantically consistent.