Variational Autoencoder with Interactive Attention for Affective Text Generation

Ruijun Chen, Jin Wang, Xuejie Zhang

Abstract: Human language has the ability to convey the speaker’s emotions, such as happiness, sadness, or anger. Existing text generation methods mainly focus on the sequence-to-sequence (Seq2Seq) model that applied an encoder to transform the input text into latent representation and a decoder to generate texts from the latent representation. To control the sentiment of the generated text, these models usually concatenate a disentangled feature into the latent representation. However, such a method is only suitable for short texts, since the sentiment information may gradually dissipate as the text becomes longer. To address this issue, a variational autoencoder with interactive variation attention was proposed in this study. Unlike the previous method of directly connecting sentiment information with the latent variables to control text generation, the proposed model adds the sentiment information into variational attention with a dynamic update mechanism. At each timestep, the model leverage both the variational attention and hidden representation to decode and predict the target word and then uses the generated results to update the emotional information in attention. It can keep track of the attention history, which encourages the attention-based VAE to control better the sentiment and content of generating text. The empirical experiments were conducted using the SST dataset to evaluate the generation performance of the proposed model. The comparative results showed that the proposed method outperformed the other methods for affective text generation. In addition, it can still maintain accurate sentiment information and sentences smoothness even in the longer text.