Paraphrase generation based on VAE and pointer-generator networks

Lohith Ravuru, Hyungtak Choi, Siddarth K.M., Hojung Lee, Inchul Hwang

Abstract:

Paraphrase generation is a challenging task that involves expressing the meaning of a sentence using synonyms or different phrases, either to achieve variations or a certain stylistic response. Most previous sequence-to-sequence (Seq2Seq) models focus on either generating variations or preserving the content. We mainly address the issue of preserving the content in a sentence while generating diverse paraphrases. In this paper, we propose a novel approach for paraphrase generation using variational autoencoder (VAE) and Pointer
Generator Network (PGN). The proposed model uses a copy mechanism to control the content transfer, a VAE to introduce variations and a training technique to restrict the gradient flow for efficient learning. Our evaluations on QUORA and MS COCO datasets show that our model outperforms the state-of-the-art approaches and the generated paraphrases are highly diverse as well as consistent with their original meaning.

Index Terms— Paraphrase Generation, Variational Autoencoder, Pointer Generator Network, Sequence to Sequence, Long Short-Term Memory (LSTM)