Effective Unsupervised Constrained Text Generation based on Perturbed Masking

Anonymous

Abstract: Unsupervised constrained text generation aims to generate text under a given set of constraints without any supervised data. Current state-of-the-art methods stochastically sample edit positions which may cause unnecessary search steps. In this paper, we propose PMCTG to improve effectiveness by searching for the best position and action in each step. Specifically, PMCTG extends the perturbed masking technique to effectively search for the best edit position. Then it uses proposed multi-aspect scoring functions to select edit action to further reduce search difficulty. Since PMCTG does not require supervised data, it can extend to different generation tasks. We show PMCTG achieves state-of-the-art results in keywords-to-sentence generation and paraphrasing.