Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation

Anonymous

Abstract: Large pretrained language models offer powerful generation capabilities, but suffer from a lack of interpretability and fine-grained control. We propose an approach to fine-grained control in generating text directly from a semantic representation, Abstract Meaning Representation (AMR) by augmenting the nodes with syntactic tags. We experiment with English-language generation of three modes of syntax relevant to the framing of a sentence - verb voice (active or passive), verb tense, and realization of human entities - and demonstrate that they can be reliably controlled. Controlling how information is framed is important for applications such as summarization, which aim to highlight salient information.