Annotating FrameNet via Structure-Conditioned Language Generation

Anonymous

Abstract: Despite the mounting evidence for generative capabilities of language models in understanding and generating natural language, their effectiveness on explicit manipulation and generation of linguistic structures remain understudied. In this paper, we investigate the task of generating new sentences preserving a given semantic structure, following the FrameNet formalism. We propose a framework to produce novel frame-semantically annotated sentences following an overgenerate-and-filter approach. Our results show that conditioning on rich, explicit semantic information tends to produce generations with high human acceptance, under both prompting and finetuning. Nevertheless, we discover that generated frame-semantic structured data is ineffective at training data augmentation for frame-semantic role labeling. Our study concludes that while generating high-quality, semantically rich data might be within reach, their downstream utility remains to be seen, highlighting the outstanding challenges with automating linguistic annotation tasks.

Paper Type: short

Research Area: Semantics: Sentence-level Semantics, Textual Inference and Other areas

Contribution Types: Approaches to low-resource settings, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: English