The Case for a Single Model that can Both Generate Continuations and Fill-in-the-Blank

Anonymous

Paper Link: https://openreview.net/forum?id=F9NzZm3zqyu

Paper Type: Short paper (up to four pages of content + unlimited references and appendices)

Abstract: The task of inserting text into a specified position in a passage, known as fill in the blank (FitB), is useful for a variety of applications where writers interact with a natural language generation (NLG) system to craft text. While previous work has tackled this problem with models trained specifically to do fill in the blank, a more useful model is one that can effectively perform _both_ FitB and continuation tasks. In this work, we evaluate the feasibility of using a single model to do both tasks. We show that models pre-trained with a FitB-style objective are capable of both tasks, while models pre-trained for continuation are not. Finally, we show how these models can be easily finetuned to allow for fine-grained control over the length and word choice of the generation.

Presentation Mode: This paper will be presented in person in Seattle

Copyright Consent Signature (type Name Or NA If Not Transferrable): Daphne Ippolito

Copyright Consent Name And Address: Google, 1600 Amphitheatre Pkwy, Mountain View, CA 94043