You Make me Feel like a Natural Question: Training QA Systems on Transformed Trivia Questions

Anonymous

TL;DR: The zero-shot setting imagines a world without NQ: can we build a system that does similarly well as existing systems with our transformed probing questions?

Abstract: Training question answering (QA) and information retrieval systems for web queries requires large, expensive datasets that are difficult to annotate and time-consuming to gather. Moreover, while "natural" datasets of information-seeking questions are often prone to ambiguity or ill-formed, for many languages there are troves of freely available carefully crafted questions. Thus, we automatically generate shorter, information-seeking questions, resembling web queries in the style of the Natural Questions (NQ) dataset from longer trivia data. However, because not all of the generated questions are high quality or match the desired domain, we also use a classifier trained on linguistic, grammatical, style, and topic dependent feature to find questions that match traditional training data in style and topic. Training a QA system on these transformed questions is a viable strategy for alternate to more expensive training setups and contrast the final systems.

Paper Type: long

Research Area: Question Answering

Contribution Types: NLP engineering experiment, Approaches to low-resource settings, Data resources

Languages Studied: English