QUADRo: Dataset and Models for QUestion-Answer Database Retrieval
Submission Type: Regular Long Paper
Submission Track: Resources and Evaluation
Submission Track 2: NLP Applications
Keywords: question answering, semantic similarity, nlp application, question answering database, question answering resources, question ranking and retrieval
TL;DR: This paper proposes an effective paradigm for building Question Answering systems by using previously answered questions, achieving competitive performance with other QA methods, and providing data and models for future research.
Abstract: An effective approach to design automated Question Answering (QA) systems is to efficiently retrieve answers from pre-computed databases containing question/answer pairs.
One of the main challenges to this design is the lack of training/testing data. Existing resources are limited in size and topics and either do not consider answers (question-question similarity only) or their quality in the annotation process.
To fill this gap, we introduce a novel open-domain annotated resource to train and evaluate models for this task. The resource consists of 15,211 input questions. Each question is paired with 30 similar question/answer pairs, resulting in a total of 443,000 annotated examples. The binary label associated with each pair indicates the relevance with respect to the input question.
Furthermore, we report extensive experimentation to test the quality and properties of our resource with respect to various key aspects of QA systems, including answer relevance, training strategies, and models input configuration.
Submission Number: 4236
Loading