Evidence Extraction for Machine Reading Comprehension

Hai Wang

Abstract:

Remarkable success has been achieved in the last few years on some limited machine read- ing comprehension (MRC) tasks. However, it is still difficult to interpret the predictions of existing MRC models. In this paper, we fo- cus on extracting evidence sentences that can explain or support the answers of multiple- choice MRC tasks, where the majority of an- swer options cannot be directly extracted from reference documents.
Due to the lack of ground truth evidence sen- tence labels in most cases, we apply distant supervision to generate imperfect labels and then use them to train an evidence sentence extractor. To denoise the noisy labels, we apply a recently proposed deep probabilistic logic learning framework to incorporate both sentence-level and cross-sentence linguistic indicators for indirect supervision. We feed the extracted evidence sentences into exist- ing MRC models and evaluate the end-to-end performance on three challenging multiple- choice MRC datasets: MultiRC, RACE, and DREAM, achieving comparable or better per- formance than the same models that take as in- put the full reference document. To the best of our knowledge, this is the first work extracting evidence sentences for multiple-choice MRC.