A Novel Question-Context Interaction Method for Machine Reading Comprehension

Tuan-Anh Phan, Hoang Ngo, Khac-Hoai Nam Bui

Abstract: Machine reading comprehension (MRC) is a challenging NLP task that requires machines to model the complex interactions between questions and specific contexts. In Question-Answering (QA) tasks, most existing works rely on the powerful encoder of pre-trained language models (PrLM) in order to represent word/subword embeddings for extracting the answer. In this study, we present a novel method for enriching the context representation by exploiting the question-context interaction at the sentence level. In particular, we introduce the sentence-based question-context interaction (S-QCI) block, which combines two main layers such as the question-aware layer and the cross-sentence layer, to represent the sentence embedding of the context. The sentence information is then used to enrich question information for the context representation at the word level. The main idea is that the word units in the sentence, which have a high attention score of question-sentence interaction, can be enriched with more question information for the final output of the extractive-span MRC task. The experiment on NewsQA, a benchmark dataset in this research field, indicates that the proposed method has significant improvements compared with the baselines using PrLM and achieves new state-of-the-art results.