SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering

Chenguang Zhu, Michael Zeng, Xuedong Huang

Keywords: Machine Reading Comprehension, Conversational Question Answering

TL;DR: A neural method for conversational question answering with attention mechanism and a novel usage of BERT as contextual embedder

Abstract: Conversational question answering (CQA) is a novel QA task that requires the understanding of dialogue context. Different from traditional single-turn machine reading comprehension (MRC), CQA is a comprehensive task comprised of passage reading, coreference resolution, and contextual understanding. In this paper, we propose an innovative contextualized attention-based deep neural network, SDNet, to fuse context into traditional MRC models. Our model leverages both inter-attention and self-attention to comprehend the conversation and passage. Furthermore, we demonstrate a novel method to integrate the BERT contextual model as a sub-module in our network. Empirical results show the effectiveness of SDNet. On the CoQA leaderboard, it outperforms the previous best model's F1 score by 1.6%. Our ensemble model further improves the F1 score by 2.7%.

Code: https://www.dropbox.com/s/5fiabkrgc8gtr10/SDNet.zip?dl=0

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/arxiv:1812.03593/code)