Path-Aware Cross-Attention Network for Question Answering

Ziye Luo, Ying Xiong, Buzhou Tang

Abstract: Reasoning is an essential ability in QA systems, and the integration of this ability into QA systems has been the subject of considerable research. A prevalent strategy involves incorporating domain knowledge graphs using Graph Neural Networks (GNNs) to augment the performance of pre-trained language models. However, this approach primarily focuses on individual nodes and fails to leverage the extensive relational information present within the graph fully. In this paper, we present a novel model called Path-Aware Cross-Attention Network (PCN), which incorporates meta-paths containing relational information into the model. The PCN features a multi-layered, bidirectional cross-attention mechanism that facilitates information exchange between the textual representation and the path representation at each layer. By integrating rich inference information into the language model and contextual semantic information into the path representation, this mechanism enhances the overall effectiveness of the model. Furthermore, we incorporate a self-learning mechanism for path scoring, enabling weighted evaluation. The performance of our model is assessed across three benchmark datasets, covering the domains of commonsense question answering (CommonsenseQA, OpenbookQA) and medical question answering (MedQA-USMLE). The experimental results validate the efficacy of our proposed model.