Fusing Temporal Graphs into Transformers for Time-Sensitive Question Answering

Xin Su, Phillip Howard, Nagib Hakim, Steven Bethard

Submission Type: Regular Long Paper

Submission Track: Machine Learning for NLP

Submission Track 2: Question Answering

Keywords: Time-sensitive Question Answering, Temporal Graph Fusion, Temporal Reasoning

Abstract: Answering time-sensitive questions from long documents requires temporal reasoning over the times in questions and documents. An important open question is whether large language models can perform such reasoning solely using a provided text document, or whether they can benefit from additional temporal information extracted using other systems. We address this research question by applying existing temporal information extraction systems to construct temporal graphs of events, times, and temporal relations in questions and documents. We then investigate different approaches for fusing these graphs into Transformer models. Experimental results show that our proposed approach for fusing temporal graphs into input text substantially enhances the temporal reasoning capabilities of Transformer models with or without fine-tuning. Additionally, our proposed method outperforms various graph convolution-based approaches and establishes a new state-of-the-art performance on SituatedQA and three splits of TimeQA.

Submission Number: 1178