TKBert: Two-Stage Knowledge Infused Behavioral Fine-Tuned BERT

Jiahao Chen, Zheyu He, Yujin Zhu, Liang Xu

Abstract: Machine Reading Comprehension (MRC) is a technique to make machine answer questions corresponding to the given passage. However, the existing MRC systems confront challenges to deal with the complex real-world fine-grained question answering tasks. To address the issue, we propose a novel method named TKB\(^2\)ert that generates prior knowledge data and utilizes it in behavioral fine-tuning at the first stage and trains itself through the dual attentive streams at the second stage. Subsequently, we take part in the MRC track of 2021 Language and Intelligence Challenge and win the first place. The result on the newly-built fine-grained MRC competition dataset validates the effectiveness of TKB\(^2\)ert.