SA-NLI: A Supervised Attention based framework for Natural Language Inference

Peiguang Li, Hongfeng Yu, Wenkai Zhang, Guangluan Xu, Xian Sun

Abstract: Natural Language Inference (NLI) aims to determine the relationship of a pair of sentences. As a critical component of NLI models, attention mechanism has been proven to be effective in the representation and interaction of sentences. However, the attention methods adopted in the existing NLI models are non-parametric or trained inside the model without explicit supervision, and the attention results are poorly explained. In this paper, we propose a Supervised Attention based Natural Language Inference (SA-NLI) framework to solve this problem. In our framework, the intra attention module is trained to focus on syntactically related tokens, while the inter attention module is constrained to capture alignment between sentences. Moreover, the supervised training of intra attention module and inter attention module are unified with the training of the NLI model by multi-task learning and transfer learning, respectively. We conduct extensive experiments on multiple NLI datasets, and the results demonstrate the effectiveness of our supervised attention based method. Further visual analysis validates the interpretability of attention results, and the extended experimental results indicate the generalization of our SA-NLI framework.