Learning Entailment-Based Sentence Embeddings from Natural Language Inference

Rabeeh Karimi Mahabadi*, Florian Mai*, James Henderson

TL;DR: We propose a natural language inference model whose interaction layer imposes a direct interpretation of the induced sentence embeddings in terms of entailment and contradiction.

Abstract:

Large datasets on natural language inference are a potentially valuable resource for inducing semantic representations of natural language sentences. But in many such models the embeddings computed by the sentence encoder goes through an MLP-based interaction layer before predicting its label, and thus some of the information about textual entailment is encoded in the interpretation of sentence embeddings given by this parameterised MLP.
In this work we propose a simple interaction layer based on predefined entailment and contradiction scores applied directly to the sentence embeddings. This parameter-free interaction model achieves results on natural language inference competitive with MLP-based models, demonstrating that the trained sentence embeddings directly represent the information needed for textual entailment, and the inductive bias of this model leads to better generalisation to other related datasets.

Keywords: sentence embeddings, textual entailment, natural language inference, interpretability