Robust Natural Language Representation Learning for Natural Language Inference by Projecting Superficial Words out

Wanyun Cui, Guangyu Zheng, Wei Wang

TL;DR: We use neural networks to project superficial information out for natural language inference by defining and identifying the superficial information from the perspective of first-order logic.

Abstract: In natural language inference, the semantics of some words do not affect the inference. Such information is considered superficial and brings overfitting. How can we represent and discard such superficial information? In this paper, we use first order logic (FOL) - a classic technique from meaning representation language – to explain what information is superficial for a given sentence pair. Such explanation also suggests two inductive biases according to its properties. We proposed a neural network-based approach that utilizes the two inductive biases. We obtain substantial improvements over extensive experiments.

Keywords: natural language inference, first order logic