Guiding Explanation-based NLI through Symbolic Inference Types

ACL ARR 2024 December Submission657 Authors

Abstract: In this work, we investigate the localised, quasi-symbolic inference behaviours in distributional representation spaces by focusing on the Explanation-based Natural Language Inference (NLI), exemplified by the syllogistic-deductive NLI, where two explanations (premises) are provided to derive a single conclusion. We first establish the connection between natural language and symbolic inferences by characterising quasi-symbolic NLI behaviours, named symbolic inference types. Next, we establish the theoretical connection between distributional and symbolic inferences by formalising the Transformer encoder-decoder NLI model as a latent variable model. We provide extensive experiments to reveal that the symbolic inference types can enhance model training and inference dynamics, and deliver localised, symbolic inference control. Based on these findings, we conjecture the different inference behaviours are encoded as functionally separated subspaces in latent parametric space, as the future direction to probe the composition and generalisation of symbolic inference behaviours in distributional representation spaces.

Paper Type: Long

Research Area: Semantics: Lexical and Sentence-Level

Research Area Keywords: Quasi-symbolic inference control; natural language inference; empirical evaluation to representation space

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 657