Interventional Probing in High Dimensions: An NLI Case Study

Anonymous

Keywords: NLI, Interpretability, Semantics, Probing, Interventional Probing, Explainability

Abstract: Probing strategies have been shown to detect semantic features intermediate to certain fragments of NLI.In the case of natural logic, the relation between these features and the entailment label is explicitly known: as such, this provides a ripe setting for interventional studieson the NLI models' representations, allowing for stronger causal conjectures and a deeper critical analysis interventional probing methods.In this work, we carry out new and existing vector-level interventions to investigatethe effect of these semantic features on NLI classification: we perform amnesic probing (which forgets features as directed by learned probes)and introduce the mnestic probing variation (which forgets all dimensions except the probe-selected ones).Furthermore, we delve into the limitations of these methods and outline pitfalls thathave been obscuring the effectivity of such studies.

Paper Type: long

Research Area: Semantics: Sentence-level Semantics, Textual Inference and Other areas