Plausibility Processing in Transformers: Where is this ability coming from?

Anonymous

Abstract: Transformers are found to process semantic knowledge in a human-like way. However, it has not been examined where and how semantic knowledge is processed inside the model. This paper aims to deepen understanding of how Transformers preserve and process semantic knowledge, focusing on semantic knowledge of plausible relations between nouns and verbs. In particular, I investigate how knowledge of semantic plausibility is localized in Transformer models and how such localized components make causal contributions to Transformers' plausibility processing ability. A set of experiments showed that i) Transformers have attention heads that detect plausible relations between nouns and verbs, and that ii) they collectively contribute to the Transformer's ability to process plausibility, though each attention head makes a different amount of contribution.

Paper Type: long

Research Area: Interpretability and Analysis of Models for NLP