Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPT2

Anonymous

Abstract: The goal of this paper is to enhance our understanding of how Transformer language models process semantic knowledge, especially regarding the plausibility of noun-verb relations. First, I demonstrate GPT2 exhibits a higher degree of similarity with humans in plausibility processing compared to other Transformer language models. Next, I delve into how knowledge of plausibility is contained within attention heads of GPT2 and how these heads causally contribute to GPT2's plausibility processing ability. Through several experiments, it was found that: i) GPT2 has a number of attention heads that detect plausible relationships between nouns and verbs; ii) these heads collectively contribute to the Transformer's ability to process plausibility, albeit to varying degrees; and iii) attention heads' individual performance in detecting plausible noun does not necessarily build a causal relation with GPT2's plausibility processing ability.

Paper Type: long

Research Area: Interpretability and Analysis of Models for NLP