GPT-2 Small Fine-Tuned on Logical Reasoning Summarizes Information on Punctuation Tokens

Sonakshi Chauhan, Atticus Geiger

Keywords: Interpretability, Causality, Reasoning

TL;DR: GPT-2 Small Fine-Tuned on Logical Reasoning Summarizes Information on Punctuation Tokens

Abstract: How is information stored and aggregated within a language model performing inference? Preliminary evidence suggests that representations of punctuation tokens might serve as ``summary points'' for information about preceding text. We add to this body of evidence by demonstrating that GPT-2 small fine-tuned on the RuleTaker logical inference dataset aggregates crucial information about rules and sentences above period tokens.

Email Of Author Nominated As Reviewer: [e-mail]

Submission Number: 11