Identifying and Analyzing Task-Encoding Tokens in Large Language Models

Yu Bai, Heyan Huang, Cesare Spinoso-Di Piano, Sanxing Chen, Marc-Antoine Rondeau, Yang Gao, Jackie CK Cheung

Keywords: Task-encoding tokens, In-context learning, Large language models, Interpretability and analysis

Abstract: In-context learning (ICL) has emerged as an effective solution for few-shot learning with large language models (LLMs). Previous research suggests that LLMs perform ICL by analogizing from the provided demonstrations, similar to how humans learn new tasks. However, how LLMs leverage demonstrations to specify a task and learn a corresponding computational function through ICL remains underexplored. Drawing from the way humans learn from content-label mappings in demonstrations, we categorize the tokens in an ICL prompt into content, stopword, and template tokens, with the latter two typically ignored by humans due to their uninformative nature. Our goal is to identify the type of tokens whose representations highly and directly influence LLM's performance, a property we refer to as **task-encoding**. By ablating representations from the attention of the test example, we find that the representations of informative content tokens have less influence on performance, while template and stopword tokens are more prone to be task-encoding tokens, which contrasts with the human attention to informative words. We further give evidence about the function of task-encoding tokens by showing that their representations aggregate information from the content tokens. Moreover, we demonstrate experimentally that lexical meaning, repetition, and structural cues are the main distinguishing characteristics of these tokens. Our work sheds light on how LLMs learn to perform tasks from demonstrations and deepens our understanding of the roles different types of tokens play in LLMs.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7037