In-Context Learning Creates Task Vectors

Roee Hendel, Mor Geva, Amir Globerson

Submission Type: Regular Short Paper

Submission Track: Interpretability, Interactivity, and Analysis of Models for NLP

Submission Track 2: Language Modeling and Analysis of Language Models

Keywords: Large Language Models, In-Context Learning, Interpretability

Abstract:

In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood.
In particular, it is challenging to map it to the "standard" machine learning framework, where one uses a training set $S$ to find a best-fitting function $f(x)$ in some hypothesis class. Here we make progress on this problem by showing that the functions learned by ICL often have a very simple structure: they correspond to the transformer LLM whose only inputs are the query $x$ and a single "task vector" calculated from the training set. Thus, ICL can be seen as compressing $S$ into a single task vector $\boldsymbol{\theta}(S)$ and then using this task vector to modulate the transformer to produce the output.
We support the above claim via comprehensive experiments across a range of models and tasks.

Submission Number: 5764