Bayesian scaling laws for in-context learning

Aryaman Arora, Dan Jurafsky, Christopher Potts, Noah Goodman

Keywords: in-context learning, scaling laws, Bayesian inference, many-shot jailbreaking

Abstract:

In-context learning (ICL) is a powerful technique for getting language models to perform complex tasks with no training updates.
Prior work has established strong correlations between the number of in-context examples provided and the accuracy of the model's predictions.
In this paper, we seek to explain this correlation by showing that ICL approximates a Bayesian learner. This perspective gives rise to a family of novel Bayesian scaling laws for ICL.
In experiments with GPT-2 models of different sizes, our scaling laws match existing scaling laws in accuracy while also offering interpretable terms for task priors, learning efficiency, and per-example probabilities.
To illustrate the analytic power that such interpretable scaling laws provide, we report on controlled synthetic dataset experiments designed to inform real-world studies of safety alignment. In our experimental protocol, we use SFT to
suppress an unwanted existing model capability and then use ICL to try to bring that capability back (many-shot jailbreaking). We then experiment on real-world instruction-tuned LLMs using capabilities benchmarks as well as a new many-shot jailbreaking dataset.
In all cases, Bayesian scaling laws accurately predict the conditions under which ICL will cause the suppressed behavior to reemerge, which sheds light on the ineffectiveness of post-training at increasing LLM safety.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11708