Enough Coin Flips Can Make LLMs Act Bayesian

ACL ARR 2025 February Submission6662 Authors

Abstract: Large language models (LLMs) exhibit the ability to generalize given few-shot examples in their input prompt, an emergent capability known as in-context learning (ICL). ICL allows models to generalize without explicit weight updates. Despite its empirical success, the underlying mechanism of ICL remains opaque. It is unclear whether LLMs perform structured reasoning akin to Bayesian inference or rely solely on pattern matching. In this work, we investigate the Bayesian nature of ICL in a controlled setting by analyzing LLMs’ ability to model biased coin flips. Our findings reveal several key insights: (1) LLMs often possess biased priors, leading to initial divergence in zero-shot settings, (2) in-context evidence outweighs explicit bias instructions provided in a prompt, (3) when updating beliefs, they broadly adhere to Bayesian posterior updates, with deviations stemming from miscalibrated priors rather than incorrect updates, and (4) attention magnitude has little impact on Bayesian inference.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: Large Language Models (LLMs), In-Context Learning (ICL), Bayesian Inference

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 6662