Miscalibrated Belief Updates in LLM Agents under Strategic Uncertainty

Harry Ilanyan, Krish Jain

Keywords: LLM agents, belief calibration, Bayesian updating, poker, strategic uncertainty, exploitability, mechanism design

TL;DR: LLM agents exhibit belief inertia in fixed-limit Texas Hold'em: updates fail to scale with Bayesian evidence and are miscalibrated, leading to exploitable play.

Abstract: LLM agents exhibit systematic failures in scaling belief updates with evidence strength in strategic environments. Using a heads-up poker environment with reference Bayesian oracles, we compare elicited LLM beliefs against a card-only posterior (combinatorial prior) and a strategy-aware posterior (Bayesian update incorporating opponent actions). Across 1,084 belief elicitations, Llama 3.1 70B beliefs remain closer to the card-only baseline than to the strategy-aware posterior ($\Delta = 0.014$ Jensen-Shannon distance, 95% CI $[0.011, 0.017]$). We show severe base-rate neglect: the model assigns 17% probability to "trash" hands versus the oracle's 66% ($\approx 4\times$ underweight). The model attempts to update beliefs, but updates are weakly coupled to the Bayesian signal ($r \approx 0.06$) and inflated in magnitude ($3$-$6\times$). These findings reveal that LLM agents exhibit belief inertia, applying near-fixed magnitude updates largely independent of evidence strength. This has implications for deployment in mechanism design contexts.

Track: Short Paper

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 71