Go to ICLR 2026 Conference homepageCharacterising Overprecision in Black-Box LLMs: A Cognitive Science Inspired Framework
Keywords: LLMs; Uncertainty; Overconfidence
TL;DR: Characterising Overprecision in Black-Box LLMs
Abstract: Overconfidence in large language models (LLMs) has attracted growing attention due to its implications for the reliability of model outputs. Most existing approaches study verbalized confidence, where LLMs are asked to state their certainty, but such methods are prone to biases and hallucinations. Inspired by the cognitive science notion of overprecision—excessive certainty in narrow interval judgments—we propose a framework for evaluating overprecision in black-box LLMs. Our protocol comprises three phases: (1) generation, where models produce numerical confidence intervals under imposed confidence levels; (2) refinement, where intervals are adjusted via aggregation or self-refinement strategies; and (3) evaluation, where outcomes are assessed using calibration and correlation metrics adapted from cognitive science. Using datasets spanning general knowledge, medical, and financial domains, we find that: (i) LLMs are systematically miscalibrated, with large gaps between imposed confidence and actual coverage; (ii) interval lengths do not scale with requested confidence, showing limited responsiveness to explicit confidence instructions; (iii) calibration quality varies by task, domain, and answer scale, with finance and medicine posing greater challenges than general knowledge; and (iv) Refinement helps only when it trivially widens (union); reflective self-refinement tends to narrow and can worsen coverage. Taken together, these findings show that miscalibration persists across settings. This work is an exploratory, descriptive study: our goal is to characterize black-box LLM behavior under a fixed protocol, not to benchmark or optimize models for maximal performance.
Primary Area: applications to computer vision, audio, language, and other modalities
Submission Number: 7348
Loading