Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet

ICLR 2026 Conference Submission19973 Authors

Keywords: test-time scaling, reasoning models, factuality hallucination

TL;DR: Increasing test-time computation does not consistently improve factual accuracy or reduce hallucinations in current reasoning models.

Abstract: Test-time scaling increases inference-time computation by allowing models to generate long reasoning chains, and has shown strong performance across many domains. However, in this work, we show that this approach is not yet effective for knowledge-intensive tasks, where high factual accuracy and low hallucination rates are essential. We conduct a comprehensive evaluation of test-time scaling using 14 reasoning models on two knowledge-intensive benchmarks. Our results reveal that increasing test-time computation does not consistently improve accuracy and, in many cases, it even leads to more hallucinations. We then analyze how extended reasoning affects hallucination behavior. We find that reduced hallucinations often result from the model choosing to abstain after thinking more, rather than from improved factual recall. Conversely, for some models, longer reasoning encourages attempts on previously unanswered questions, many of which result in hallucinations. Case studies show that extended reasoning can induce confirmation bias, leading to overconfident hallucinations. Despite these limitations, we observe that compared to non-thinking, enabling thinking remains beneficial.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 19973