The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Compute

ICLR 2026 Conference Submission17681 Authors

Keywords: sequential reasoning, test-time scaling, inverse-entropy voting, uncertainty-aware aggregation, chain-of-thought, matched compute, reasoning benchmarks, LLMs

TL;DR: At matched compute, sequential refinement with inverse-entropy voting beats parallel self-consistency across strong LLMs/benchmarks, making uncertainty-aware sequential scaling a better default for test-time reasoning.

Abstract: We revisit test-time scaling for language model reasoning and ask a fundamental question: at equal token budget and compute, is it better to run multiple independent chains in parallel, or to run fewer chains that iteratively refine through sequential steps? Through comprehensive evaluation across 5 state-of-the-art open source models and 3 challenging reasoning benchmarks, we find that sequential scaling where chains explicitly build upon previous attempts consistently outperforms the dominant parallel self-consistency paradigm in 95.6% of configurations with gains in accuracy upto 46.7%. Further, we introduce inverse-entropy weighted voting, a novel training-free method to further boost the accuracy of sequential scaling. By weighing answers in proportion to the inverse entropy of their reasoning chains, we increase our success rate over parallel majority and establish it as the optimal test-time scaling strategy. Our findings fundamentally challenge the parallel reasoning orthodoxy that has dominated test-time scaling since Wang et al.’s self-consistency decoding (Wang et al., 2022), positioning sequential refinement as the robust default for modern LLM reasoning and necessitating a paradigm shift in how we approach inference-time optimization.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 17681