PACR: Progressively Ascending Confidence Reward for LLM Reasoning

ICLR 2026 Conference Submission19917 Authors

Keywords: Large Language Models, Reasoning, Process Reward

TL;DR: We propose Progressively Ascending Confidence Reward (PACR), a dense, model-intrinsic process reward computed directly from the model’s evolving belief in the correct answer for reasoning in LLMs.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has significantly improved LLM reasoning, but its sparse, outcome-based reward provides no guidance for intermediate steps, slowing exploration. We propose Progressively Ascending Confidence Reward (PACR), a dense, model-intrinsic reward computed directly from the model’s evolving belief in the correct answer. PACR encodes the inductive bias that, along a well-formed reasoning trajectory, the probability of the ground-truth answer should have a generally ascending trend. We provide empirical and theoretical analysis validating that such an inductive bias constrains the exploration search space to regions richer in logically sound reasoning. We demonstrate that PACR accelerates exploration, reaches reward saturation with fewer trajectories, and yields improvements on multiple benchmarks. Our results suggest that dense, model-intrinsic shaping signals can make RLVR training more effective and reliable. Code will be released.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 19917