Once-More: Continuous Self-Correction for Large Language Models via Perplexity-Guided Intervention

ICLR 2026 Conference Submission21625 Authors

Keywords: Large Language Model, Natural Language Processing, Self-Correction, Agent, Guided Generation, Post-Hoc Refinement

TL;DR: Once-More is a training-free framework that prevents LLM errors from compounding by using token-level perplexity and verifier feedback to enable inference-time self-correction via logit redistribution.

Abstract: Large Language Models (LLMs) often experience compounding errors during long text generation. Early mistakes can propagate and lead to drift, faulty reasoning, or repetition. While scaling up models improves capabilities, it requires substantial computational resources, and the resulting self-correction behaviour remains unpredictable at inference time. Self-correction is a promising technique for addressing this issue. However, existing approaches have limitations. Supervised training methods can build self-correcting behaviours into models, but require training data collection and lack cross-domain generalizability. Current post-hoc iterative refinement methods operate only at inference time, but must wait for substantial portions of the draft to be generated before providing feedback. This feedback does not guarantee effective guidance, and the same mistake patterns can still reappear. In this paper, we introduce Once-More, a model-agnostic post-hoc self-correction framework that intervenes during generation. Once-More leverages token-level perplexity and feedback from verifiers to provide continuous guided steering of the generation path through a logit redistribution mechanism. This approach essentially helps accumulate "more correct" steps throughout the generation process. Evaluation on multiple benchmarks demonstrates that Once-More achieves state-of-the-art results compared to other self-correction methods. To our knowledge, Once-More is the first post-hoc method to leverage token perplexity and external feedback to perform continuous guided self-correction.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 21625