DEval: Enhancing Large Language Models' Decoding with Evaluation for Better Generation

Ziqin Luo, Jinyi Han, Haokun Zhao, Dixuan Wang, Guochao Jiang, Jiaqing Liang, Deqing Yang

Abstract: Existing Large Language Models (LLMs) generate texts through unidirectional autoregressive decoding methods to respond to various queries. These methods tend to consider token selection in a simple sequential manner, making it easy to fall into suboptimal options when encountering uncertain tokens, referred to as chaotic points in this paper. Many chaotic points exist in the texts generated by LLMs, and they often significantly affect the quality of subsequently generated tokens, which would interfere with LLMs’ generation. In this paper, we propose DEval, a decoding framework for enhancing LLMs’ generation. Analogous to human decision-making, DEval integrates speculation and evaluation into decoding, allowing LLMs to make more careful decisions, and thus optimize token selection at chaotic points. Experiments across various tasks using different LLMs demonstrate DEval’s effectiveness. The code and more technical details are available at our repository1.

External IDs:dblp:conf/ijcnn/LuoHZWJLY25