Hyperparameter-Free-Sampling: Entropy Equilibrium for Text Generation

ACL ARR 2026 January Submission1139 Authors

Keywords: large language model, sampling method, natural language processing

Abstract:

Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), a hyperparameter-free approach inspired by information theory that can dynamically adjust candidate sets by balancing normalized entropy with probability mass. We evaluate EES on both reasoning and generation tasks across a range of model architectures. Our results show that EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity. By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance.
Code is released at \url{https://anonymous.4open.science/r/Entropy-Equilibrium-Sampling-B196}.

Paper Type: Long

Research Area: Natural Language Generation

Research Area Keywords: human evaluation, automatic evaluation, inference methods

Contribution Types: NLP engineering experiment, Theory

Languages Studied: English

Submission Number: 1139