From Pixels to Tokens: Otsu Thresholding for Accurate yet Confident Decoding

ACL ARR 2026 January Submission3744 Authors

Keywords: Probabilistic Decoding, Otsu's Method, Uncertainty Evaluation

Abstract: This paper introduces a new decoding approach, Otsu thresholding, inspired by the well-known image processing technique, which computes a threshold between low-confidence and high-confidence tokens for each generation step. The method is tested across several tasks and instruction-based models, revealing results competitive to other state-of-the-art decoding methods, while keeping output quality. Also, it is shown that tuning specific characteristics of the model, such as the pass from a softmax function of the distribution and the limitation to K highest candidates before selecting a token, can highly affect model performance. In general, this approach is a great alternative method that offers a good balance between quality, confidence, uncertainty, and diversity.

Paper Type: Long

Research Area: Natural Language Generation

Research Area Keywords: Automatic Evaluation, Text-to-Text Generation, Inference Methods

Languages Studied: English

Submission Number: 3744