Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation

ACL ARR 2025 July Submission858 Authors

Abstract: Inference-time scaling via repeated sampling has shown promise in reasoning tasks, but its effectiveness in multilingual generation remains underexplored. We evaluate this approach using perplexity- and reward-based verifiers on two multilingual benchmarks: the Aya Evaluation Suite and m-ArenaHard. Our results show consistent quality improvements, with gains exceeding 35% in some cases. While perplexity-based scoring is effective for open-ended prompts, only reward-based verifiers improve performance on tasks requiring reasoning (e.g., math, code). Our results demonstrate the broader utility of repeated sampling for multilingual text generation and underscore the importance of selecting right verifiers for the task.

Paper Type: Short

Research Area: Multilingualism and Cross-Lingual NLP

Research Area Keywords: multilingual evaluation, multilingualism

Contribution Types: NLP engineering experiment, Approaches to low-resource settings

Languages Studied: Arabic, English, Hindi, Punjabi, Portuguese, Russian, Telugu, Turkish, Chinese, Vietnamese, Czech, Japanese

Submission Number: 858