Sample-Aware Dual Actions for Prompt Optimization

Yufeng Zhou, Jiuzheng Wang, Dongliang Xue, Wenpeng Hu, Xiaoying Bai, Guotong Geng

Keywords: LLM, Prompt Optimization

Abstract: In recent years, large language models (LLMs) have achieved remarkable progress in reasoning, question answering, and decision-making tasks in natural language processing. High-quality prompts play a crucial role in guiding LLMs to generate outputs that meet expectations. However, manually designing effective prompts for specific tasks is often time-consuming and heavily reliant on expertise, limiting the scalability and efficiency of model applications. Consequently, automated prompt optimization has become an important direction for enhancing LLM performance. To address this, we propose a sample-aware dual actions Monte Carlo Tree Search (MCTS) framework for automated prompt optimization, enabling the search process to leverage sample performance for more effective optimization. This method not only efficiently utilizes training samples to guide prompt improvement but also directs the optimization trajectory based on the overall state of the training samples. We validate our framework on the Big-Bench Hard (BBH) and MMLU datasets, and experimental results demonstrate that it outperforms traditional prompt optimization methods and recent baselines in both accuracy and optimization stability.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 24506