Go to ICLR 2026 Conference homepageProRefine: Inference-time Prompt Refinement with Textual Feedback
Keywords: prompt optimization, inference-time, multi llm workflows, large language models, textual feedback
TL;DR: ProRefine, an inference-time method using LLM agent loops to iteratively refine prompts with textual feedback, no extra training or labels. Improves multi-step reasoning by optimizing prompts; helps smaller LLM match larger LLM while reducing cost.
Abstract: Agentic workflows, where multiple LLM based agents collaborate to accomplish complex tasks like reasoning or planning, play a substantial role in many cutting-edge commercial applications, and continue to fascinate researchers across fields for their potential to accomplish expensive, complex tasks that, until recently, only humans have been trusted to do. These workflows critically depend on the prompts used to provide the roles models play in such workflows. Poorly designed prompts that fail even slightly to guide individual agents can lead to sub-optimal performance that may snowball within a system of agents, limiting their reliability and scalability. To address this important problem of inference-time prompt optimization, we introduce ProRefine, an innovative inference-time optimization method that uses an agentic loop of LLMs to generate and apply textual feedback. ProRefine dynamically refines prompts for multi-step reasoning tasks without additional training or ground truth labels. Evaluated on five benchmark mathematical reasoning datasets, ProRefine significantly surpasses zero-shot Chain-of-Thought baselines by 3 to 37 percentage points. This approach not only boosts accuracy but also allows smaller models to approach the performance of their larger counterparts. This highlights its potential for building more cost-effective and powerful hybrid AI systems, thereby democratizing access to high-performing AI.
Primary Area: foundation or frontier models, including LLMs
Submission Number: 22087
Loading