Prompting to Prompt: Meta-Template Learning for Transferable Prompt Optimization

Xinyu Zhang, Deyang Li, Minjun Zhao, Meng Han, Ruifeng Shi, Zhicheng Dou

Keywords: Prompt optimization, Meta-Template, Transferability

Abstract: Prompt optimization plays a key role in fully leveraging the capabilities of large language models (LLMs). Despite their respective advantages, offline, online, and hybrid prompt optimization methods all suffer from limited transferability and strong reliance on task-specific data. To systematically resolve these limitations, we propose Prompting to Prompt (PTP), a novel framework for optimizing meta-templates, inspired by the idea of learning to learn. PTP introduces meta-templates as structured intermediate representations that decompose prompts into transferable elements, enabling generalization across diverse task. PTP employs a bi-level optimization process: the inner loop that refines prompt elements for individual samples using gradient feedback and element list guidance, and the outer loop that captures transferable structural patterns by comparing element-level changes before and after inner-loop updates. Instead of learning task-specific features, the outer loop generalizes structural knowledge across tasks, continuously updating meta-template structures and selection strategies. This enables PTP to unify offline and online prompt optimization, supporting task-level and query-level prompt generation without retraining. Extensive experiments on six benchmark datasets show that PTP consistently outperforms state-of-the-art baselines, achieving up to 10.52-point gains on challenging tasks like Arena-hard. These results demonstrate that PTP offers a promising solution for more transferable and efficient prompt optimization.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 6482