Decomposing Prompts: Discovering Reusable Scaffolds and Task-Specific Residuals

Yupeng Chang, Yi Chang, Yuan Wu

Keywords: Prompt Optimization, Compositionality, Large Language Models, Instructional Scaffolds, Knowledge Reuse.

TL;DR: We decompose prompts into a reusable scaffold and a task-specific residual to make prompt optimization more efficient and reliable.

Abstract: Traditional automatic prompt optimization methods treat prompts as monolithic text blocks, necessitating costly, from-scratch optimization for each new task and precluding knowledge reuse across related tasks. We revisit this monolithic paradigm by proposing an approach where prompts are decomposed into two components: a reusable, domain-invariant **Instructional Scaffold** that captures high-level task structure, and a concise **Task-specific Residual** for fine-grained adaptation. We introduce **DiSR** (**D**iscovering **i**nstructional **S**caffolds and **R**esiduals), a two-stage algorithm inspired by the Minimum Description Length (MDL) principle, which motivates decomposing prompts into reusable components. This approach systematically discovers the components, allowing for efficient knowledge reuse and task-specific adaptation. In contrast to existing methods like OPRO and AutoPrompt, DiSR enables task-level knowledge reuse, improving performance and reliability across different tasks. Extensive experiments demonstrate that DiSR not only achieves competitive accuracy but also generates better-calibrated confidence estimates, especially on large models and in professional domains. This compositional hypothesis is validated through semantic analysis, revealing that optimized prompts form distinct, scaffold-centered clusters in their embedding space. Our findings establish a compositional view of prompt engineering, facilitating more robust, interpretable, and reusable optimization strategies.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 4230