MAPGD: Multi-Agent Prompt Gradient Descent for Collaborative Prompt Optimization

ACL ARR 2026 January Submission416 Authors

Keywords: Prompt Optimization, Multi-Agent Systems, Large Language Models, Gradient Descent

Abstract: Prompt engineering is crucial for fully leveraging large language models, yet existing prompt optimization methods often rely on a single refinement trajectory, leading to instability, conflicting update signals, and inefficient use of query budgets. We propose Multi-Agent Prompt Gradient Descent (MAPGD), a gradient-inspired framework that coordinates multiple complementary prompt editing signals for robust and interpretable optimization. MAPGD decomposes prompt refinement into orthogonal dimensions, such as instruction clarity, example selection, format, and style, and aggregates textual pseudo-gradients via semantic embedding, conflict-aware clustering, and adaptive fusion. To improve robustness, we introduce Hypersphere-Constrained Gradient Clustering, which enforces angular separation between conflicting directions, and Channel-Adaptive Agent Weighting, which dynamically calibrates editing contributions based on validation feedback. Experiments on diverse classification and reasoning benchmarks show that MAPGD achieves consistent gains in accuracy over strong baselines, while ablation results indicate that coordinated gradient fusion and adaptive weighting are critical to stable optimization. Together, these findings suggest that MAPGD offers a robust and interpretable framework for automatic prompt optimization.

Paper Type: Long

Research Area: AI/LLM Agents

Research Area Keywords: prompting, LLM/AI agents, optimization methods, robustness

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 416