MAFPO: A Multi-Agent Framework for Prompt Optimization in Complex Question Answering

Shuo Wang, Zeyu Wei, Guangjun Shi, Xiaohui Rong, Jianjun Yu

Abstract: Large language models (LLMs) have demonstrated strong capabilities in complex question answering (CQA) tasks. However, their performance largely depends on the quality of the prompts, which typically require extensive manual design and lack robustness to input variations. To address this limitation, we propose MAFPO, a framework designed to enhance both the performance and robustness of LLMs in CQA. MAFPO decomposes the prompt optimization process into four agents: Decomposer Agent transforms the input question into structured semantic triples; Recomposer Agent verifies semantic fidelity through back-translation and similarity assessment; Knowledge Collector Agent retrieves and refines relevant supporting evidence through multi-stage retrieval and summarization; and Prompt Optimization Agent employs simulated annealing to efficiently explore the discrete prompt space. Experiments on three standard CQA benchmarks (HotpotQA, 2WikiMultihopQA, and QASC) demonstrate that MAFPO consistently outperforms existing prompt optimization baselines under both zero-shot and knowledge-augmented settings. Ablation and robustness analyses further verify the contribution of each agent and highlight the framework’s resilience to diverse and noisy inputs.

External IDs:doi:10.1007/978-981-95-4091-4_1