S-Agent: an Agent Collaborative Framework Inspired by the Scientific Methodology

Anonymous

Abstract: An increasing number of advancements have been accomplished in agents empowered by Large Language Models (LLM), particularly in resolving simple dialogue tasks. However, existing agents still face intractable robustness issues for solving complex tasks, encountering the cascading hallucinations induced by multi-step invocations of LLM. Certain recent studies utilize multi-step reasoning, planning strategies, and domain workflows to improve the success rate of complex tasks, yet they neglect the scientific methodology that encompasses the accumulated wisdom derived from centuries of scientific inquiry. Drawing inspiration from the scientific methodology, we propose the S-Agent - an agent collaborative framework meticulously designed to actively experiment and refine theories based on the analysis of experimental results, thereby enhancing the deductive capabilities of LLMs and complementing their inductive and communicative strengths. Additionally, we introduce an innovative parallel planning methodology, wherein agents with identical roles collaborate to simultaneously address the same inquiry. Extensive experiments demonstrate the effectiveness and efficiency of our approach. Notably, we achieve a new state-of-the-art 96.3% pass@1 accuracy on the HumanEval coding benchmark with GPT-4.

Paper Type: long

Research Area: NLP Applications

Contribution Types: NLP engineering experiment

Languages Studied: English