EvoFSM: Controllable Self-Evolution for Deep Research with Finite State Machines

ACL ARR 2026 January Submission805 Authors

Keywords: DeepResearch, LLM-based Agents, Finite State Machines, Self-Evolution

Abstract: While LLM-based agents have shown promise for deep research, most existing approaches rely on fixed workflows that struggle to adapt to real-world, open-ended queries. Recent work therefore explores self-evolution by allowing agents to rewrite their own code or prompts to improve problem-solving ability, but unconstrained optimization often triggers instability, hallucinations, and instruction drift. We propose EvoFSM, a structured self-evolving framework that achieves both adaptability and control by evolving an explicit Finite State Machine (FSM) instead of relying on free-form rewriting. EvoFSM decouples the optimization space into macroscopic Flow (state-transition logic) and microscopic Skill (state-specific behaviors), enabling targeted improvements under clear behavioral boundaries. Guided by a critic mechanism, EvoFSM refines the FSM through a small set of constrained operations, and further incorporates a self-evolving memory that distills successful trajectories as reusable priors and failure patterns as constraints for future queries. Extensive evaluations on five multi-hop QA benchmarks demonstrate the effectiveness of EvoFSM. In particular, EvoFSM reaches 58.0\% accuracy on the DeepSearch benchmark. Additional results on interactive decision-making tasks further validate its generalization.

Paper Type: Long

Research Area: AI/LLM Agents

Research Area Keywords: LLM/AI agents; open-domain QA

Languages Studied: English, Chinese

Submission Number: 805