Eliciting Sophisticated Agentic RAG Skills via Automated Data Synthesis

ACL ARR 2026 January Submission8897 Authors

Keywords: RAG, Agent, Data synthesis

Abstract: Agentic Retrieval-Augmented Generation (RAG) empowers large language models to autonomously plan and retrieve information for complex problem-solving. However, the development of robust agents is hindered by the scarcity of high-quality training data that reflects the noise and complexity of real-world retrieval environments. Conventional manual annotation is unscalable and often fails to capture the dynamic reasoning strategies required to handle retrieval failures. To bridge this gap, we introduce RAGShaper, a novel data synthesis framework designed to automate the construction of RAG tasks and robust agent trajectories. RAGShaper incorporates an Information Curator to build dense information trees enriched with adversarial distractors spanning Perception and Cognition levels. Furthermore, we propose a constrained navigation strategy that forces a teacher agent to confront these distractors, thereby eliciting trajectories that explicitly demonstrate error correction and noise rejection. Comprehensive experiments confirm that models trained on our synthesized corpus significantly outperform existing baselines, exhibiting superior robustness in noise-intensive and complex retrieval tasks.

Paper Type: Long

Research Area: Information Extraction and Retrieval

Research Area Keywords: Information Retrieval and Text Mining,AI / LLM Agents

Contribution Types: NLP engineering experiment, Data resources

Languages Studied: English

Submission Number: 8897