AURORA: An Agentic Writing System Leveraging Rubric-Guided Reinforcement Learning

ACL ARR 2025 May Submission2278 Authors

Abstract: We present AURORA, a modular agentic system framework for automated academic survey generation and iterative refinement. At its core is Agentic Reinforcement Learning (ARL), where multiple reviewer agents evaluate drafts using a shared rubric, producing structured feedback that guides a fixed-policy refinement agent across successive revisions. The system comprises five coordinated components: citation preparation, knowledge base construction, outline generation, paper composition, and self-evaluation—each designed for modularity, reproducibility, and interoperability. To evaluate AURORA’s effectiveness as a survey generation system, we compare its outputs against two baselines: (1) ten recent (2023–2025) human-written survey papers across diverse domains from arXiv and peer-reviewed venues, and (2) outputs from state-of-the-art automatic survey generation approaches. Experimental results show that AURORA outperforms both, achieving an average rubric-aligned score of 92.48. This score is derived from a 100-point evaluation rubric grounded in professional peer-review standards, covering seven dimensions and twenty subcategories such as clarity, originality, relevance, and literature coverage. These findings validate the effectiveness of AURORA’s agentic refinement loop and rubric-as-reward framework in generating high-quality, transparent, and academically rigorous survey papers.

Paper Type: Long

Research Area: Generation

Research Area Keywords: automatic evaluation; few-shot generation; analysis; data-to-text generation; text-to-text generation; retrieval-augmented generation; interactive and collaborative generation;

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 2278