ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
Abstract: Scientific Research, vital for improving human life, is hindered by its inherent complexity, slow pace, and the need for specialized experts. To enhance its productivity, we propose a ResearchAgent, a Large Language Model (LLM)-powered research idea generation agent, which automatically defines problems, proposes methods and designs experiments, while iteratively refining them based on the feedback from LLM-powered reviewing agents. Specifically, starting with a core scientific paper as the primary focus to generate ideas, our ResearchAgent is augmented not only with relevant publications by connecting information over an academic graph but also entities retrieved from an entity-centric knowledge store based on their shared underlying concepts, mined across numerous papers. Then, mimicking the human approach to iteratively improving ideas with peer discussions, we leverage multiple ReviewingAgents that provide reviews and feedback via iterative revision processes. These reviewing agents are instantiated with human preference-aligned LLMs whose criteria for evaluation are elicited from actual human judgments via LLM prompting. We experimentally validate our ResearchAgent on scientific publications across multiple disciplines, showing its effectiveness in generating novel, clear, and valid ideas based on human and model-based evaluation results.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: knowledge-augmented methods, retrieval-augmented generation, retrieval-augmented models
Contribution Types: NLP engineering experiment
Languages Studied: English
Section 2 Permission To Publish Peer Reviewers Content Agreement: Authors grant permission for ACL to publish peer reviewers' content
Submission Number: 53
Loading