Few-Shot Idea Auto-Generation: Reasoning Over Idea Representations to Predict New Research Ideas

ICLR 2026 Conference Submission17889 Authors

Keywords: LLM agent, AI Scientist, Idea generation, idea verification

TL;DR: We present an LLM-agent framework that, given only a few seed papers, represents their core contributions, cross-pollinates them to auto-generate practical research ideas, and quantitatively rates each idea’s novelty and relevance.

Abstract:

Large language models have demonstrated powerful reasoning capabilities on user-provided contexts, inspiring researchers to explore their potential for automated research. A critical component of research is idea generation—identifying novel contributions, advantages, and distinctions from existing work. However, we show that naively prompting pre-trained LLMs to generate research ideas produces largely meaningless results.

We introduce a novel task: few-shot idea auto-generation, where models generate research ideas based on a small set of existing papers. Our key insight is that meaningful ideas typically build upon prior work rather than emerging from scratch—for instance, adapting solutions from one domain to address similar challenges in another, often combined with novel algorithmic approaches. To enable effective few-shot idea generation, we address three fundamental challenges: (1) How can we effectively represent the core ideas of existing papers? (2) How can we generate practical, implementable ideas while filtering out infeasible ones? (3) How can we validate the generated ideas effectively?

Our contributions are threefold. First, we develop an idea representation method that effectively captures papers' core contributions through multi-agent extraction with synopsis and procedural profiling. Second, we design an LLM-agent-based generation framework that performs cross-pollination via systematic gap-bridging between paper pairs. Third, we propose an evaluation methodology using semantic similarity analysis with recency-weighted novelty scoring and construct a benchmark for few-shot idea generation across 3,353 papers from 8 computer science domains.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 17889