Go to ICML 2025 Conference homepageResearchTown: Simulator of Human Research Community
TL;DR: A graph-based multi-agent simulator for human research community
Abstract: Large Language Models (LLMs) have demonstrated remarkable potential in scientific domains, yet a fundamental question remains unanswered: Can we simulate human research communities with LLMs? Addressing this question can deepen our understanding of the processes behind idea brainstorming and inspire the automatic discovery of novel scientific insights. In this work, we propose ResearchTown, a multi-agent framework for research community simulation. Within this framework, the human research community is simplified as an agent-data graph, where researchers and papers are represented as agent-type and data-type nodes, respectively, and connected based on their collaboration relationships. We also introduce TextGNN, a text-based inference framework that models various research activities (e.g., paper reading, paper writing, and review writing) as special forms of a unified message-passing process on the agent-data graph. To evaluate the quality of the research community simulation, we present ResearchBench, a benchmark that uses a node-masking prediction task for scalable and objective assessment based on similarity. Our experiments reveal three key findings: (1) ResearchTown can provide a realistic simulation of collaborative research activities, including paper writing and review writing; (2) ResearchTown can maintain robust simulation with multiple researchers and diverse papers; (3) ResearchTown can generate interdisciplinary research ideas that potentially inspire pioneering research directions.
Lay Summary: Scientific discovery is often driven by collaboration—researchers reading, writing, and reviewing papers together. But can we simulate human research communities with LLMs?
We introduce ResearchTown, a simulation framework that represents the research community as a graph of agents (researchers) and data (papers). Each LLM-powered agent engages in key research activities—reading, writing, and reviewing papers—while collaborating with other agents. To formalize these interactions, we develop TextGNN, a text-based message-passing framework that captures all collaborative activities through a unified modeling approach.
To evaluate the quality of the simulation, we build a benchmark that measures whether the simulator can generate realistic research content. Our results show that ResearchTown can accurately model collaborative research workflows and even produce novel interdisciplinary ideas—for example, combining insights from natural language processing and criminology. This work opens up new opportunities to study how scientific ideas emerge and offers a path toward AI-assisted research innovation.
Link To Code: https://github.com/ulab-uiuc/research-town
Primary Area: Deep Learning->Graph Neural Networks
Keywords: Multi-agent; Graph Neural Network; Large Language Model; Automatic Research
Submission Number: 8966
Loading