Research Town: Simulator of Research Community
Keywords: multi-agent simulation; automatic research; large language model
Abstract: Large Language Models (LLMs) have demonstrated remarkable potential in scientific domains, yet a fundamental question remains unanswered: Can we simulate human research communities using LLMs? Addressing this question could deepen our understanding of the processes behind research idea generation and inspire the automatic discovery of novel scientific insights. In this work, we propose ResearchTown, a multi-agent framework for simulating research communities. Within this framework, the real-world research community is simplified and modeled as an agent-data graph (i.e. community graphs), where researchers and papers are represented as agent-type and data-type nodes, respectively. We also introduce TextGNN, a text-based inference framework that models diverse research activities (e.g., paper reading, paper writing, and review writing) as specific forms of a generalized message-passing process on the agent-data graph. To evaluate the quality of research simulation, we present ResearchBench, a benchmark that uses a node-masking prediction task for scalable and objective assessment. Our experiments reveal three key findings: (1) ResearchTown effectively simulates collaborative research activities by accurately predicting the attribute of masked nodes in the graph; (2) the simulation process in ResearchTown uncovers insights, like not every author contributes equally to the final paper, which is aligned with real-world research communities; (3) ResearchTown has the potential to foster interdisciplinary research by generating reasonable paper ideas that span across domains.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 11145
Loading