Reimagining ABM with LLM Agents via Shachi

So Kuroki, Yingtao Tian, Kou Misaki, Takashi Ikegami, Takuya Akiba, Yujin Tang

Submission Track: Paper Track (up to 8 pages)

Keywords: Agent-based modeling, Large language model, Social science

Abstract: Large language models (LLMs) are increasingly used as agents in agent-based modeling (ABM) to simulate complex social and economic behaviors. However, existing implementations often rely on minimal wrappers around LLMs, lacking structure, modularity, and support for reuse, hindering reproducibility, generalization and exploratory research. We introduce Shachi (鯱), a general and modular agent framework for building and evaluating LLM-driven agents in ABM. Shachi defines a standardized architecture comprising four core components (LLM, tools, memory, and configuration) and supports clear agent-environment and agent-agent interactions. To evaluate its effectiveness, we provide a benchmark suite of tasks spanning three levels of complexity, from single-agent behavior to multi-agent communication. Using Shachi, we reproduce prior LLM-based ABM studies, conduct systematic agent-task cross-evaluations, and demonstrate novel exploratory study results. Shachi enables scalable, reusable, and insightful research at the intersection of LLMs and ABM for social science, and is released as an open-source toolkit. Code: https://anonymous.4open.science/r/bench-F6E7

Submission Number: 15