Virtual Community: A Generative Social World for Embodied AI

Qinhong Zhou, Hongxin Zhang, Yutian Chen, Zheyuan Zhang, Xiangye Lin, Jincheng Yang, Lixing Fang, Jiageng Liu, Xinyu Sun, Chuang Gan

Keywords: embodied AI

Abstract: We present Virtual Community, a social world simulation platform designed to support embodied AI research, featuring large-scale community scenarios derived from the real world. Virtual Community introduces two key features to enrich the virtual social world with generative AI: scalable 3D Scene creation, which supports the generation of expansive outdoor and indoor environments at any location and scale, addressing the lack of a large-scale, interactive, open-world scene for embodied AI research; and embodied agents with grounded characters and social relationship networks, the first to simulate socially connected agents at a community level, that also have scene-grounded characters. We design two novel challenges to showcase that Virtual Community provides testbeds to evaluate the social reasoning and planning capabilities of embodied agents in open-world scenarios: Route Planning and Election Campaign. The Route Planning task examines the agent's ability to reason about time, location, and tools in the community to plan fast and economical commutes in daily life. The Election Campaign task evaluates an agent's ability to explore and connect with other agents as a new member of the community. We evaluate several baseline agents on these challenges and demonstrate the performance gap of current methods in addressing embodied social challenges within open-world scenarios, which our simulator is designed to unlock. We plan to open-source this simulation and hope Virtual Community can accelerate the development in this direction.

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1942