Scalable Urban Simulation for Generalizable Micromobility Robot Learning

Wayne Wu, Honglin He, Bolei Zhou

Keywords: Generalizable Robot Learning, Simulation, Micromobility

TL;DR: A simulator for scalable and generalizable robot learning in urban spaces.

Abstract: Micromobility, which utilizes lightweight mobile machines operating in urban public spaces, such as delivery robots and electric wheelchairs, emerges as a promising alternative to vehicular mobility. Current micromobility depends mostly on human manual operation (in-person or remote control), which raises safety and efficiency concerns when navigating busy urban environments full of unpredictable obstacles and pedestrians. Assisting humans with AI agents in maneuvering micromobility devices presents a viable solution for enhancing safety and efficiency. In this work, we present a scalable urban simulation solution to advance autonomous micromobility by supporting the development of generalizable agents capable of handling out-of-distribution (OOD) scenarios. First, we build *URBAN-SIM* -- a high-performance robot learning platform for large-scale training of embodied agents in interactive urban scenes. *URBAN-SIM* contains three critical modules: Hierarchical Urban Generation pipeline, Interactive Dynamics Generation strategy, and Asynchronous Scene Sampling scheme, to improve the diversity, realism, and efficiency of robot learning in simulation. Then, we propose *URBAN-BENCH* -- a suite of essential tasks and benchmarks to evaluate the robustness and generalization capabilities of AI agents in achieving autonomous micromobility. *URBAN-BENCH* includes eight tasks based on three core skills of the agents: Urban Locomotion, Urban Navigation, and Urban Traverse. We evaluate four robots with heterogeneous embodiments, such as wheeled and legged robots, across these tasks. Experiments demonstrate the potential of our platform for training agents that generalize to novel urban layouts, unseen obstacles, and long-horizon decision-making.

Submission Number: 2