Toward Cooperative 3D Object Reconstruction with Multi-agent

Xiong Li, Zhenyu Wen, Leiqiang Zhou, Chenwei Li, Yejian Zhou, Taotao Li, Zhen Hong

Abstract: We study the problem of object reconstruction in a multi-agent collaboration scenario. Specifically, we focus on the reconstruction of specific goals through several cooperative agents equipped with vision sensors to achieve higher efficiency than single agents. Our main insight is that a complete 3D object can be split into several local 3D models and assigned to different agents. In addition, we can use the salient characteristics of the collaboration agent itself to help realize the integration of local models. We develop a novel pipeline that first restores local 3D models from the images obtained from different agents, then the relative poses between collaborative agents are estimated by aligning intrinsic features. After that, all local models are integrated using the estimated parameters. Extensive experiments show that our proposed method is capable of accurately reconstructing 3D objects in the real world in a multi-agent collaborative manner. The full reconstruction pipeline is released to the public as an open-source project.