Multi-Robot Scene Completion: Towards Task-Agnostic Collaborative PerceptionDownload PDF

16 Jun 2022, 10:45 (modified: 16 Nov 2022, 06:13)CoRL 2022 PosterReaders: Everyone
Student First Author: yes
Keywords: Multi-Robot Perception, Scene Completion, Representation Learning
Abstract: Collaborative perception learns how to share information among multiple robots to perceive the environment better than individually done. Past research on this has been task-specific, such as detection or segmentation. Yet this leads to different information sharing for different tasks, hindering the large-scale deployment of collaborative perception. We propose the first task-agnostic collaborative perception paradigm that learns a single collaboration module in a self-supervised manner for different downstream tasks. This is done by a novel task termed multi-robot scene completion, where each robot learns to effectively share information for reconstructing a complete scene viewed by all robots. Moreover, we propose a spatiotemporal autoencoder (STAR) that amortizes over time the communication cost by spatial sub-sampling and temporal mixing. Extensive experiments validate our method's effectiveness on scene completion and collaborative perception in autonomous driving scenarios. Our code is available at
Supplementary Material: zip
18 Replies