Scalable Multi-Robot 3D Mapping With Real-Time Collaborative Visualization

Ilya Nalivaiko, Mark Fiala, Martin Jaegersand

Keywords: robotics, 3D mapping, teleoperation, real-time systems, collaborative environments, remote sensing, multi-agent systems, human–robot interaction

TL;DR: A real-time framework that combines free-space carving from multiple robots into a shared 3D model, enabling simultaneous visualization by multiple remote operators.

Abstract: Teleoperation of mobile robots using live video is bandwidth-heavy and suffers from latency under real-world network conditions, reducing operator performance. To address this, we extend free-space carving (CARV) to a networked, multi-robot, multi-operator setting. Our system modifies CARV to export geometry in the glTF format and apply singular value decomposition (SVD) to keyframes for efficient, view-dependent texturing. A Unity-based central server aggregates updates from several robots via ZeroMQ and distributes the resulting 3D model to multiple clients using real-time protocols. On consumer hardware (Intel Core i7-12700H, 16 GB RAM, Wi-Fi), the framework supported 4 CARV instances and 4 simultaneous viewers without performance bottlenecks. Operators remain free to explore previously delivered geometry even when updates are delayed, avoiding the direct coupling of camera motion and network latency seen in traditional video teleoperation. This approach enables collaborative 3D mapping by teams of robots and operators, with promising applications in time-critical domains such as disaster response.

Submission Number: 34