Work-in-Progress: Towards Real-time Collaborative 3D Object Detection Systems with Request-free Communication

Shiqi Sun, Yantao Lu, Ning Liu, Bo Jiang, Jinchao Chen, Ying Zhang

Abstract: Collaborative 3D object detection by sharing features among agents significantly enhances performance compared to single-agent detection. However, directly sharing full-sized features introduces a large communication bandwidth load. To address this challenge, existing collaborative methods adopt a request-response framework, where the ego agent sends a request, and collaborative agents respond with only the necessary parts of the features after analyzing the request. However, the frequent communication in this request-response cycle impacts real-time system performance in real-world environments by increasing overall processing time and raising the risk of message loss and communication delays. To address this challenge and enable real-time system implementation, we propose a request-free collaborative 3D object detection framework that eliminates the request-response cycle through a novel request-free response generator, named Position and Occlusion Response Generator (PORG). PORG consists of two specialized components, Position-aware Mask Generator (PaMG) and Occlusion-aware Feature Mask Generator (OaMG), which use attention mechanisms to generate the necessary response features without the request from the ego agent. To evaluate the efficiency of our proposed PORG, we conducted evaluations on both public datasets and real-world settings. We provide system implementation for both the request-response and request-free frameworks on Jetson Orin Series embedded devices, and extensive evaluation shows that PORG outperforms the baselines, achieving higher Average Precision (AP) with lower communication bandwidth in public datasets and superior real-time performance on embedded devices.