Complete Situational Awareness through the Sensing Harmonization of Connected Vehicles and Smart Infrastructure

Zonglin Meng, Xin Xia, Jiaqi Ma

Keywords: Multi-agent Autonomy; Computer Vision; Perception;

Abstract:

As an instantiate of a multi-agent system, connected vehicles supported by smart infrastructure have been considered the next generation of road mobility and attract significant attention given their potential in terms of safety enhancement, fuel efficiency improvement, and environmental sustainability. As the core of connected vehicle technology, multi-agent perception is to achieve complete situational awareness of the complicated environment and serve as the foundation for collective intelligence. However, the effectiveness of multi-agent perception has been compromised in real-world scenarios due to the multi-agent heterogeneous feature extraction methods and the high communication cost. To bring connected vehicles onto real roads by addressing these fundamental challenges, this paper overcomes the heterogeneity of feature extraction and leverages the shared memory in a computation-and-communication-light method for enhanced situational awareness.
Drawing inspiration from human inference, our approach employs a memory-informed mechanism that uses an attention-driven memory module to capture multi-agent semantic interactions and motion dynamics from temporal data, thereby enhancing cooperative perception capabilities.
Extensive experiments conducted on various benchmark tasks show the superior scalability of our approach, particularly in addressing the fundamental problems of the multi-agent perception, thereby establishing its potential as a practical solution for resilient AI systems.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 11115