Sparse-Gated RGB-Event Fusion for Small Object Detection in the Wild

Yangsi Shi, Miao Li, Nuo Chen, Yihang Luo, Shiman He, Wei An

Abstract: Detecting small moving objects under challenging lighting conditions, such as overexposure and underexposure, remains a critical challenge in computer vision applications including surveillance, autonomous driving, and anti-UAV systems. Traditional RGB-based detectors often suffer from degraded object visibility and highly dynamic illumination, leading to suboptimal performance. To address these limitations, we propose a novel RGB-Event fusion framework that leverages the complementary strengths of RGB and event modalities for enhanced small object detection. Specifically, we introduce a Temporal Multi-Scale Attention Fusion (TMAF) module to encode motion cues from event streams at multiple temporal scales, thereby enhancing the saliency of small object features. Furthermore, we design a Sparse Noisy Gated Attention Fusion (SNGAF) module, inspired by the mixture-of-experts paradigm, which employs a sparse gating mechanism to adaptively combine multiple fusion experts based on input characteristics, enabling flexible and robust RGB-Event feature integration. Additionally, we present RGBE-UAV, which is a new RGB-Event dataset tailored for small moving object detection under diverse exposure conditions. Extensive experiments on our RGBE-UAV and public DSEC-MOD datasets demonstrate that our method outperforms existing state-of-the-art RGB-Event fusion approaches, validating its effectiveness and generalization under complex lighting conditions.

External IDs:doi:10.3390/rs17173112