CLT-Det: Correlation Learning Based on Transformer for Detecting Dense Objects in Remote Sensing Images

Yong Zhou, Silin Chen, Jiaqi Zhao, Rui Yao, Yong Xue, Abdulmotaleb El-Saddik

Abstract: Challenges still exist in the task of object detection in remote sensing images with densely distributed objects due to large variation in scale and neglect of the relative position and correlation. To address these issues, a correlation learning detector based on transformer (CLT-Det) is proposed for detecting dense objects in remote sensing images. A transformer attention module (TAM) is designed to improve the densely packed objects’ model representation ability by learning pixelwise attention with a transformer. To alleviate the semantic gap caused by the variations in scale, a feature refinement module (FRM) is proposed by improving the multiscale feature pyramid. A correlation transformer module (CTM) is proposed to extract correlation information and it encodes position information of dense objects’ features on the classification branch for fully using the position information and correlation among objects. Extensive experiments compared with several state-of-art methods on two challenging remote sensing datasets, namely, dataset for object detection in aerial images (DOTA) and HRSC2016, demonstrate that the proposed CLT-Det achieves promising and competitive performance.