SFFR: Spatial-Frequency Feature Reconstruction for Multispectral Aerial Object Detection

Xin Zuo, Chenyu Qu, Haibo Zhan, Jifeng Shen, Wankou Yang

Abstract: Recent multispectral object detection methods have primarily focused on spatial-domain feature fusion based on CNNs or Transformers, while the potential of frequency-domain feature remains underexplored. In this work, we propose a novel spatial and frequency feature reconstruction (SFFR) method, which leverages the spatial–frequency feature representation mechanisms of the Kolmogorov–Arnold network (KAN) to reconstruct complementary representations in both spatial and frequency domains prior to feature fusion. The core components of SFFR are the proposed frequency component exchange KAN (FCEKAN) module and multiscale Gaussian KAN (MSGKAN) module. The FCEKAN introduces an innovative selective frequency component exchange strategy that effectively enhances the complementarity and consistency of cross-modal features based on the frequency feature of RGB and IR images. The MSGKAN module demonstrates excellent nonlinear feature modeling capability in the spatial domain. By leveraging multiscale Gaussian basis functions, it effectively captures the feature variations caused by scale changes at different uncrewed aerial vehicle (UAV) flight altitudes, significantly enhancing the model’s adaptability and robustness to scale variations. It is experimentally validated that our proposed FCEKAN and MSGKAN modules are complementary and can effectively capture the frequency and spatial semantic features, respectively, for better feature fusion. Extensive experiments on the SeaDroneSee, DroneVehicle, and DVTOD datasets demonstrate the superior performance and significant advantages of the proposed method in the UAV multispectral object perception task. Code will be available at https://github.com/qchenyu1027/SFFR

External IDs:dblp:journals/tgrs/ZuoQZSY25