Implicit alignment and query refinement for RGB-T semantic segmentation

Chang Liu, Haizhuang Liu, Junbao Zhuo, Bochao Zou, Jiansheng Chen, Qianchuan Zhao, Huimin Ma

Abstract: Highlights•We propose a key deformable alignment block in the implicit spatial alignment module, which implicitly learns the mapping relationship between RGB-T images for better alignment and fusion without deformation field supervision.•We propose a dynamic query generation block in the query-based segmentation module to enhance semantic segmentation, which dynamically generates queries and guides the network to capture class-aware or instance-aware information accurately.•Experimental results show that our method achieves significant progress in segmentation accuracy. Besides, extensive ablation studies validate the effectiveness of the proposed modules.