MSPN: Multiple Semantics Perception Network for Remote Sensing Change Captioning

Anonymous

Abstract: Remote sensing images usually cover a large surface area, so the change information is usually difficult to be precisely localized. Especially, some changes are easy to be overlooked due to their inconspicuous locations and fuzzy shapes. In addition, unlike the natural image change description task, the remote sensing image change description task aims to capture the most significant changes without various influencing factors, such as light, seasonal influences and complex land cover. To address the above challenges, in this paper, we propose a multiple semantic perception network (MSPN) model to extract more accurate feature representations to guide the decoder in generating high-quality change descriptions. In the visual encoder stage, the global efficient semantic awareness module is designed for global feature embedding, the self-semantic awareness module digs deep into the internal connections between features, and the change semantic interaction module effectively distinguishes semantic changes from irrelevant ones. In the description generation phase, the Transformer-based decoder is designed to guide the change description generation. Extensive experiments on the LEVIR-CC dataset demonstrate the superiority of the MSPN model over many state-of-the-art techniques.

Paper Type: long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Contribution Types: Model analysis & interpretability, Data analysis

Languages Studied: Python