Contextual Inference Feature Extraction Approach Based on Generative Adversarial Network for SAR-To-Optical Image Translation

Jinjin Luo, Xuezhi Yang, Hongbo Liang, Guan Wu

Abstract: Conditional generative adversarial networks (cGANs) have dominated the research of synthetic aperture radar (SAR)-to-optical (S2O) image translation, attributing to the feature generalization ability of residual convolutional blocks. Nevertheless, the fixed geometric structures of convolutional kernels hinder the feature inference from local to global, resulting in unclear contours and missing details in the generated image. To address this challenge, we proposed a contextual inference transformer block in the generator, dubbed CoIT. It enables the network to capture key features in SAR images through context awareness, providing a more comprehensive feature representation from local details to global structure. The proposed CoIT block contextually encodes input keys to establish the relationship between SAR images and optical images, improving the quality of generated images. Experiments on the public datasets WHU-SEN-City and SEN12MS show that the proposed method not only achieves better visual effects but also makes certain progress in evaluation indicators.