Retrieval-guided Cross-view Image Synthesis

hongji yang, Yiru Li, Yingying Zhu

Keywords: Cross-view Image Synthesis; Domain Gap; Semantic Segmentation Maps; Retrieval

Abstract:

Cross-view image synthesis task aims to synthesize a photo-realistic ground-view image in correspondence with the aerial image in another view or vice versa. However, the following limitations exist: 1) existing works require extra semantic segmentation maps or preprocessing modules to bridge the domain gap. 2) the current models focus only on shared semantics in the view transformation and ignore exclusive semantics, thus performing poorly in terms of image quality and realism. 3) cross-view image synthesis for urban areas is more difficult and challenging than that of existing datasets due to the complex surroundings and building textures,the two existing datasets,however, are primarily rural and suburban scenarios. With these challenges in mind, the findings of this study can be summarized as follows: 1) a novel retrieval-guided framework, which adopts a retrieval network as the embedder to reduce the domain gap. 2) a new generator, which enhances the semantic consistency and the diversity of exclusive semantics in the target view. 3) a new dataset (named VIGOR-GEN), which offers more practical cross-view image pairs in urban areas and enriches the cross-view datasets. Extensive experiments on CVUSA, CVACT and VIGOR-GEN benchmarks verify the effectiveness of our proposed method to synthesize the photo-realistic images from the given single image in another view, outperforming the existing state-of-the-art methods.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4187