CIMGEN: Controlled Satellite Image Manipulation by Finetuning Pretrained Generative Models on Limited Data

Chandrakanth Gudavalli, Erik Rosten, Lakshmanan Nataraj, Shivkumar Chandrasekaran, B. S. Manjunath

Abstract: Content creation and image editing can significantly benefit from flexible user controls. A common interpretable low-dimensional representation of an image is its semantic map, that has information about the objects present in the image. When compared to raw RGB pixels, the modification of semantic map to insert or remove objects is much easier, especially for satellite images as the satellite images are typically associated with an underlying semantic map. One can take a semantic map and easily modify it to selectively insert, remove, or replace objects in the map. The method proposed in this paper takes in the modified map of a given geographic area and alters corresponding satellite image to reflect the changes made to the map. We achieve this with traditional pre-trained image-to-image translation GANs like CycleGAN or Pix2Pix GAN, by fine-tuning them on a limited dataset of reference images associated with semantic maps. We discuss the qualitative and quantitative performance of our technique highlighting its potential for applications in satellite imagery manipulation. We also demonstrate how this method can effectively challenge numerous deep learning-based image forensic techniques, emphasizing the urgent need for robust and generalizable image forensic tools to combat the spread of manipulated data.