A multifaceted benchmarking of GAN architectures on generating synthetic satellite imagery

Kevin Wells, Felipe A. Lopes, Vasit Sagan, Flavio Esposito

Abstract: Machine learning models require a large labeled dataset to build robust algorithms that can be generalized over various classification or regression applications. Generative adversarial networks (GANs) have demonstrated their effectiveness in generating realistic labeled images across various domains. However, the choice of GAN architecture can significantly impact the suitability of the generated imagery for specific tasks. Although there are quality metrics (e.g., Fréchet inception distance) to evaluate the synthetic images generated from GAN models, this work argues that such metrics should be combined to provide a fair performance comparison of GAN models. This paper presents a multifaceted benchmark methodology and compares three state-of-the-art GAN architectures, i.e., DCGAN, WGAN, and PRGAN, for synthesizing imagery from a widely-used open dataset comprising satellite images of icebergs. Our evaluation methodology encompasses a comparison between synthetic and real images, employing probability metrics such as correlation and intersection. Then, a regression analysis was performed to examine the fidelity of spectral profiles. The findings reveal that DCGAN appears to be the most effective GAN architecture for generating synthetic satellite imagery, considering the analyzed architectures and dataset. It consistently produced reliable images that closely resemble real satellite images, rendering them visually indistinguishable. Each evaluated GAN architecture generates synthetic satellite imagery with distinct characteristics, encompassing varying shapes, spectral profiles, and pixel value distributions. Our evaluation approach and results contribute to a deeper understanding of their utility in this context.

External IDs:dblp:conf/aipr/WellsLSE23