On the Role of Receptive Field inUnsupervised Sim-to-Real Image Translation

Nikita Jaipuria, Shubh Gupta, Praveen Narayanan, Vidya N. Muralu

Abstract: Generative Adversarial Networks (GANs) are now widely used for photo-realisticimage synthesis. In applications where a simulated image needs to be translatedinto a realistic image (sim-to-real), GANs trained on unpaired data from the twodomains are susceptible to failure in semantic content retention as the image istranslated from one domain to the other. This failure mode is more pronounced incases where the real data lacks content diversity, resulting in a contentmismatchbetween the two domains - a situation often encountered in real-world deployment.In this paper, we investigate the role of the discriminator’s receptive field in GANsfor unsupervised image-to-image translation with mismatched data, and study itseffect on semantic content retention. Experiments with the discriminator architec-ture of a state-of-the-art coupled Variational Auto-Encoder (VAE) - GAN model ondiverse, mismatched datasets show that the discriminator receptive field is directlycorrelated with semantic content discrepancy of the generated image.