DINO: A Conditional Energy-Based GAN for Domain Translation

Konstantinos Vougioukas, Stavros Petridis, Maja Pantic

Keywords: Generative Modelling, Domain Translation, Conditional GANs, Energy-Based GANs

Abstract: Domain translation is the process of transforming data from one domain to another while preserving the common semantics. Some of the most popular domain translation systems are based on conditional generative adversarial networks, which use source domain data to drive the generator and as an input to the discriminator. However, this approach does not enforce the preservation of shared semantics since the conditional input can often be ignored by the discriminator. We propose an alternative method for conditioning and present a new framework, where two networks are simultaneously trained, in a supervised manner, to perform domain translation in opposite directions. Our method is not only better at capturing the shared information between two domains but is more generic and can be applied to a broader range of problems. The proposed framework performs well even in challenging cross-modal translations, such as video-driven speech reconstruction, for which other systems struggle to maintain correspondence.

One-sentence Summary: A framework for domain translation which uses a novel mechanism for conditioning energy-based GANs.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Code: [ DinoMan/DINO](https://github.com/DinoMan/DINO)

Data: [CelebAMask-HQ](https://paperswithcode.com/dataset/celebamask-hq), [Cityscapes](https://paperswithcode.com/dataset/cityscapes)

Community Implementations: [ 4 code implementations](https://www.catalyzex.com/paper/arxiv:2102.09281/code)