Integrating Categorical Semantics into Unsupervised Domain Translation

Samuel Lavoie-Marchildon, Faruk Ahmed, Aaron Courville

Keywords: Unsupervised Domain Translation, Unsupervised Learning, Image-to-Image Translation, Deep Learning, Representation Learning

Abstract: While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical semantic features could broaden its applicability. In particular, we demonstrate that categorical semantics improves the translation between perceptually different domains sharing multiple object categories. We propose a method to learn, in an unsupervised manner, categorical semantic features (such as object labels) that are invariant of the source and target domains. We show that conditioning the style encoder of unsupervised domain translation methods on the learned categorical semantics leads to a translation preserving the digits on MNIST$\leftrightarrow$SVHN and to a more realistic stylization on Sketches$\to$Reals.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: We present a method for learning domain invariant categorial semantics which enable UDT on two setups.

Code: [ lavoiems/Cats-UDT](https://github.com/lavoiems/Cats-UDT)

Data: [ImageNet](https://paperswithcode.com/dataset/imagenet)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2010.01262/code)