Content Disentanglement for Semantically Consistent Synthetic-to-Real Domain Adaptation

Mert Keser, Artem Savkin, Federico Tombari

Abstract: Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. However, deep learning perception algorithms trained solely on synthetic data encounter serious performance drops when they are tested on real data. Such performance drops are commonly attributed to the domain gap between real and synthetic data. Domain adaptation methods that have been applied to mitigate the aforementioned domain gap achieve visually appealing results, but usually introduce semantic inconsistencies into the translated samples. In this work, we propose a novel, unsupervised, end-to-end domain adaptation network architecture that enables semantically consistent sim2real image transfer. Our method performs content disentanglement by employing shared content encoder and fixed style code.