Conditional multichannel generative adversarial networks with an application to traffic signs representation learning

Farzin Ghorban, Narges Milani, Daniel Schugk, Lutz Roese-Koerner, Yu Su, Dennis Müller, Anton Kummert

Abstract: Generative adversarial networks (GANs) are known to produce photorealistic representations. However, we show in this study that this is only valid when the input channels come from a regular RGB camera sensor. In order to alleviate this shortcoming, we propose a general solution to which we refer to as multichannel GANs (MCGANs). In contrast to the existing approaches, MCGANs can process multiple channels with different textures and resolutions. This is achieved by using known concepts in deep learning such as weight sharing and specially separated convolutions. The proposed pipeline enables particular kernels to learn low-level characteristics from the different channels without the need for exhaustive hyper-parameter tuning. We demonstrate the improved representational ability of the framework on traffic sign samples that are captured by a camera with a so-called red-clear-clear-clear pixel topology. Furthermore, we extend our solution by applying the concept of conditions, that offers a whole spectrum of new features, especially for the generation of traffic signs. Throughout this paper, we further discuss relevant applications for the generated synthetic data.