AlphaGAN: Fully Differentiable Architecture Searchfor Generative Adversarial Networks

Anonymous

Keywords: Generative models, AutoML, Gradient-based

Abstract: Generative Adversarial Networks (GANs) are formulated as minimax game problems, whereby generators attempt to approach real data distributions by virtue of adversarial learning against discriminators. In this work, we aim to boost model learning from the perspective of network architectures, by incorporating recent progress on automated architecture search into GANs. To this end, we propose a fully differentiable search framework for generative adversarial networks, dubbed {\em alphaGAN}. The searching process is formalized as solving a bi-level minimax optimization problem, where the outer-level objective aims for seeking a suitable network architecture towards pure Nash Equilibrium conditioned on the network parameters optimized in the inner level. The entire optimization performs a first-order method by alternately optimizing the two-level objective in a fully differentiable manner, enabling architecture search to be completed in an enormous search space. Extensive experiments on CIFAR-10 and STL-10 datasets show that our algorithm can obtain high-performing architectures only with $3$-GPU hours on a single GPU in the search space comprised of approximate $2\times 10^{11}$ possible configurations.