Lower dimensional kernels for video discriminators

Emmanuel Kahembwe, Subramanian Ramamoorthy

Abstract: Highlights • We provide an empirical analysis of previous video GAN discriminators. • We show that the loss hessian deteriorates with kernel dimensionality. • We present a set of guiding principles for video discriminator design. • We present a discriminator architecture that improves the performance of video GANs. • We set state-of-the-art results for single GPU video generation. • We present the first video GAN results at resolutions of up to 512 × 512. Abstract This work presents an analysis of the discriminators used in Generative Adversarial Networks (GANs) for Video. We show that unconstrained video discriminator architectures induce a loss surface with high curvature which make optimization difficult. We also show that this curvature becomes more extreme as the maximal kernel dimension of video discriminators increases. With these observations in hand, we propose a methodology for the design of a family of efficient Lower-Dimensional Video Discriminators for GANs (LDVD-GANs). The proposed methodology improves the performance and efficiency of video GAN models it is applied to and demonstrates good performance on complex and diverse datasets such as UCF-101. In particular, we show that LDVDs can double the performance of Temporal-GANs and provide for state-of-the-art performance on a single GPU using the proposed methodology.