Improved Adversarial Image CaptioningDownload PDF

27 Mar 2019, 19:59 (modified: 11 Jul 2022, 20:40)DeepGenStruct 2019Readers: Everyone
Keywords: image captioning, discrete GAN training
TL;DR: Image captioning as a conditional GAN training with novel architectures, also study two discrete GAN training methods.
Abstract: In this paper we study image captioning as a conditional GAN training, proposing both a context-aware LSTM captioner and co-attentive discriminator, which enforces semantic alignment between images and captions. We investigate the viability of two discrete GAN training methods: Self-critical Sequence Training (SCST) and Gumbel Straight-Through (ST) and demonstrate that SCST shows more stable gradient behavior and improved results over Gumbel ST.
3 Replies