SuperCaptioning: Image Captioning Using Two-dimensional Word EmbeddingDownload PDF

May 28, 2019 (edited Jun 04, 2019)ICML 2019 Workshop Deep Phenomena Blind SubmissionReaders: Everyone
  • Keywords: Image Captioning, Two-dimensional Word Embedding, Transfer Learning
  • TL;DR: Image captioning using two-dimensional word embedding.
  • Abstract: Language and vision are processed as two different modal in current work for image captioning. However, recent work on Super Characters method shows the effectiveness of two-dimensional word embedding, which converts text classification problem into image classification problem. In this paper, we propose the SuperCaptioning method, which borrows the idea of two-dimensional word embedding from Super Characters method, and processes the information of language and vision together in one single CNN model. The experimental results on Flickr30k data shows the proposed method gives high quality image captions. An interactive demo is ready to show at the workshop.
1 Reply