Generative adversarial network based on semantic consistency for text-to-image generation

Yue Ma, Li Liu, Huaxiang Zhang, Chunjing Wang, Zekang Wang

Abstract: Although text-to-image generation technology has made significant progress in visually realistic images, the generated images cannot be completely consistent with the texts. In this paper, a novel generative adversarial network based on semantic consistency is proposed to generate semantically consistent and realistic images according to text descriptions. The proposed method explores the semantic consistency between text and image for an efficient cross-modal generation that combines image generation and semantic correlation. A generation network with a hybrid attention is utilized to generate different resolution images, which improves the authenticity of the generated images. In addition, a semantic comparison module is presented to map the texts and the generated images to the same semantic space for comparison through consistency refinement and information classification. Extensive experiments on public benchmark datasets demonstrate that the proposed method outperforms the comparative methods.