G-Refine: A General Refiner for Text-to-Image Generation

Chunyi Li, Haoning Wu, Hongkun Hao, Zicheng Zhang, Tengchuan Kou, Chaofeng Chen, Xiaohong Liu, LEI BAI, Weisi Lin, Guangtao Zhai

Abstract: With the evolution of Text-to-Image (T2I) models, the quality defects of AI-Generated Images (AIGIs) pose a significant barrier to their widespread adoption. In terms of both perception and alignment, existing models cannot always guarantee high-quality results. To mitigate this limitation, we introduce G-Refine, a general image quality refiner designed to enhance low-quality images without compromising the integrity of high-quality ones. The model is composed of three interconnected modules: a perception quality indicator, an alignment quality indicator, and a general quality enhancement module. Based on the mechanisms of the Human Visual System (HVS) and syntax trees, the first two indicators can respectively identify the perception and alignment deficiencies, and the last module can apply targeted quality enhancement accordingly. Extensive experimentation reveals that when compared to alternative optimization methods, AIGIs after G-Refine outperform in 10+ quality metrics across 4 datasets. This improvement significantly contributes to the practical application of contemporary T2I models, paving the way for their broader adoption.

Primary Subject Area: [Experience] Interactions and Quality of Experience

Secondary Subject Area: [Generation] Generative Multimedia

Relevance To Conference:

This work is a typical application of multimedia field technology in generative AI, which includes image and text modalities.

For image quality, we perform Image quality assessment (IQA), an important issue for multimedia technology. For text quality, we conduct dependency syntactic parsing for the prompt.

By using both qualities as an indicator, the G-Refine proposed in this work can effectively optimize the generation results of Text-to-Image (T2I) models. The G-Refine can enhance low-quality AI-Generated Images (AIGIs) while avoiding the negative optimization of high-quality ones. Under the conference theme "Multimedia in the Generative AI Era", this work can effectively drive the application of contemporary mainstream T2I models.

Submission Number: 2786