Could ChatGPT Imagine: Content Control for Artistic Painting Generation Via Large Language Models

Yue Lu, Chao Guo, Yong Dou, Xingyuan Dai, Fei-Yue Wang

Abstract: Intelligent systems and human-machine interactions have consistently provided convenience in both work and daily life. Artificial Intelligence Generated Content (AIGC) can assist humans in artistic creation by generating painting images based on textual descriptions. However, the quality of generated painting images depends heavily on well-designed prompts, which are labor-intensive and time-consuming in painting creation. Large Language Models (LLMs) like ChatGPT have shown impressive performance in linguistic tasks such as question answering and logical inference, demonstrating strong linguistic intelligence. This paper proposes an assistant painting creation approach to provide precise content control for painting generation by combining LLMs with text-to-image generative models and evaluates the performance of the proposed approach on painting content generation and painting element arrangement. The experimental results show that our approach can provide clear guidance on rich painting content and reasonable arrangements of painting elements, demonstrating its ability of text-based painting scene imagination. In painting generation tasks, LLMs like ChatGPT can help the text-to-image models with precise control over the painting content and improve the overall painting results.