Geometric-Aware Control in Diffusion Model for Handwritten Chinese Font Generation

Gang Yao, Kemeng Zhao, Chengyu Deng, Ning Ding, Tianqi Zhao, Yao Tao, Liangrui Peng

Abstract: With the progress of artificial intelligence generated content (AIGC) technologies, style-controlled image generation methods based on diffusion models have shown promising performance on different tasks including Chinese font generation. However, most of the current diffusion model-based Chinese font generation methods focus on generating printed Chinese character images and pay insufficient attention to the geometric characteristics of character images with variations in handwriting style. This paper investigates incorporating geometric-aware control into diffusion models to generate target character images with new content templates and given style references. First, a deformable attention mechanism is utilized in the content aggregation process to adapt to variations in handwritten character structure. Second, the edge contour of the new content template is incorporated into the style control process. Third, the geometric information such as the corner points of a target character is utilized to weight the image reconstruction loss function for better control of character shape. The effectiveness of the proposed method for handwritten Chinese font generation is validated on the CASIA-HWDB 1.0/1.1 Chinese handwriting dataset.