Chinese Character Image Inpainting with Skeleton Extraction and Adversarial Learning

Di Sun, Tingting Yang, Xiangyu Pan, Jiahao Wang, Gang Pan

Abstract: Chinese character image inpainting aims to restore the missing textual regions with realistic contents. Existing algorithms for text image inpainting are primarily designed for English characters, however, their performance is suboptimal when applied to Chinese characters. The primary challenge in Chinese character image inpainting lies in the scarcity of open-source datasets for this task. Additionally, conventional image inpainting algorithms fail to account for the guiding significance of the stroke topology structure inherent in Chinese characters during the inpainting process. In this paper, we propose a skeleton extraction algorithm based on line thinning, and contribute a dataset of Chinese character images and their skeleton images accordingly. In particular, we propose a skeleton extraction guided generative framework skeletonGAN for Chinese character inpainting, where the skeleton of Chinese characters is used as prior knowledge to guide the inpainting process. The whole framework comprises two parts: an SE network for skeleton-based Chinese character skeleton extraction and inpainting, and an SR network dedicated to Chinese character image inpainting. Experimental results demonstrate that the proposed method successfully fills the missing character information and achieves significant image inpainting results.