Stylized Handwriting Generation of Arbitrary Structures and OOV Expressions: A Decoupled Approach via Layout-Offsets

Ji Gan, Baijin Li, Yan-Ming Zhang, Jiaxu Leng, Weiqiang Wang, Xinbo Gao

Keywords: Handwriting Generation, Handwritten Mathematical Expressions.

Abstract: To truly understand human handwriting, machines must not only recognize glyphs but also generate them. However, most existing approaches are limited to synthesizing isolated characters or handwritten texts of linear sequences, whereas the stylized synthesis of handwriting with arbitrary layout structures remains largely underexplored, such as handwritten mathematical expression generation (HMEG). Existing approaches have failed to address such cases, as it is challenging to simultaneously generate complex layout structures and imitate calligraphic styles, especially for out-of-vocabulary (OOV) expressions. Inspired by how humans write, where layout structuring and glyph shaping are inherently separated, we therefore propose a glyph-layout decoupled paradigm for stylized HMEG. To better facilitate the generation of arbitrary layout structures, we leverage printed layouts as strong prior guidance and propose generating layout offsets instead of absolute positions. To achieve stylized glyph-layout synthesis, we further incorporate implicit context adaptation via cross-attention to jointly mimic structured layouts and calligraphic glyphs from reference examples. By treating reference layouts and glyphs as external implicit contexts, our model selectively attends to relevant stylistic features of each symbol and its bounding box. Experiments demonstrate that our method outperforms previous SoTA approaches in terms of visual quality, semantic and structural correctness, and style consistency for stylized HMEG.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 2935