The Impact of Visual Information in Chinese Characters: Evaluating Large Models’ Ability to Recognize and Utilize Radicals
Abstract: The glyphic writing system of Chinese incorporates information-rich visual features below the character level, such as radicals that provide hints about meaning or pronunciation.
However, there has been no investigation into whether contemporary Large Language Models (LLMs) and Vision-Language Models (VLMs) can harness these features in Chinese language processing (CLP).
In this study, we establish a benchmark to evaluate LLMs' and VLMs' understanding of Chinese characters' visual elements, namely radicals, composition structures, strokes, and stroke counts. Our results reveal that models exhibit some, but limited, knowledge of the visual information, regardless of whether images of characters are provided.
To investigate models' ability of using radicals, we further experiment whether incorporating radicals into prompts is beneficial for LLMs in language understanding tasks. Our experiments indicate that models possess knowledge in utilizing radicals to a certain extent. For example, we observe consistent improvement in POS tagging after providing correct radicals.
Paper Type: Long
Research Area: Phonology, Morphology and Word Segmentation
Research Area Keywords: Morphology
Contribution Types: Data resources, Data analysis
Languages Studied: Chinese
Submission Number: 306
Loading