OBI CHARiot: Full-page OBI Rubbing Segmentation with Dual Data Flywheels

Xingjian Hu, LiZhong Zhang, Baole Wei, Yiyang Zhao, Jianhua Zhu, Liangcai Gao

Keywords: Oracle Bone Inscriptions, Data Flywheels, Instance Segmentation, Semantic Segmentation

TL;DR: A two-stage training framework to address data misalignment and scarcity in full-page OBI rubbing segmentation sequentially

Abstract:

Oracle Bone Inscriptions (OBI), one of the earliest mature writing systems globally, serve as a crucial carrier of human civilization.
However, directly segmenting OBI characters from full-page rubbings remains unexplored, primarily due to the scarcity of high-quality annotated data.
To address this challenge, we propose a two-stage training framework, named OBI CHARiot, which establishes a cycle of mutual improvement between model performance and data quality. In the first stage, a data flywheel mechanism is employed to iteratively train a SAM2 while automatically aligning existing low-quality annotations, rather than directly utilizing the model's initial predictions. In the second stage, we employ an iterative strategy on a large collection of unlabeled rubbings, integrating automatic annotation with continuous model refinement.
For reliable evaluation, we invite domain experts to annotate 2,226 rubbings, resulting a test set OBIMDTest.
Experimental results demonstrate that OBI CHARiot offers advantages in both model performance and data quality. Specifically, training SAM2 with our framework yields a remarkable 14.99% gain in mask AP$_{50}$ over the baseline trained on raw data. Similarly, various off-the-shelf instance segmentation models exhibit improved performance when trained on data processed by OBI CHARiot. Moreover, the characters segmented by our framework yield a 22.75% improvement in top-1 accuracy on the downstream deciphering task. These findings confirm the significant potential of OBI CHARiot to advance OBI research. To support future studies, our model and the processed data will be made publicly available.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 8880