Automatic Transcription of Handwritten Old Occitan Language

Esteban Garces Arias, Vallari Pai, Matthias Schöffel, Christian Heumann, Matthias Aßenmacher

Submission Type: Regular Long Paper

Submission Track: Multilinguality and Linguistic Diversity

Submission Track 2: Computational Social Science and Cultural Analytics

Keywords: handwritten text recognition, low-resource languages, transformer, computer vision, natural language processing

TL;DR: we achieve state-of-the-art performance on the transcription of handwritten old occitan text

Abstract: While existing neural network-based approaches have shown promising results in Handwritten Text Recognition (HTR) for high-resource languages and standardized/machine-written text, their application to low-resource languages often presents challenges, resulting in reduced effectiveness. In this paper, we propose an innovative HTR approach that leverages the Transformer architecture for recognizing handwritten Old Occitan language. Given the limited availability of data, which comprises only word pairs of graphical variants and lemmas, we develop and rely on elaborate data augmentation techniques for both text and image data. Our model combines a custom-trained Swin image encoder with a BERT text decoder, which we pre-train using a large-scale augmented synthetic data set and fine-tune on the small human-labeled data set. Experimental results reveal that our approach surpasses the performance of current state-of-the-art models for Old Occitan HTR, including open-source Transformer-based models such as a fine-tuned TrOCR and commercial applications like Google Cloud Vision. To nurture further research and development, we make our models, data sets, and code publicly available.

Submission Number: 1575