FSTs vs ICL: Generalisation in LLMs for an under-resourced language

ACL ARR 2025 February Submission4783 Authors

Abstract: LLMs have been widely adopted to tackle many traditional NLP tasks. Their effectiveness remains uncertain in scenarios where pre-trained models have limited prior knowledge of a language. In this work, we examine LLMs' generalization in under-resourced settings through the task of orthographic normalization across Otomi language variants. We develop two approaches: a rule-based method using a finite-state transducer (FST) and an in-context learning (ICL) method that provides the model with string transduction examples. We compare the performance of FSTs and neural approaches in low-resource scenarios, providing insights into their potential and limitations. Our results show that while FSTs outperform LLMs in zero-shot settings, ICL enables LLMs to surpass FSTs, stressing the importance of combining linguistic expertise with machine learning in current approaches for low-resource scenarios

Paper Type: Short

Research Area: Special Theme (conference specific)

Research Area Keywords: Under-resourced languages, generalization, orthographic normalization, finite-state transducer (FST), in-context learning (ICL)

Contribution Types: Model analysis & interpretability, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: Otomi

Submission Number: 4783