Chinese Spelling Corrector Is Just Language Learner

Anonymous

Abstract: This paper emphasizes the Chinese spelling correction of self-supervised learning, which means there are no annotated errors within the training data. This setting is a pivotal issue that has received broad attention in the community. Our intuition is that humans are naturally good correctors with exposure to monolingual sentences, which contrasts with current unsupervised methods that strongly rely on the usage of confusion sets to produce parallel sentences. In this paper, we demonstrate that learning a spelling correction model is identical to learning a language model from monolingual data alone, with decoding it in a greater search space.We propose \emph{Denoising Decoding Correction (D\textsuperscript{2}C)}, which selectively imposes noise upon the source sentence to solve out the underlying correct characters. Our method largely inspires the ability of language models to perform correction, including both BERT-based models and large language models (LLMs). We show that the self-supervised learning manner generally outstrips using confusion sets in specific domains because it bypasses the need to introduce error characters to the training data which can impair the patterns in the target domains. We evaluate our methods on multi-domain datasets Syn-LEMON proposed by our work and ECSpell \citep{Ecspell}.

Paper Type: long

Research Area: NLP Applications

Contribution Types: Approaches to low-resource settings

Languages Studied: Chinese