A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Models
Abstract: This work proposes a simple yet effective approach for leveraging large language models (LLMs) in Chinese spelling correction (CSC) task. Our approach consists of two components: a large language model and a minimal distortion model. At each decoding step, the large language model calculates the probabilities of the next token based on the preceding context. Then, the distortion model adjusts these probabilities to penalize the generation of tokens that deviate too far from the input. Different from the prior supervised fine-tuning and prompt-based approaches, our approach enables efficient CSC without requiring additional training or task-specific prompts. To address practical challenges, we propose a length reward strategy to mitigate the local optima problem during beam search decoding, and a faithfulness reward strategy to reduce over-corrections. Comprehensive experiments on five public datasets demonstrate that our approach significantly improves LLM performance, enabling them to compete with state-of-the-art domain-general CSC models.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: Chinese spelling correction
Contribution Types: NLP engineering experiment
Languages Studied: Chinese
Submission Number: 1782
Loading