The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking
Abstract: Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors, which are mainly caused by phonologically or visually similarity. Recently, due to the development of various pre-trained language models (PLMs), many CSC methods have achieved great progress. However, PLMs will pay more attention to common characters because of the pre-training settings. Therefore, there exists a gap between the learned knowledge of PLMs and the essential of CSC task. To address this issue, we propose an Error-driven COntrastive Probability Optimization (ECOPO) framework to refine the knowledge representation of PLMs for CSC. Particularly, ECOPO guides the model to avoid predicting common but improper characters through an error-driven way. Besides, ECOPO is model-agnostic so that it can be easily combined with existing CSC methods to achieve better performance. Extensive experiments and detailed analysis on three standard benchmarks demonstrate that ECOPO is simple yet effective.
0 Replies
Loading