Unify the Usage of Lexicon in Chinese Named Entity Recognition

Wenjia Wu, Changyou Zhang, Shuzi Niu, Lin Shi

Abstract: Lexicon plays a critical role in Chinese Named Entity Recognition (CNER). The major reason lies in that words in the lexicon, lexicon words for short, are highly related to entity mention boundaries. Most lexicon enhanced CNER approaches focus on introducing lexicon words to the input and hidden layers. However, existing lexicon enhanced methods make the method hard to be adaptable and put weak lexicon constraints on architectures. To tackle these challenge, we propose a unified lexicon enhanced CNER framework. Specifically, lexicon word identification (LWI) task is proposed to locate and classify textual references to lexicon words. Similar to CNER task, this task is formalized either as sequence labeling or character relation classification, adopts CRF or Co-Predictor (Character relation classification) as the task specific layer, and is optimized with log-likelihood function of sequence probability or cross entropy of character pair relation type distribution. LWI task shares the input and hidden layers with CNER task. The whole framework is pre-trained with LWI task and fine-tuned with CNER task. Experimental results on two benchmark CNER datasets show the better effectiveness and flexibility than state-of-the-art baselines.