LMR-IPGN: An Effective Model for automatic summarization of Chinese long text

Dangguo Shao, Gaoan Huang, Lei Ma, Yuxin Wu, Kaiqiang Tang, Sanli Yi, Jingtao Li, Nuoyun Duan, Chunyun Pu

Abstract: Chinese Text Summarization has been a hot topic in last few years. But it faces two inherent constraints compared to other languages: (1) The lack of explicit morphological changes in Chinese combined with long-distance dependency issues leads to semantic understanding deviations; (2) Flexible character combinations create ambiguous structures that interfere with capturing global semantic features, resulting in generated summary with semantic confusion and fabricated content. Additionally, cross-entropy based learning methods do not always yield optimal evaluation results. To address these challenges, we propose a hybrid model(LMR-IPGN) that integrates multiple types of attention based on encoder-decoder model and policy learning. First, during the preprocessing stage, Chinese lexical resources and word segmentation tools are employed to process source documents, reducing redundancy through synonym substitution while resolving Chinese word segmentation ambiguities. Specifically, the hierarchical level intra multi-head attention mechanism enhances key information extraction by allocating more attention to low-frequency yet critical words, thereby mitigating excessive focus on high-frequency but less significant terms or phrases. Secondly, the model integrates temporal attention with the pointer mechanism to record attention distributions of historically generated words, penalizing repetitively attended regions to avoid information redundancy. Finally, a hybrid reward signal combining cross-entropy and ROUGE-N is introduced during the decoding stage, optimizing generation quality through a mixed learning strategy. Experimental results on the NLPCC2017 dataset and CSL dataset show that our proposed model significantly outperforms other baselines in terms of Rouge-N scores. We also perform human evaluation to demonstrate the effectiveness of our model by evaluating the generated summary using several subjective metrics.