Multi-attention deep neural network fusing character and word embedding for clinical and biomedical concept extraction

Shengyu Fan, Hui Yu, Xiaoya Cai, Yanfang Geng, Guangzhen Li, Weizhi Xu, Xia Wang, Yaping Yang

Abstract: Highlights•Local and global self-attention mechanisms are used for character embedding.•CNN with multi-size filters are used to extract character information for NER.•A cross-attention method that fuses character and word embedding for NER is proposed•A modified Mogrifier LSTM is presented to improve the performance of NER.•Proposed methods integrated with a transformer-based model achieve good performance.