Mitigating Prototype Shift: Few-Shot Nested Named Entity Recognition with Prototype-Attention

Anonymous

Abstract: Nested entities are prone to obtain similar representations in pre-trained language models, posing challenges for Named Entity Recognition (NER), especially in the few-shot setting where prototype shifts often occur due to distribution differences between the support and query sets. In this paper, we regard entity representation as the combination of prototype and non-prototype representations. With a hypothesis that using the prototype representation specifically can help mitigate potential prototype shifts, we propose a Prototype-Attention mechanism in the Contrastive Learning framework (PACL) for the few-shot nested NER. PACL first generates prototype-enhanced span representations to mitigate the prototype shift by applying a prototype attention mechanism. It then adopts a novel prototype-span contrastive loss to reduce prototype differences further and overcome the O-type's non-unique prototype limitation by comparing prototype-enhanced span representations with prototypes and original semantic representations. Our experiments on three English, German, and Russian nested NER datasets show that the PACL outperformed seven baseline models on the 1-shot and 5-shot tasks in terms of $F_1$ score. Further analyses indicate that our Prototype-Attention mechanism has high generality, enhancing the performance of two baseline models, and can serve as a valuable tool for NLP practitioners facing few-shot nested NER tasks.

Paper Type: long

Research Area: Information Extraction