A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition

Zihan Wang, Ziqi Zhao, Yougang Lyu, Zhumin Chen, Maarten de Rijke, Zhaochun Ren

Track: Web mining and content analysis

Keywords: Information extraction, Named entity recognition, Zero-shot learning, Large language models, Multi-agent system

Abstract:

Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora. This task presents substantial challenges due to minimal human intervention. Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates. And it advances the models’ self-learning ability by incorporating self-annotated demonstrations. Two important challenges persist: (i) Correlations between contexts surrounding entities are overlooked, leading to wrong type predictions or entity omissions. (ii) The indiscriminate use of task demonstrations, retrieved through shallow similarity-based strategies, severely misleads the inferences made by LLMs.

In this paper, we introduce CMAS, or cooperative multi-agent system, a framework for zero-shot NER that uses the collective intelligence and tailored abilities of multiple agents to address the challenges outlined above. Cooperative multi-agent system (CMAS) has four main agents: (i) a self-annotator, (ii) a type-related feature (TRF) extractor, (iii) a demonstration discriminator, and (iv) an overall predictor. To explicitly capture correlations between contexts surrounding entities, CMAS reformulates NER into two subtasks: recognizing named entities and identifying entity type-related features within the target sentence. Moreover, pseudo-labels for TRFs are generated using mutual-information criteria without requiring human effort, facilitating the prediction of the TRF extractor. To assess the quality of demonstrations, a demonstration discriminator is established to incorporate the self-reflection mechanism, automatically evaluating helpfulness scores for the target sentence and enabling controllable utilization of demonstrations.

Experimental results show that CMAS significantly improves zero-shot NER performance across six benchmarks, including both domain-specific and general-domain scenarios. Furthermore, CMAS demonstrates its effectiveness in few-shot settings and with various LLM backbones.

Submission Number: 812