Automatic Evaluation of Relation Extraction Systems on Large-scale

Mirko Bronzi, Zhaochen Guo, Filipe de Sá Mesquita, Denilson Barbosa, Paolo Merialdo

Abstract: The extraction of relations between named entities from natural language text is a longstanding challenge in information extraction, especially in large-scale. A major challenge for the advancement of this research field has been the lack of meaningful evaluation frameworks based on realistic-sized corpora. In this paper we propose a framework for large-scale evaluation of relation extraction systems based on an automatic annotator that uses a public online database and a large web corpus.