Automatic Evaluation of Relation Extraction Systems on Large-scale

Mirko Bronzi, Zhaochen Guo, Denilson Barbosa, Filipe Mesquita, Paolo Merialdo

Abstract: The extraction of relations between named entities from natural language text is a long- standing challenge in information extraction, especially in large-scale. A major challenge for the advancement of this research field has been the lack of meaningful evaluation frame- works based on realistic-sized corpora. In this paper we propose a framework for large-scale evaluation of relation extraction systems based on an automatic annotator that uses a public online database and a large web corpus