Evaluating a Universal Dependencies Conversion Pipeline for Icelandic

Þórunn Arnardóttir, Hinrik Hafsteinsson, Atli Jasonarson, Anton Karl Ingaon, Steinþór Steingrímsson

Keywords: Universal Dependencies, UDConverter, Treebank, Conversion, Evaluation

TL;DR: The paper evaluates UDConverter, a tool for converting phrase structure treebanks to UD treebanks, by comparing converted sentences to manually corrected ones. Results are used to improve the tool and determine its benefit.

Abstract: We describe the evaluation and development of a rule-based treebank conversion tool, UDConverter, which converts treebanks from the constituency-based PPCHE annotation scheme to the dependency-based Universal Dependencies (UD) scheme. The tool has already been used in the production of three UD treebanks, although no formal evaluation of the tool has been carried out as of yet. By manually correcting new output files from the converter and comparing them to the raw output, we measured the labeled attachment score (LAS) and unlabeled attachment score (UAS) of the converted texts. We obtain an LAS of 82.87 and a UAS of 87.91. In comparison to other tools, UDConverter currently provides the best results in automatic UD treebank creation for Icelandic.