Parser Evaluation for Analyzing Swedish 19th-20th Century Literature

Sara Stymne, Carin Östman, David Håkansson

Keywords: Dependency parsing, evaluation, lightweight annotation, 19th century Swedish, multilingual parsing

TL;DR: We explore a lightweight annotation task for parser evaluation of subordinate clauses in 19th-20th century Swedish literature.

Abstract: In this study, we aim to find a parser for accurately identifying different types of subordinate clauses, and related phenomena, in 19th--20th-century Swedish literature. Since no test set is available for parsing from this time period, we propose a lightweight annotation scheme for annotating a single relation of interest per sentence. We train a variety of parsers for Swedish and compare evaluations on standard modern test sets and our targeted test set. We find clear trends in which parser types perform best on the standard test sets, but that performance is considerably more varied on the targeted test set. We believe that our proposed annotation scheme can be useful for complementing standard evaluations, with a low annotation effort.