A Method for Automatically Building and Evaluating Dictionary Resources

Smaranda Muresan, Judith Klavans

Abstract: This paper describes a method toward automatically building dictionaries from text. We present DEFINDER, a rule-based system for extraction of definitions from on-line consumer-oriented medical articles. We provide an extensive evaluation on three  dimensions: i) performance of the definition extraction technique in terms of precision  and recall, ii) quality of the built dictionary as judged both by specialists and lay users, iii) coverage of existing on-line dictionaries. The corpus we used for the study is publicly available. A major contribution of the paper is the range of quantitative and qualitative evaluation methods.