A Case for Intrinsic Evaluation of Optical Music Recognition

Jul 19, 2018 ISMIR 2018 WoRMS Submission readers: everyone
  • Abstract: Evaluating Optical Music Recognition (OMR) has long been an acknowledged sore spot of the field. This short position paper attempts to bring some clarity to what are actually open problems in OMR evaluation: a closer look reveals that the main problem is finding an edit distance between some practical representations of music scores. While estimating these editing costs in the transcription use-case of OMR is difficult, I argue that the problems with modeling the subsequent editing workflow can be de-coupled from general OMR system development using an intrinsic evaluation approach, and sketch out how to do this.
  • Keywords: Optical Music Recognition, Evaluation, Music Notation
  • TL;DR: It is possible to have a thorough automated evaluation metric for OMR in general, rather than for individual use-cases, that frees OMR researchers from having to think about how people edit music scores.
0 Replies