A Case for Intrinsic Evaluation of Optical Music Recognition

Jan Hajič jr.

Abstract:

Evaluating Optical Music Recognition (OMR) has long been an acknowledged
sore spot of the field. This short position paper attempts to bring some clarity
to what are actually open problems in OMR evaluation:
a closer look reveals that the main problem is finding an edit distance between
some practical representations of music scores.
While estimating these editing costs in the transcription use-case of OMR
is difficult,
I argue that the problems with modeling the subsequent editing workflow
can be de-coupled from general OMR system development
using an intrinsic evaluation approach, and sketch out how to do this.

Keywords: Optical Music Recognition, Evaluation, Music Notation

TL;DR: It is possible to have a thorough automated evaluation metric for OMR in general, rather than for individual use-cases, that frees OMR researchers from having to think about how people edit music scores.