Improve the Sample Efficiency of Machine for Interactive Data Annotation

Anonymous

Abstract: To reduce human labor on manual annotations, interactive annotation leverages a model to provide annotation suggestions for the human to approve or correct. When the model is under-trained due to limited data, it tends to make wrong suggestions, requiring extra human labor to correct. To this end, we resort to analogical reasoning and propose a general sample-efficient plug-in module. This module builds analogies to historical annotated data and refines the suggestions through a dynamic weighting mechanism, thus reducing human labor. Empirical studies show the flexibility of our method in being compatible with various annotation tasks. With our method, the model, on average, saves a relative 145.08% of annotated data to reach the required accuracy. It translates to an estimated 20% less human labor compared to the original interactive annotation.

Paper Type: long

Research Area: Dialogue and Interactive Systems