Comparing the Effects of Annotation Type on Machine Learning Detection Performance

James F. Mullen Jr., Franklin R. Tanner, Phil A. Sallee

Abstract: The most prominent machine learning (ML) methods in use today are supervised, meaning they require groundtruth labeling of the data on which they are trained. Annotating data is arduous and expensive. Additionally, data sets for image object detection may be annotated by drawing polygons, drawing bounding boxes, or providing single points on targets. Selection of annotation technique is a tradeoff between time to annotate and accuracy of the annotation. When annotating a dataset for machine object recognition algorithms, researchers may not know the most advantageous method of annotation for their experiments. This paper evaluates the performance tradeoffs of three alternative methods of annotating imagery for use in ML. A neural network was trained using the different types of annotations and compares the detection accuracy of and differences between the resultant models. In addition to the accuracy, cost is analyzed for each of the models and respective datasets.