Analysis of Annotation Quality of Human Activities Using Knowledge Graphs

Shusaku Egami, Mikiko Oono, Mai Otsuki, Takanori Ugai, Ken Fukuda

Abstract: Human activity data from cameras and sensors have potential applications in diverse domains. However, annotation quality varies, and label inconsistency remains a challenge. Annotators’ different interpretations also cause other issues. In this paper, we proposed an approach for analyzing the annotation quality for videos of human activities focusing on annotation issues. Specifically, we annotated the same videos using different strategies: (1) fine temporal granularity using primitive action ontology vocabularies and (2) coarse temporal granularity focusing on meaningful action sequences. We then constructed knowledge graphs based on the annotated data and analyzed the relationship between annotation types and noise label tendencies. The results of this study could potentially support the construction of high-quality annotated datasets necessary for understanding human activities in various scenes, from daily life to service operations.