Playing games as a way to improve automatic image annotation

Rui M. Jesus, Duarte Gonçalves, Arnaldo J. Abrantes, Nuno Correia

Abstract: Image annotation is hard to do in an automatic way. In this paper, we propose a framework for image annotation that combines the benefits of three paradigms: automatic annotation, human intervention and entertainment activities. We also describe our proposal inside this framework, the ASAA (application for semi-automatic annotation) interface, a new computer game for image tagging. The application has a 3D game interface, and is supported by a game engine that uses a system for automatic image classification and gestural input to play the game. We present results of the performance of semantic models obtained with a training set enlarged by images annotated during the game activity as well as usability tests of the application.