Are we done with object recognition? The iCub robot's perspective

Giulia Pasquale, Carlo Ciliberto, Francesca Odone, Lorenzo Rosasco, Lorenzo Natale

Abstract: Highlights • We study the application of deep learning approaches to object categorization and identification problems in robot vision scenarios. • We release a novel dataset used for our investigation, iCubWorld Transformations, collected through a natural interaction with the iCub robot and rich in terms of objects and viewpoint transformations. • While confirming the remarkable improvement yield by deep learning in this setting, we identify specific open challenges that need be addressed for deployment of these methods in robotics. Abstract We report on an extensive study of the benefits and limitations of current deep learning approaches to object recognition in robot vision scenarios, introducing a novel dataset used for our investigation. To avoid the biases in currently available datasets, we consider a natural human–robot interaction setting to design a data-acquisition protocol for visual object recognition on the iCub humanoid robot. Analyzing the performance of off-the-shelf models trained off-line on large-scale image retrieval datasets, we show the necessity for knowledge transfer. We evaluate different ways in which this last step can be done, and identify the major bottlenecks affecting robotic scenarios. By studying both object categorization and identification problems, we highlight key differences between object recognition in robotics applications and in image retrieval tasks, for which the considered deep learning approaches have been originally designed. In a nutshell, our results confirm the remarkable improvements yield by deep learning in this setting, while pointing to specific open challenges that need be addressed for seamless deployment in robotics.