Deep Convolutional Ranking for Multilabel Image Annotation

Sergey Ioffe, Alexander Toshev, Yangqing Jia, Thomas Leung, Yunchao Gong

Decision: submitted, no decision

Abstract: Multilabel image annotation is one of the most important challenges in computer vision with many real-world applications. While existing work usually use conventional visual features for multilabel annotation, the recent deep convolutional feature shows potentials to significantly boost performance. In this work, we propose to leverage the advantage of such features and analyze key components that lead to better performances. Specifically, we show that a significant performance gain could be obtained by combining convolutional architectures with an approximate top-$k$ ranking objective function, as such objectives naturally fit the multilabel tagging problem. Our experiments on the publicly available NUS-WIDE dataset outperforms the conventional visual features by about $10%$, obtaining the best reported performance in the literature.