Query translation based on visual information

Jiao Zhang, Yonggang Huang, Qingzhao Jiang, Wenpeng Lu, Hualei Shen

Abstract: In the field of cross language information retrieval, how to translate the query into the target language, namely query translation, is a fundamental problem. Because of the ambiguity phenomenon, query translation is always a challenge. Existing researches always rely on mining the text information, such as the contextual relationship or word occurrence. Different from existing research efforts, in this paper, we address the query translation issue by mining the visual information of images, and a new query translation method based on visual information (QTVI) is proposed. QTVI has three steps: image search, image set denoising, and translation candidate selection. In step 1, the query and candidate translation are associated with corresponding image set via image search. Since the resulted image sets from step 1 may be unclean, in step 2, we de-noise the image sets via clustering strategy. Finally, in step 3, the final translation is selected from candidates by constructing multi-class classifier based on cleaned image sets. Empirical experiments show that QTVI outperforms Baidu Translation and Google Translation for the query translation task.