Towards Large Scale Cross-Media Retrieval via Modeling Heterogeneous Information and Exploring an Efficient Indexing Scheme

Bo Lu, Guoren Wang, Ye Yuan

Abstract: With the rapid development of Internet and multimedia technology, cross-media retrieval is concerned to retrieve all the related media objects with multi-modality by submitting a query media object. In this paper, we propose a novel method which is dedicate to achieve effective and accurate cross-media retrieval. Firstly, a Multi-modality Semantic Relationship Graph (MSRG) is constructed by using the semantic correlation amongst the media objects with multi-modality. Secondly, all the media objects in MSRG are mapped onto an isomorphic semantic space. Further, an efficient indexing MK-tree based on heterogeneous data distribution is proposed to manage the media objects within the semantic space and improve the performance of cross-media retrieval. Extensive experiments on real large scale cross-media datasets indicate that our proposal dramatically improves the accuracy and efficiency of cross-media retrieval, outperforming the existing methods significantly.