Multi-Query Augmentation-Based Web Landmark Photo Retrieval

Lin Wu, Xiaodi Huang, John Shepherd, Yang Wang

Abstract: Given a query photo characterizing a location-aware landmark shot by a user, landmark retrieval is about returning a set of photos ranked in their similarities to the query. Existing studies on landmark retrieval focus on conducting a matching process between candidate photos and a query photo by exploiting location-aware visual features. Notwithstanding the good results achieved, these approaches are based on an assumption that a landmark of interest is well-captured and distinctive enough to be distinguished from others. In fact, distinctive landmarks may be badly selected, e.g. changes on viewpoints or angles. This will discourage the recognition results if a biased query photo is issued. In this paper, we present a novel technique that exploits user communities in social media networks. Given a biased query photo containing some landmarks taken by a user, we select multiple users to complement this user for retrieval. Multiple photos are then used to enrich the query photo, constituting a more representative yet robust multi-query set. A pattern mining method is developed to obtain a compact feature representation of photos from the multi-query set. Such a representation is utilized to efficiently query the database so as to improve retrieval results. Extensive experiments on real-world datasets demonstrate the effectiveness and efficiency of our approach.