Visual Reranking with Local Learning Consistency

Xinmei Tian, Linjun Yang, Xiuqing Wu, Xian-Sheng Hua

Abstract: The graph-based reranking methods have been proven effective in image and video search. The basic assumption behind them is the ranking score consistency, i.e., neighboring nodes (visually similar images or video shots) in a graph having close ranking scores, which is modeled through a regularizer term. The existing reranking methods utilise pair-wise regularizers, e.g., the Laplacian regularizer and the normalized Laplacian regularizer, to estimate the consistency over the graph from the pair-wise perspective by requiring the scores to be close for pairs of samples. However, since the consistency is a term defined over the whole set of neighboring samples, it is characterized by the local structure of the neighboring samples, i.e., the multiple-wise relations among the neighbors. The pair-wise regularizers fail to capture the desired property of consistency since they treat the neighboring samples independently. To tackle this problem, in this paper, we propose to use local learning regularizer to model the multiple-wise consistency, by formulating the consistent score estimation over a local area into a learning problem. Experiments on the TRECVID benchmark dataset and a real Web image dataset demonstrate the superiority of the local learning regularizer in visual reranking.