Automatically generating labels based on unified click model

Guichun Hua, Min Zhang, Yiqun Liu, Shaoping Ma, Liyun Ru

Abstract: Ground truth labels are one of the most important parts in many test collections for information retrieval. Each label, depicting the relevance between a query-document pair, is usually judged by a human, and this process is time-consuming and labor-intensive. Automatically Generating labels from click-through data has attracted increasing attention. In this paper, we propose a Unified Click Model to predict the multi-level labels, which aims at comprehensively considering the advantages of the Position Models and Cascade Models. Experiments show that the proposed click model outperforms the existing click models in predicting the multi-level labels, and could replace the labels judged by humans for test collections.