Evaluating the effectiveness of Web search engines on results diversification
Abstract: In the last couple of years the problem of diversification of search results has attracted a lot of attention in the information retrieval/Web search research community. For multi-faceted or ambiguous queries, a search engine is generally favoured if it is able to identify relevant documents on a wider range of different aspects. We evaluate the performance of three major Web search engines: Google, Bing and Ask manually using 200 multi-faceted or ambiguous queries from TREC. Both classical metrics and intent-aware metrics are used to evaluate search results. Experimental results show that on average Bing and Google are comparable and Ask is slightly worse than the former two. However, Ask does very well in one subtype of queries – ambiguous queries. The average performance of the three search engines is better than the average of the top two runs submitted to the TREC web diversity task in 2009-2012. Generally, all three Web search engines do well, this indicates that all of them must use state-of-the-art technology to support the diversification of search results.
Loading