Diversity-aware strategies for static index pruning

Sevgi Yigit-Sert, Ismail Sengor Altingovde, Özgür Ulusoy

Abstract: Highlights•Typical methods for static index pruning may harm topical diversity of search results.•We consider topical diversity while tailoring and evaluating pruning methods.•We exploit clustering and word embeddings in our diversity-aware pruning methods.•We assume retrieved results are for the end user or to be further diversified.•Our methods are effective in maintaining diversity and even fairness of query results.