Joint upper & expected value normalization for evaluation of retrieval systems: A case study with Learning-to-Rank methods

Dongji Feng, Shubhra Santu K. Karmaker

Abstract: Highlights•A new framework for IR evaluation using query-specific expected-value normalization.•Instantiation of the proposed framework on metrics nDCG and MAP.•New metric shows higher Discriminatory Power and better Consistency.•New metric impacts “Uninformative” queries more than “Informative” ones.