Assumption-free stability for ranking problems

TMLR Paper8623 Authors

Abstract: In this work, we consider ranking problems among a finite set of candidates: for instance, selecting the top-$k$ items from a list or obtaining the full ranking of all items in the set. These problems are often unstable: estimating a ranking from noisy data can exhibit high sensitivity to small perturbations. Concretely, if we use data to assign scores to items (say, by aggregating user preference data), then for two items with similar scores, small fluctuations in the data can alter their relative rankings. Many existing theoretical results sidestep this challenge by assuming a separation condition, but real-world data often contains near-ties, limiting the applicability of existing theory. To address this gap, we develop a new algorithmic stability framework for ranking problems, and propose two novel ranking operators for achieving stability: the \emph{inflated top-$k$} for the top-$k$ selection problem and the \emph{inflated full ranking} for ranking the full list, each of which allows for expressing some uncertainty in the output. Both our proposed methods provide guaranteed stability, with no assumptions on data distributions and no dependence on the total number of candidates to be ranked. Experiments on real-world data confirm that the proposed methods offer stability while retaining informativeness.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Lirong_Xia1

Submission Number: 8623