Locator: Local Stability for Rankings

Felix S. Campbell, Yuval Moskovitch

Abstract: Rankings play a crucial role in decision-making. However, if minor changes to items significantly alter their rankings, the quality of the decisions being made can be compromised. The stability of ranking is a measure used to assess how modifications to the ranking algorithm or data affect results. We introduce a novel measure we refer to as local stability, indicating the effect of minor changes to the values of an item in the ranking on its rank. Our proposed definition takes into account the presence of multiple items with similar qualities in the ranking, called dense regions, permitting minor modifications to swap the positions of items within the region. We present Locator, a system utilizing local stability to provide ranking stakeholders--producers, participants, and consumers--with insights into the ranking. Ranking producers can use Locator to adjust their ranking functions, enhancing item stability. They can then share the local stability information with ranking stakeholders to provide a meaningful understanding of how uncertainty in an item's attributes affects its rank without exposing the ranking methodology.

External IDs:dblp:conf/sigmod/CampbellM25