Marginal-Certainty-aware Fair Ranking Algorithm

Tao Yang, Zhichao Xu, Zhenduo Wang, Anh Tran, Qingyao Ai

Abstract:

Ranking systems are ubiquitous in modern Internet services, including
online marketplaces, social media, and search engines. Traditionally,
ranking systems only focus on how to get better relevance
estimation. When relevance estimation is available, they usually
adopt a user-centric optimization strategy where ranked lists are
generated by sorting items according to their estimated relevance.
However, such user-centric optimization ignores the fact that item
providers also draw utility from ranking systems. It has been shown
in existing research that such user-centric optimization will cause
much unfairness to item providers, followed by unfair opportunities
and unfair economic gains for item providers.
To address ranking fairness, many fair ranking methods have
been proposed. However, as we show in this paper, these methods
could be suboptimal as they directly rely on the relevance estimation
without being aware of the uncertainty (i.e., variance of the
estimated relevance). To address this uncertainty, we propose a
novel Marginal-Certainty-aware Fair algorithm named MCFair.
MCFair jointly optimizes fairness and user utility, while relevance
estimation is constantly updated in an online manner. In MCFair,
we first develop a ranking objective that includes uncertainty, fairness,
and user utility. Then we directly use the gradient of the
ranking objective as the ranking score. We theoretically prove
that MCFair based on gradients is optimal for the aforementioned
ranking objective. Empirically, we find that on semi-synthesized
datasets, MCFair is effective and practical and can deliver superior
performance compared to state-of-the-art fair ranking methods. To
facilitate reproducibility, we release our code.