FairSync: Ensuring Amortized Group Exposure in Distributed Recommendation Retrieval

Chen Xu, Jun Xu, Yiming Ding, Xiao Zhang, Qi Qi

Keywords: distributed retrieval, recommender system, amortized group exposures

TL;DR: We developed a model called FairSync to ensure minimum amortized group exposures in the recommendation retrieval process, catering to distributed and online requirements.

Abstract:

Driven by considerations of fairness, business, and balanced development needs, the recommender system (RS) often necessitates ensuring that certain groups have a minimum level of exposure within a period of time. For example, RS platforms often have the demand to ensure adequate exposure for new providers or specific categories of items according to their needs.
Modern industry RS usually adopts a two-stage pipeline: stage-1 (retrieval stage) retrieves hundreds of candidates from millions of items distributed across various servers, and stage-2 (ranking stage) focuses on presenting a small-size but accurate selection from items chosen in stage-1. Existing efforts for ensuring amortized group exposures focus on stage-2, however, stage-1 is also critical for the task.
Without a high-quality set of candidates, the stage-2 ranker cannot ensure the required exposure for the selected groups.
Previous fairness-aware works designed for stage-2 typically require accessing and traversing of all items. In stage-1, however, millions of items are distributively stored in servers, making it infeasible to traverse all of them. How to ensure the global amortized group exposures in the distributed retrieval process is a challenging question. To address this issue, we introduce a model named FairSync, which transforms the problem into a constrained distributed optimization problem. Specifically, FairSync resolves the issue by moving it to the dual space, where a central node aggregates historical fairness data into a vector and distributes it to all servers. In theory, with local and distributed searching, we can ensure the amortized exposures within the dual space. To trade-off the efficiency and retrieval accuracy, the gradient descent technique is used to periodically update the parameter of the dual vector. The experiment results on two public recommender retrieval datasets showcased that FairSync outperformed all the baselines, achieving the desired minimum level of exposures while maintaining a high level of retrieval accuracy.

Track: Responsible Web

Submission Guidelines Scope: Yes

Submission Guidelines Blind: Yes

Submission Guidelines Format: Yes

Submission Guidelines Limit: Yes

Submission Guidelines Authorship: Yes

Student Author: Yes

Submission Number: 561