On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective
Track: Systems and infrastructure for Web, mobile, and WoT
Keywords: On-device Recommender Systems, Single-shot Model Compression, Click-through Rate Prediction
TL;DR: To achieve fast compression for on-device recommendation models, we put forward a cooperative game-based approach to quantify each embedding parameter's importance to allow for model pruning to any given budget in no time.
Abstract: Content-based Recommender Systems (CRSs) play a crucial role in shaping user experiences in e-commerce, online advertising, and personalized recommendations. However, due to the large amount of categorical features, the embedding tables used in CRS models pose a significant storage bottleneck for real-world deployment, especially on resource-constrained devices. To address this problem, various embedding pruning methods have been proposed, but most existing ones require expensive retraining steps for each target parameter budget, leading to large computational costs. In reality, this computation cost is a major hurdle in real-world applications with diverse storage requirements, such as federated learning and streaming settings. In this paper, we propose SHApley Value-guided Embedding Reduction (Shaver) as our response. With Shaver, we view the problem from a cooperative game perspective, and quantify each embedding parameter's contribution with Shapley values to facilitate contribution-based parameter pruning. To address the inheriently high computation costs of Shapley values, we propose an efficient and unbiased method to estimate Shapley values of a CRS's embedding parameters. Moreover, in the pruning stage, we put forward a field-aware codebook to mitigate the information loss in the traditional zero-out treatment. Through extensive experiments on three real-world datasets, Shaver has demonstrated competitive performance with lightweight recommendation models across various parameter budgets. The source code is available at https://anonymous.4open.science/r/shaver-E808.
Submission Number: 803
Loading