Federated Recommendation with Reinforcement Learning based Knowledge Distillation

Haochen Yuan, Yang Zhang, Xiang He, Quan Z. Sheng, Zhongjie Wang

Keywords: Federated Learning, Recommender System, Knowledge Distillation, Reinforcement Learning

Abstract: Federated recommendation (FR) has emerged as a promising paradigm to enhance user privacy by training models in a distributed manner, where participants share model updates instead of raw data. Despite its advantages, three major challenges remain: (i) Privacy leakage. Direct parameter sharing risks exposing model information. (ii) Degraded performance. Parameter-transfer based methods often underperforms compared to centralized training due to limited knowledge exchange. (iii) Communication overhead. Frequent synchronization of large models incurs prohibitive communication costs. To address these challenges, we propose FedKDRec, a novel FR framework based on bidirectional knowledge distillation (KD) with agents. Instead of exchanging parameters, FedKDRec transfers soft predictions under a privacy-preserving mechanism, thereby protecting both user data and model assets. To further improve effectiveness and efficiency, we introduce: (1) a server-oriented agent that dynamically assigns weights to client knowledge in multi-teacher KD, and (2) a client-oriented agent that selectively transfers informative yet lightweight samples from the server. Extensive experiments across diverse datasets and models demonstrate that FedKDRec significantly achieves superior performance and reduces communication overhead compared to parameter-transfer FR methods and existing KD-based baselines.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 11605