Disentangled Causal Embedding with Unbiased Knowledge Distillation for Recommendation

Nan Liu, Shunmei Meng, Xiao Liu, Qianmu Li, Yu Jiang

Abstract: Knowledge Distillation, a model compression method that transfers knowledge from a complex teacher model to a simpler student model, has been widely applied in recommendation systems. Existing KD recommendation models primarily focus on improving the performance and inference speed of the student model during the inference stage, while overlook the significant issue of popularity bias. It is evident that the issue of bias exists, arising from the soft labels provided by the teacher model, is further propagated and intensified throughout the distillation process. Using causal embeddings or propensity-based unbiased learning to eliminate the bias effect is proved to be effective. Nevertheless, by solely amalgamating various causes of popular bias and interactions into a unified representation, the robustness and interpretability of the model cannot be assured. In view of these challenges, we present DCUKD, a new unbiased knowledge distillation approach with disentangling causal embedding. It addresses the root causes of deviation, aiming to produce unbiased results and significantly enhance the model’s recommendation performance. In particular, we introduce a causal mechanism with popularity bias to verify that popularity bias is not entirely detrimental. Then DCUKD decomposes popularity bias based on time effects from causal graph, which eliminates detrimental bias and retains benign bias. Through causal intervention, the recommendation result is adjusted with the desired bias to guide the student model. Finally, we explore the effectiveness of DCUKD on two real datasets, showing that our method outperforms all considered baselines.