Privacy Risks of Federated Knowledge Graph Embedding: New Membership Inference Attacks and Personalized Differential Privacy Defense

Yuke Hu, Yang Wang, Jian Lou, Wei Liang, Ruofan Wu, Weiqiang Wang, Xiaochen Li, Jinfei Liu, Zhan Qin

Abstract: Knowledge Graph Embedding (KGE) has been widely studied as an important semantic enhancement technique that extracts expressive representation from Knowledge Graph (KG) to facilitate various downstream applications, such as knowledge reasoning, Semantic Web, and question answering. The advent of Federated KGE (FKGE) allows for collaborative training across distributed KGs without revealing clients’ private raw KGs that contain sensitive knowledge graph triples. Despite this, FKGE remains susceptible to privacy threats as demonstrated in previously studied federated learning models. However, utilizing and addressing these vulnerabilities remain uninvestigated for FKGE which exhibits unique characteristics distinct from other models. In this work, we conduct the first comprehensive study of the privacy issues in FKGE from both attack and defense perspectives. On the attack side, we introduce five new inference attacks, highlighting the privacy vulnerabilities by successfully deducing the presence of KG triples from the targeted dataset. On the defense side, we present PDP-Flames, a novel differentially private FKGE scheme that leverages the sparse gradient nature of FKGE for better privacy-utility trade-off by integrating advanced private selection techniques. We further introduce a dynamic defense policy based on the observation that the privacy risk fluctuates throughout the training procedure. Additionally, we incorporate a personalized procedure to provide a customized model tailored to the unique data distributions of individual clients. Joint differential privacy is introduced to guarantee the privacy of the personalized models. Comprehensive experiments demonstrate that PDP-Flames effectively mitigates privacy concerns, notably diminishing the attack success rate while maintaining decent model utility.