GIE : Gradient Inversion with Embeddings

Zenghao Guan, Yucan Zhou, Xiaoyan Gu, Bo Li

Abstract: Federated Learning (FL) has emerged as a promising approach to preserve privacy for distributed machine learning, through aggregating gradients calculated from local data of multiple clients. Recent studies show that gradients can be inverted to reconstruct private data retained by local clients in a process called gradient inversion. However, the performance of existing gradient inversion attacks declines as batch size increases. In this work, we propose a simple but effective method to tackle this challenge. Our method utilizes an interesting property between weights, embeddings and their corresponding gradients in fully connected layer, which enables the rapid and accurate recovery of embeddings. And these embeddings benefit the recovery of private data as additional prior knowledge. Extensive experiments demonstrate that our method can perform high-quality recovery of private data.