Graph Personalized Federated Learning via Client Network Learning

Jiachen Zhou, Han Xie, Carl Yang

Abstract: Graph classification is a widely studied problem for applications such as molecule/protein function prediction and drug discovery. Powerful graph neural networks (GNNs) have demonstrated state-of-the-art performance for the classification of complex graphs, but training such models can require significant amounts of high-quality labeled graphs that are expensive to collect. When individual institutes do not possess sufficient graph data, federated learning (FL) becomes a handy solution for them to collaboratively obtain powerful graph models without directly sharing their own graph data. However, existing FL frameworks for graph data do not consider the realistic setting of personalized FL with heterogeneous data, where each client aims to leverage the data of certain other clients to boost its own model performance. In this work, inspired by graph structure learning, we propose to learn a dynamic client network that tracks the graph data similarity across clients to guide model sharing along FL. Specifically, we rely on the marginal parameters of local GNNs to dynamically learn the client network, and refer to a set of fundamental graph properties to guide its learning. Extensive experiments on three real-world graph datasets demonstrate the consistent effectiveness of our two major proposed modules, which also mutually verify the effectiveness of each other.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

1. Add detailed pseudo-code of our methods as Algorithm 1.
2. Rewrite Equation 2 to the following format:
$$\theta^{t+1}\_i \xleftarrow{} \theta^t\_i + \mathop{AGGR}\_{j\in\mathbf{N}(i)}(\phi(\Delta\theta^t\_i,\Delta\theta^t\_j,w\_{ij})),$$ to better explain our aggregation process.
3. Add a new part to Section 3.2 named "Client network reconstruction", explaining the details of our graph reconstructor $GNN_{server}$. And remove the previous unclear descriptions of $GNN_{server}$ at the end of Section 3.1
4. Add Section 3.5, analyzing the overall complexity of GPFL (Previous Section 3.5 moved to Section 3.6).
5. Add two new baselines FED-PUB([1]) and FedSelect([2]) in our main experiment (Section 4.2 and Table 2)

[1] Jinheon Baek, Wonyong Jeong, Jiongdao Jin, Jaehong Yoon, and Sung Ju Hwang. Personalized subgraph federated learning. In International conference on machine learning, pp. 1396–1415. PMLR, 2023.

[2] Rishub Tamirisa, Chulin Xie, Wenxuan Bao, Andy Zhou, Ron Arel, and Aviv Shamsian. Fedselect: Personalized federated learning with customized selection of parameters for fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23985–23994, 2024.

Code: https://github.com/Jiachen2cc/Graph-Personalized-Federated-Learning

Assigned Action Editor: ~Shuiwang_Ji1

Submission Number: 4630