Dynamic Locally Linear Graph Learning for Geometry-Aware GNNs

Kailong Zhao, Zhengquan Luo, Zeke Xie, zhiqiang xu

Keywords: graph neural networks, dynamic graph construction, locally linear embedding, variational autoencoders

TL;DR: We propose a VAE-LLE framework for dynamic graph construction that preserves local geometry, reshapes spectra, and mitigates over-smoothing in GNNs.

Abstract: The accuracy of GNN-based node classification depends critically on the quality of the constructed graph. Common heuristics such as $k$NN or Gaussian kernels rely on Euclidean distances, which in high dimensions fail to capture manifold structure and often introduce spurious cross-class edges. During message passing, these edges propagate inconsistent signals and, with deeper layers, lead to over-smoothing where node embeddings become indistinguishable. To address these issues, we propose VLGNN, an end-to-end framework that integrates graph construction with GNN training. A variational autoencoder (VAE) encodes the full data distribution into a latent space, thereby capturing global structure, while an LLE-inspired module refines the graph by enforcing local manifold consistency. The graph structure is updated jointly with the VAE-based encoder and the GNN-based classifier, allowing neighborhoods to adapt to the learned representations and to preserve meaningful within-class relations during training. Spectral analysis further shows that our LLE-based method enlarges Laplacian eigengaps and reduces inter-class conductance, indicating weaker cross-class propagation and alleviation of over-smoothing. On standard benchmarks, VLGNN achieves higher accuracy with reduced cross-class mixing.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 17632