Boltzmann Graph Networks: Efficient Energy-Based Framework for Graph Representation Learning

Tarik Lemkadem, Iyad Lahsen-cherif, EL KHADIMI AHMED, NAJID Abdellah

Keywords: Graph Neural Networks, Energy-Based Learning, Restricted Boltzmann Machines, Node Representation Learning

TL;DR: Boltzmann Graph Networks (BGNs) introduce an efficient, energy-based GNN for robust node representation learning in citation network node classification.

Abstract:

With the rapid growth of interconnected data, graph-structured representations have become essential for modeling complex relational systems. Graph Neural Networks (GNNs) are widely used for leveraging semantic information encoded in nodes and edges. Current GNN models encounter multiple challenges, including over-smoothing, limited ability to model long-range dependencies, and high computational cost on complex graph structures.

This paper introduces the Boltzmann Graph Network (BGN), a novel and efficient GNN architecture that integrates energy-based probabilistic modeling with deterministic graph convolution. By conceptualizing the graph as an energy landscape, BGN employs a Boltzmann-inspired energy function to capture intricate node--edge interactions, enabling robust representation learning.
The use of $k$-step persistent contrastive divergence, while remaining compatible with gradient-based optimization, mitigates over-smoothing and enhances long-range dependency modeling.

Comprehensive evaluations on citation-network benchmarks show that BGN achieves state-of-the-art performance on both random and Geom-GCN splits, with test accuracies of 88.0\% (Cora), 75.6\% (CiteSeer), and 85.6\% (PubMed) on random splits. On Geom-GCN splits, the model attains 85.8\% (Cora), 75.5\% (CiteSeer), and 88.2\% (PubMed), demonstrating consistent improvements over existing methods.

To further assess generalization and robustness, we evaluate BGN on heterophilous benchmarks (Texas, Wisconsin, Cornell, and Actor), where it consistently outperforms homophily-oriented baselines (GCN, GAT, APPNP) and remains competitive with specialized heterophily-aware models. On large-scale evaluation using ogbn-arxiv, BGN maintains competitive performance while exhibiting a lower memory footprint compared to deep propagation-based methods.

These results highlight BGN’s scalability, robustness, and efficiency, positioning it as a powerful framework for advancing graph-based learning across diverse applications.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 14037