Information-Theoretic Foundations and Advances in Graph Machine Learning: A Comprehensive Survey

Qingyun Sun, Yi Huang, Haonan Yuan, Xingcheng Fu, Yisen Gao, Jia Wu, Shujian Yu, Angsheng Li, Jianxin Li, Philip S. Yu

Abstract: Graph Machine Learning (GML) has emerged as a powerful paradigm for modeling complex relational data across diverse domains. However, the intrinsic irregularity, high-dimensional dependencies, and heterogeneity of graph structures pose fundamental challenges to representation learning and model generalization. Information theory provides a principled framework to quantify uncertainty, capture interdependencies, and guide model design in analyzing the representational and algorithmic foundations of GML. Information-theoretic graph learning methods have shown impressive achievements over the past years. This survey provides a comprehensive and principled review of GML through the unifying and rigorous lens of information theory. We comprehensively survey over 200 methods and provide a systematic review that advances information theory for GML comprising three progressive layers: (i) entropy-based measures for quantifying uncertainty and structural complexity in graphs, (ii) mutual information and divergence for capturing interdependencies and distributional discrepancies, and (iii) information-theoretic principles for guiding model design. By systematically connecting data characterization, relational analysis, and learning objectives, this survey offers an integrative perspective that advances the understanding of graph machine learning. We further identify emerging frontiers, theoretical gaps, and open challenges, laying the foundation for future developments at the intersection of GML and information theory.

External IDs:doi:10.36227/techrxiv.176800867.72049627/v1