Optimal Node Embedding Dimension Selection Using Overall Entropy

Xinrun Xu, Zhiming Ding, Yurong Wu, Jin Yan, Shan Jiang, Qinglong Cui

Abstract: Graph node embedding learning has gained significant attention with the advancement of graph neural networks (GNNs). The essential purpose of graph node embedding is down-scaling high-dimensional graph features to a lower-dimensional space while maximizing the retention of original structural information. This paper focuses on selecting the appropriate graph node embedding dimension for hidden layers, ensuring the effective representation of node information and preventing overfitting. We propose an algorithm based on the entropy minimization principle, called Minimum Overall Entropy (MOE), which combines graph node structural information and attribute information. We refer to one-dimensional and multi-dimensional structural entropy (MDSE) as a graph’s structural entropy. A novel algorithm combines graph Shannon entropy, MDSE, and prior knowledge for faster convergence of optimal MDSE. We introduce an inner product-based metric, attribute entropy, to quantify node characteristics and simplify its calculation. Extensive experiments on Cora, Citeseer, and Pubmed datasets reveal that MOE, requiring just one computation round, surpasses baseline GNNs.