TESH-GCN: Text Enriched Sparse Hyperbolic Graph Convolutional Networks
Abstract: Heterogeneous networks, which connect informative nodes containing semantic information with different edge types, are routinely used to store and process information in various realworld applications. Graph Neural Networks (GNNs) and their hyperbolic variants provide a promising approach to encode such networks in a low-dimensional latent space through neighborhood aggregation and hierarchical feature extraction, respectively. However, these approaches typically ignore metapath structures and the available semantic information. Furthermore, these approaches are sensitive to the noise present in the training data. To tackle these limitations, in this paper, we propose Text Enriched Sparse Hyperbolic Graph Convolution Network (TESH-GCN). In TESH-GCN, we use semantic node information to identify relevant nodes and extract their local neighborhood and graph-level metapath features. This is done by applying a reformulated hyperbolic graph convolution layer to the sparse adjacency tensor using the semantic node information as a connection signal. These extracted features in conjunction with semantic features from the language model (for robustness) are used for the final downstream tasks. Experiments on various heterogeneous graph datasets show that our model outperforms the state-of-the-art approaches by a large margin on the task of link prediction. We also report a reduction in both the training time and model parameters compared to the existing hyperbolic approaches through a reformulated hyperbolic graph convolution. Furthermore, we illustrate the robustness of our model by experimenting with different levels of simulated noise in both the graph structure and text, and also, present a mechanism to explain TESH-GCN’s prediction by analyzing the extracted metapaths.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: - Prev: “...nodes containing text…”.
New: “...nodes containing semantic information…”
- Prev: “...(TESH-GCN) to capture the graph’s metapath structures using semantic signals and further improve prediction in large heterogeneous graphs. In TESH-GCN, we extract semantic node information, which successively acts as a connection signal to extract relevant nodes’ local neighborhood and graph-level metapath features from the sparse adjacency tensor in a reformulated hyperbolic graph convolution layer…”.
New: “..(TESH-GCN). In TESH-GCN, we use semantic node information to identify relevant nodes and extract their local neighborhood and graph-level metapath features. This is done by applying a reformulated hyperbolic graph convolution layer to the sparse adjacency tensor using the semantic node information as a connection signal.…”
- Previous: “... for the final downstream task.”
New: “... for the final downstream tasks.”
- Previous: “... the current state-of-the-art approaches…”.
New: “... the state-of-the-art approaches…”
- Prev: “...nodes containing text…”.
New: “...nodes containing semantic information…”
- Prev:”...to computational constraints. The adjacency tensor of a heterogeneous graph can be used to extract both metapath information as well as aggregate local neighborhood features…”.
New: “...to computational constraints, i.e., only a local k-hop neighborhood of a heterogeneous graph’s node is considered while learning the metapaths. However, global metapaths can capture long-term relations between the nodes. To learn metapaths, we need to encode the path between two nodes and the semantic information contained in the path. Thus, The adjacency tensor of a heterogeneous graph with a semantic signal can be used to extract both metapath information as well as aggregate local neighborhood features…”
- Prev: “...We detail…”.
New: “...we detail…”
- Prev:”...each feature…”.
New:”...each dimension…”
- Previous: “...(TESH-GCN) to capture the graph’s metapath structures using semantic signals and further improve prediction in large heterogeneous graphs. In TESH-GCN, we extract semantic node information, which successively acts as a connection signal to extract relevant nodes’ local neighborhood and graph-level metapath features from the sparse adjacency tensor in a reformulated hyperbolic graph convolution layer…”.
New: “..(TESH-GCN). In TESH-GCN, we use semantic node information to identify relevant nodes and extract their local neighborhood and graph-level metapath features. This is done by applying a reformulated hyperbolic graph convolution layer to the sparse adjacency tensor using the semantic node information as a connection signal.…”
- Prev: “...For an input adjacency tensor with elements $x \in A_k$,..”
New: “...For the $d^{th}$ input adjacency matrix with elements $x \in A_k[d]$,..”
- Added to related work: “Another line of work specifically tailored for heterogeneous graphs Fu et al. (2020); Yang et al. (2021); Hu et al. (2020), Yun et al. (2019), and Wang et al. (2019) utilizes the rich relational information through metapath aggregation.”
Added to references: “Yun, S., Jeong, M., Kim, R., Kang, J., & Kim, H. J. (2019). Graph transformer networks. Advances in neural information processing systems, 32. Wang, Xiao, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. "Heterogeneous graph attention network." In The world wide web conference, pp. 2022-2032. 2019.”
- Added to Definition 2: “where dist(a,b) is the edge distance between nodes a and b in a homogenized version of graph ”
- Prev: “hierarchy of the graph G”, “hyperbolicity implies G“.
New:“hierarchy of the graph $\mathcal{G}$”, “hyperbolicity implies $\mathcal{G}$“
- Added to Section 4.2 Baselines: “In the case of graph-based methods, we utilize the node features provided in the dataset as default, else we utilize fixed-semantic vectors from the pretrained LM Song et al. (2020).”
Assigned Action Editor: ~Guillaume_Rabusseau1
Submission Number: 571
Loading