Subgraph Permutation Equivariant Networks
Abstract: In this work we develop a new method, named Sub-graph Permutation Equivariant Networks (SPEN), which provides a framework for building graph neural networks that operate on sub-graphs, while using a base update function that is permutation equivariant, that are equivariant to a novel choice of automorphism group. Message passing neural networks have been shown to be limited in their expressive power and recent approaches to over come this either lack scalability or require structural information to be encoded into the feature space. The general framework presented here overcomes the scalability issues associated with global permutation equivariance by operating more locally on sub-graphs. In addition, through operating on sub-graphs the expressive power of higher-dimensional global permutation equivariant networks is improved; this is due to fact that two non-distinguishable graphs often contain distinguishable sub-graphs. Furthermore, the proposed framework only requires a choice of $k$-hops for creating ego-network sub-graphs and a choice of representation space to be used for each layer, which makes the method easily applicable across a range of graph based domains. We experimentally validate the method on a range of graph benchmark classification tasks, demonstrating statistically indistinguishable results from the state-of-the-art on six out of seven benchmarks. Further, we demonstrate that the use of local update functions offers a significant improvement in GPU memory over global methods.
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=X1XDtko96e&referrer=%5BTMLR%5D(%2Fgroup%3Fid%3DTMLR)
Changes Since Last Submission: We would like to thank the AC and reviewers for choosing to accept the paper and for the valuable discussion period. We note the final changes to the paper here for the camera ready version:
Chapter 1
* Minor update to the contributions of the paper to make the wording more precise based on the feedback from reviewers.
* Additional sentence to assist in notation explanation and distinction between our work and background literature.
Chapter 3
* Added citation in definition of concrete graphs as requested by reviewer.
* Updated initial explanation of the model to be more clear following review discussion.
* Updated figures to make the model clearer.
Chapter 4
* Updates to literature based on recent papers in some definitions.
Chapter 5
* Updates to explanations of specific architecture used in experiments to make clear that automorphism groups are permutation groups as requested following the discussion with reviewer N3Bm.
Append A.2
* Updates to literature based on recent papers in some definitions.
Appendix A.6
* Added algorithmic description of the model to make the model clearer.
Changes made since the last submission also include changes made during this submission, which were requested by reviewers from the prior submission. These are detailed below:
(1) Changes regarding issues with the presentation of the architecture:
Chapter 3 - Subgraph Permutation Equivariant Networks (SPEN)
* Updated to introduce new figures and point to relevant sections in the appendix.
* Updated 3.1 Re-ordered definitions, added new definitions and further details for automorphisms, naturality, and permutations.
* Updated Figure 2 and moved into section 3 to provide a clearer high-level overview of the architecture. Including a larger graph example, details that the number of automorphism groups shown is an example, a clearer presentation of the update functions (linear maps) to make the distinction between a GNN and the automorphism equivariant layer presented here, further details on the group representations used, a more clear presentation of the bag of bags of sub-graphs, better details on the motivation for splitting into bags of sub-graphs. This should also address confusion over whether multiple layers can be stacked.
* New Figure 3 which details a breakdown of a general automorphism equivariant layer.
* New Figure 4 which details a further breakdown of a function in Fig3. This should aid an understanding of the feature spaces (vector spaces) and how the different group representations act on these.
* Re-positioned old Figure 3 which is now Figure 5. It now fits in as a further breakdown from Figure 4.
* Added further text in Section 3.2 to better motivate the choice of sub-graph selection policy and the benefits it brings.
* Re-written Section 3.3 such that it flows better and should make the architecture clearer. This includes re-ordering the sub-sections within the section and re-writing each section.
* Section 3.3.1 is what was Section 3.3.2 and now contains further text to clarify the automorphism equivariance of the SPEN layers. Also how the sub-graph selection policy fits well with the automorphism equivariance choice.
* Section 3.3.2 is what was 3.3.3 and now contains further text which aids the understanding of the insert and extract to share information between sub-graphs.
* Section 3.3.3. Is what was 3.3.1 and now makes clear how the permutation equivariant update functions fit into the architecture and do not conflict with the overall automorphism equivariance.
Chapter A.1 - Mathematical Background
* Added in further definitions.
(2) Changes regarding issues with the discussion of previous work and positioning of the method:
Chapter 2 - Background
* Added in missing citations.
Chapter A.3 Previous Methods
* Added in a comparison with Autobahn
* Added in a comparison with ESAN
* Added in a comparison with $k$-reconstruction GNNs
* Added in a comparison with GRAPE
(3) Changes regarding issues with the discussion of the statistical significance of the results:
* Added a new Chapter A.7 for further discussion of the results.
* Chapter A.7 added in plots showing the distributions of the results.
* Chapter A.7 added in statistical significance of the results.
* Chapter 5 updated results table to show the statistical significance tests by highlighting statistically indistinguishable SOTA results in grey background.
* Chapter 5.1 added in reference to new appendix section A.7.
* Chapter 5.1 updated text on the results discussion based on the statistical significance analysis.
* Chapter 1 changes to contributions text to better reflect the results presented.
* Abstract updated based on statistical significance tests.
Assigned Action Editor: ~Guillaume_Rabusseau1
Submission Number: 324
Loading