HIGH SKIP NETWORKS: A HIGHER ORDER GENERALIZATION OF SKIP CONNECTIONSDownload PDF

02 Mar 2022, 12:21 (edited 19 Apr 2022)GTRL 2022 PosterReaders: Everyone
  • Keywords: Higher Order Networks, Simplicial Complex, Graph Neural Networks
  • TL;DR: Skip connections that propogate signals through higher order structures in simplicial complex domains can aid in creation of deep higher order networks that generalize well
  • Abstract: We present High Skip Networks (HSNs), a higher order generalization of skip connection neural networks to simplicial complexes. HSNs exploit higher order structure encoded in a simplicial domain by creating multiple feed-forward paths of signals computed over the input complex. Some feed-forward paths may propagate the signal through various higher order structures; e.g., if we want to propagate signals over edges, some feed-forward paths may go from edges to triangles and then back to edges. Similar to the Euclidean skip connection networks, all paths are combined together at the end by addition or concatenation. We demonstrate the effectiveness of HSNs on synthetic and real datasets. Our preliminary results show that HSNs lead to a statistically significant improvement in the generalization error when compared to base models without high skip components.
1 Reply

Loading