Structure Matters: Deciphering Neural Network's Properties from its Structure

Shashata Sawmya, Md Toki Tahmid, Gourab Saha, Arpita Saha, Nir N Shavit, Lu Mi

Keywords: Neural Representation, Graph Representation Learning, Neural Network Analysis

TL;DR: Encoding a neural network's computational graph can reveal valuable insights about its properties based solely on its structure.

Abstract: Neural networks; both biological and artificial, are commonly represented as graphs with connections between neurons, yet there is little understanding of the relationship between their graph structure and computational properties. Neuroscientists are trying to answer this question in biological neural networks or connectomes; however, there is a big opportunity to explore this in the vast domain of artificial neural networks. We present StructureReps, an architecture-agnostic framework for encoding neural networks as graphs using graph representation learning. By capturing key structural properties, StructureReps reveals strong correlations between network structure and task performance across various architectures. Additionally, this framework has potential applications beyond the decoding of neural network properties.

Submission Number: 73