Graph neural networks with polynomial activations have limited expressivity

Sammy Khalife

Abstract:

The expressivity of Graph Neural Networks (GNNs) can be entirely characterized by appropriate fragments of the first order logic. Namely, any query of the two variable fragment of graded modal logic (GC2) interpreted over labeled graphs can be expressed using a GNN whose size depends only on the depth of the query (uniformity).
As pointed out by [2020, Barcelo, 2021, Grohe], this description holds for a family of activation functions of the underlying neural network, leaving the possibibility for a hierarchy of logics uniformly expressible by GNNs depending on the chosen activation function. In this article, we show that such hierarchy indeed exists by proving that GC2 queries cannot be uniformly expressed by GNNs with polynomial activations and aggregations. This implies a separation between the expressivity of GNNs with polynomial and those with non polynomial activations (such as Rectified Linear Units) and partially answers an open question formulated by .