Rule Based Learning with Dynamic (Graph) Neural Networks
Keywords: graph neural networks, dynamic neural networks, rule based learning
TL;DR: We introduce a new type of rule based neural network layer that is able to dynamically encode expert knowledge and show an application improving graph neural networks.
Abstract: A common problem of classical neural network architectures is that additional information or expert knowledge cannot be naturally integrated into the learning process.
To overcome this limitation, we propose a two-step approach consisting of (1) generating formal rules from knowledge and (2) using these rules to define rule based layers -- a
new type of dynamic neural network layer.
The focus of this work is on the second step, i.e., rule based layers that are designed to dynamically arrange learnable parameters in the weight matrices and bias vectors for each input sample following a formal rule.
Indeed, we prove that our approach generalizes classical feed-forward layers such as fully connected and convolutional layers by choosing appropriate rules.
As a concrete application we present rule based graph neural networks (RuleGNNs) that are by definition permutation equivariant and able to handle graphs of arbitrary sizes.
Our experiments show that RuleGNNs are comparable to state-of-the-art graph classifiers using simple rules based on the Weisfeiler-Leman labeling and pattern counting.
Moreover, we introduce new synthetic benchmark graph datasets to show how to integrate expert knowledge into RuleGNNs making them more powerful than ordinary graph neural networks.
Primary Area: Graph neural networks
Submission Number: 6411
Loading