Relational Composition in Neural Networks: A Survey and Call to Action
Keywords: interpretability, feature, relation
TL;DR: A survey and analysis of possible feature binding mechanisms in neural networks, and a call to action for more research.
Abstract: Many neural nets appear to represent data as linear combinations of ``feature vectors.'' Algorithms for discovering these vectors have seen impressive recent success. However, we argue that this success is incomplete without an understanding of relational composition: how (or whether) neural nets combine feature vectors to represent more complicated relationships. To facilitate research in this area, this paper offers a guided tour of various relational mechanisms that have been proposed, along with preliminary analysis of how such mechanisms might affect the search for interpretable features. We end with a series of promising areas for empirical research, which may help determine how neural networks represent structured data.
Submission Number: 76
Loading