Learning Invariants through Soft Unification

Nuri Cingillioglu, Alessandra Russo

Keywords: representation learning, neural networks, unification

TL;DR: End-to-end learning of invariant representations with variables across examples such as if someone went somewhere then they are there.

Abstract: Human reasoning involves recognising common underlying principles across many examples by utilising variables. The by-products of such reasoning are invariants that capture patterns across examples such as "if someone went somewhere then they are there" without mentioning specific people or places. Humans learn what variables are and how to use them at a young age, and the question this paper addresses is whether machines can also learn and use variables solely from examples without requiring human pre-engineering. We propose Unification Networks that incorporate soft unification into neural networks to learn variables and by doing so lift examples into invariants that can then be used to solve a given task. We evaluate our approach on four datasets to demonstrate that learning invariants captures patterns in the data and can improve performance over baselines.

Code: https://drive.google.com/file/d/1Ema_awqOoOn-Xd4aTkkwOQz1r26QqZ4A/view?usp=sharing

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1909.07328/code)