Attention on Abstract Visual Reasoning

Lukas Hahne, Timo Lüddecke, Florentin Wörgötter, David Kappel

TL;DR: Introducing Attention Relation Network (ARNe) that combines features from WReN and Transformer Networks.

Abstract: Attention mechanisms have been boosting the performance of deep learning models on a wide range of applications, ranging from speech understanding to program induction. However, despite experiments from psychology which suggest that attention plays an essential role in visual reasoning, the full potential of attention mechanisms has so far not been explored to solve abstract cognitive tasks on image data. In this work, we propose a hybrid network architecture, grounded on self-attention and relational reasoning. We call this new model Attention Relation Network (ARNe). ARNe combines features from the recently introduced Transformer and the Wild Relation Network (WReN). We test ARNe on the Procedurally Generated Matrices (PGMs) datasets for abstract visual reasoning. ARNe excels the WReN model on this task by 11.28 ppt. Relational concepts between objects are efficiently learned demanding only 35% of the training samples to surpass reported accuracy of the base line model. Our proposed hybrid model, represents an alternative on learning abstract relations using self-attention and demonstrates that the Transformer network is also well suited for abstract visual reasoning.

Code: https://drive.google.com/file/d/19fNqoqULy1rPOf38YQ2OsOFkDlzhec-i/view?usp=sharing

Keywords: Transformer Networks, Self-Attention, Wild Relation Networks, Procedurally Generated Matrices

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1911.05990/code)