Flexible Segmentation for Rule Learning over Object Representations in Abstract Visual Reasoning

Johannes Langer, Ute Schmid

Keywords: Abstract Visual Reasoning, Raven’s Progressive Matrices, Image Segmentation, Representation Learning, Inductive Logic Programming, Neuro-Symbolic AI

TL;DR: We propose a novel framework for solving abstract visual reasoning problems by learning rules over representation of individual objects.

Abstract: The ability to reason in abstract domains is a key component of human intelligence, and is tested in many intelligence tests such as Raven's Progressive Matrices (RPM). We present ECo-NSR, a novel neuro-symbolic system which aims to explicitly separate perception and problem solving by sequentially layering instance segmentation, feature extraction and analogical reasoning. By separating these three components, the model additionally requires only segmentation masks as labels for training and can continuously improve by exchanging components for new developments in the fields of image segmentation, representation learning and symbolic reasoning. While end-to-end inference with the proposed system is currently not yet possible, our intermediary results indicate the viability of the proposed system and its transferability to other abstract visual reasoning domains.

Paper Track: Technical paper

Submission Number: 73