Learning Hierarchically using Formal Concepts

Deepika Vemuri, Sayanta Adhikari, Ankit Saha, Vineeth N. Balasubramanian

Keywords: Concepts, interpretability, formal concept analysis, lattice

TL;DR: We propose a method to distribute concepts through the network using formal concept analysis.

Abstract: Learning semantics is crucial for deep learning models to be trustworthy and more aligned with human-like reasoning. Concept-based models offer a promising approach by learning classes in terms of interpretable semantic abstractions. However, two key limitations in such an approach are: 1. concepts of varying degrees of granularity are all learned in the same layer, with the same number of parameters, 2. as the concept layer comes right before the classifier, the network that the concepts are learned from still largely remains a black-box. In order to address these challenges, we propose a method for distributing concepts across the network. We use Formal Concept Analysis (FCA) to build a hierarchy that informs where in the network specific concepts are learned based on their level of abstraction. Our experiments on real-world datasets demonstrate the effectiveness of our approach by introducing a way to obtain staged semantically grounded representations.

Submission Number: 21