Digging Deeper: Learning Multi-Level Concept Hierarchies

Oscar Hill, Mateo Espinosa Zarlenga, Mateja Jamnik

Keywords: Explainable Artificial Intelligence, Concept-based Explainability, Concept Discovery, Concept Hierarchy, Concept Bottleneck Models, Concept Embedding Models, Sparse Autoencoders

Abstract: Concept-based models promise interpretability by explaining predictions with human-understandable concepts, but they typically rely on exhaustive annotations and treat concepts as flat and independent. Hierarchical Concept Embedding Models (HiCEMs) address this by modelling concept relationships, and Concept Splitting enables their use with only coarse annotations by discovering sub-concepts. However, both are restricted to shallow hierarchies. We overcome this limitation with *Deep-HiCEMs*, which model arbitrarily deep hierarchies, and *Multi-Level Concept Splitting* (MLCS), which uncovers multi-level concept hierarchies from only top-level supervision. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.

Submission Number: 193