Concept Flow Models: Anchoring Concept-Based Reasoning with Hierarchical Bottlenecks

Ya Wang, Adrian Paschke

Abstract: Concept Bottleneck Models (CBMs) enhance interpretability by projecting learned features into a human-understandable concept space. Recent approaches leverage vision-language models to generate concept embeddings, reducing the need for manual concept annotations. However, these models suffer from a critical limitation: as the number of concepts approaches the embedding dimension, information leakage increases, enabling the model to exploit spurious or semantically irrelevant correlations and undermining interpretability. In this work, we propose Concept Flow Models (CFMs), which replace the flat bottleneck with a hierarchical, concept-driven decision tree. Each internal node in the hierarchy focuses on a localized subset of discriminative concepts, progressively narrowing the prediction scope. Our framework automatically constructs decision hierarchies from visual embeddings, distributes semantic concepts at each hierarchy level, and trains differentiable concept weights through probabilistic tree traversal. Extensive experiments on diverse benchmarks demonstrate that CFMs match the predictive performance of flat CBMs, while substantially reducing effective concept usage and information leakage. Furthermore, CFMs yield stepwise decision flows that enable transparent and auditable model reasoning.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: Camera Ready Version: Due to institutional and affiliation-related constraints, the code and the Acknowledgment section have been intentionally omitted in the camera-ready version. These changes do not affect the methodology, experiments, conclusions or appendix of the paper.

Assigned Action Editor: ~Bryon_Aragam1

Submission Number: 5625