Making AI Think Lean: Sparse Concept Bottleneck Models for Interpretable Decisions
Abstract: Concept Bottleneck Models (CBMs) provide a promising approach to enhance interpretability in machine learning models. These models excel at disentangling and anchoring visual representations into human-comprehensible concepts. We present an approach to enhance visual model interpretability by incorporating natural language text directly extracted from images. We introduce the Visual-Rationale Alignment Learning (VIRAL) framework, which incorporates natural language text directly extracted from images to improve the interpretability of visual models. Through the use of the Gumbel-Sinkhorn algorithm for sparse alignment and extensive experimental analysis, VIRAL demonstrates its effectiveness in providing human-understandable explanations for predictions, contributing to the development of more transparent and trustworthy AI multimodal systems.
Paper Type: Short
Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond
Research Area Keywords: Concept bottleneck models
Contribution Types: Model analysis & interpretability
Languages Studied: English
Submission Number: 95
Loading