Learning Label-Efficient Interpretable Medical Image Diagnosis via Semi-supervised Hypergraph Concept Bottleneck Model

Yijun Yang, Ruiqiang Xiao, Lijie Hu, Angelica I Aviles-Rivero, WuYunzhu, Jing Qin, Lei Zhu

Keywords: Concept Bottleneck Model, Medical Image Diagnosis, Interpretability, XAI

Abstract:

Deep learning has revolutionized medical image analysis, delivering exceptional diagnostic accuracy across diverse applications.
Yet, the lack of interpretability in its decision-making hinders clinical adoption, particularly in high-stakes medical contexts where transparency is paramount for trustworthiness.
For example, in Placenta Accreta Spectrum (PAS), subtle cues in ultrasound imaging challenge reliable diagnosis, rendering black-box models untrustworthy for accurate scoring.
To address this, Concept Bottleneck Models (CBMs) offer a promising avenue by embedding clinically meaningful intermediate concepts into the diagnosis pipeline, enabling clinicians to scrutinize and refine model outputs.
However, conventional CBMs falter in capturing complex inter-concept dependencies and demand costly, expert-driven concept annotations, limiting their scalability.
This study introduces a novel semi-supervised CBM framework designed for medical imaging, which leverages dual-level hypergraph learning to model high-order concept dependencies and generate domain-adaptive pseudo-labels.
Our approach achieves superior interpretability and performance by integrating a concept-level hypergraph for enhanced reasoning and an image-level hypergraph for robust pseudo-label generation.
Experiments on a newly annotated PAS ultrasound dataset and a breast ultrasound public dataset demonstrate the effectiveness of the proposed concept label-efficient interpretable framework.
Its universality is further validated on the dermoscopic image dataset SkinCon.
The core code is available at the appendix.

Primary Area: interpretability and explainable AI

Submission Number: 6910