Lesions, Latents, and Language: Interpreting Breast Ultrasound Features via Latent Probing and LLM-Driven Report Synthesis

ICLR 2026 Conference Submission17762 Authors

Keywords: Segmentation, Interpretability, Lesion Characteristics, Breast Lesions, Latent Space Probing

Abstract: Interpretability remains a critical bottleneck in the deployment of deep learning models in medical imaging. In this work, we present a novel framework that bridges deep visual representations with natural language through the lens of breast ultrasound lesion segmentation. Using the BUS-UCLM dataset, we train a supervised segmentation model and extract high-dimensional latent features that encode lesion characteristics. We then project these features into a low-dimensional latent space via t-SNE and identify visually coherent clusters. Each cluster is quantitatively characterized using lesion-level attributes such as size, boundary complexity, and class prevalence. To close the semantic gap between neural representations and clinical reasoning, we prompt large language models (LLMs) with these cluster-level summaries to generate human-interpretable natural-language descriptions of lesion types and patterns. Our experiments demonstrate that these language outputs align well with known clinical lesion types and that probing classifiers trained on latent features alone achieve strong diagnostic separation. This framework enables transparent, cluster-driven summarization of lesion types and offers an explainability interface between deep neural models and clinical end-users. Our results suggest a new path for integrating unsupervised vision-language synthesis into medical imaging pipelines without the need for textual ground-truth reports.

Primary Area: interpretability and explainable AI

Submission Number: 17762