LLM-mediated pathology models for robust cross-institution generalization

ICLR 2026 Conference Submission15614 Authors

Keywords: pathology foundation models, computational pathology, batch effects, multimodal large language models, vision-language models

TL;DR: We introduce an LLM-mediated pathology model that uses multimodal LLMs to generate robust, generalizable embeddings from histology images, mitigating batch effects across institutions without requiring large-scale histology-specific pretraining.

Abstract: Pathology foundation models (PFMs) have shown strong potential across clinical and scientific applications. Their performance, however, is often limited by batch effects, which are non-biological variations across tissue source institutions (TSIs) that distort feature representations and reduce generalization. Existing mitigation methods, such as stain normalization, have limited success in addressing these high-dimensional and complex artifacts. We introduce the General-purpose LLM-Mediated Pathology model (GLMP), a novel framework that generates robust numerical embeddings from histology image patches by first converting them into text descriptions. By leveraging pretrained multimodal large language models (MLLMs) and text encoders, GLMP prioritizes genuine biological signals over TSI-specific signatures and improves cross-TSI generalization compared to existing PFMs. Our findings demonstrate the effectiveness of broad-domain, non-specialized MLLMs in computational pathology and provide an alternative framework for developing versatile, generalizable, and robust pathology models that do not require large-scale, histology-specific pretraining data. Code is provided in Supplementary Materials for reproducibility and will be released to the public upon paper acceptance.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 15614