Going beyond H&E and Oncology: how do Histopathology Foundation Models perform for multi-stain IHC & Immunology?

Amaya Gallagher-Syed, ELENA PONTARINI, Myles J. Lewis, Gregory Slabaugh, Michael R Barnes

Keywords: Histopathology Foundation Models, Immunohistochemistry, Autoimmune, OOD Generalisation

TL;DR: Histopathology foundation models trained on cancer H&E images do not significantly outperform general computer vision models when applied to autoimmune IHC classification tasks.

Abstract: This study evaluates the generalisation capabilities of state-of-the-art histopathology foundation models on out-of-distribution (OOD) multi-stain autoimmune Immunohistochemistry (IHC) datasets. We compare 13 feature extractor models, including ImageNet-pretrained networks, and histopathology foundation models trained on both public and proprietary data, on Rheumatoid Arthritis (RA) subtyping and Sjogren's Disease (SD) diagnostic tasks. Using a simple Attention-Based Multiple Instance Learning classifier, we assess the transferability of learned representations from cancer H\&E images to autoimmune IHC images. Contrary to expectations, histopathology-pretrained models did not significantly outperform ImageNet-pretrained models. Furthermore, there was evidence of both autoimmune feature misinterpretation and biased feature importance. Our findings highlight the challenges in transferring knowledge from cancer to autoimmune histopathology and emphasise the need for careful evaluation of AI models across diverse histopathological tasks. The code to run this benchmark is available at https://github.com/AmayaGS/ImmunoHistoBench

Submission Number: 94