fmMAP: A Framework Reducing Site-Bias Batch Effect from Foundation Models in Pathology

Hai Cao Truong Nguyen, David Joon Ho

Keywords: foundation model, batch effect, image classification, histopathology

TL;DR: fmMAP leverages supervised UMAP to refine features from pathology foundation models, strongly mitigating batch effects and improving morphology, robustness, and accuracy for downstream tasks.

Abstract: Foundation models (FMs) in pathology are general-purpose models capturing heterogeneous morphological patterns on pathology images leveraged by a vast training dataset. Although FMs have demonstrated promising results in multiple downstream tasks such as classification and retrieval, confounding factors are also embedded in the features potentially causing inaccurate decisions. For example, we observe a batch effect where distinctive medical center signatures are displayed when clustering features from FMs. In this work, we propose Foundation Model-based Manifold Approximation Pipeline (fmMAP) to reduce the batch effect by adjusting features from FMs. Our framework employs supervised uniform manifold approximation (UMAP) to transform features generated by FMs into an optimal space. In this transformed space, characteristics of features of interest (i.e., biological features) are highlighted while other confounding factors are reduced. Experimental results on eight recent FMs show that raw features from the FMs are shown to be unrobust, but fmMAP transforms features to become robust on all FMs according to the robustness index. In addition, fmMAP reduces average balanced accuracy for site prediction and improves average balanced accuracy for tissue type classification achieving more than 96\% in publicly available datasets. We expect fmMAP framework will help FMs identify essential pathologic features that would enhance performance on downstream tasks. The code will be made publicly available upon acceptance.

Submission Number: 19