Early Fusion of H&E and IHC Histology Images for Pediatric Brain Tumor Classification

Christoforos Spyretos, Iulian Emil Tampu, Nadieh Khalili, Juan Manuel Pardo Ladino, Per Nyman, Ida Blystad, Anders Eklund, Neda Haj-Hosseini

Keywords: pediatric brain tumour, H\&E, immunohistochemistry (IHC), Ki-67, GFAP, computational pathology, early fusion, UNI, CLAM, foundation model

Abstract: This study explores the application of computational pathology to analyze pediatric brain tumors utilizing hematoxylin and eosin (H\&E) and immunohistochemistry (IHC) histology images. Experiments were conducted on H\&E images for predicting tumor diagnosis and fusing them with unregistered IHC images to investigate potential improvements. Patch features were extracted using UNI, a vision transformer (ViT) model trained on H\&E data, and whole slide classification was achieved using the attention-based multiple instance learning CLAM framework. In the astrocytoma tumor classification, early fusion of the H\&E and IHC significantly improved the differentiation between tumor grades (balanced accuracy: 0.82 ± 0.05 vs 0.84 ± 0.05). H\&E only stain had a balanced accuracy of 0.79 ± 0.03 for the overall classes without any improvement obtained when fused with IHC. The findings highlight the potential of using multi-stain fusion to advance the diagnosis of pediatric brain tumors, however, further fusion methods should be investigated to explore the potentials.

Submission Number: 18