Semantic Mosaicing of Histo-Pathology Image Fragments using Visual Foundation Models

Stefan Brandstätter, Maximilian Köller, Philipp Seeböck, Alissa Blessing, Felicitas Oberndorfer, Svitlana Pochepnia, Helmut Prosch, Georg Langs

Keywords: Whole-Mount Sectioning (WMS), UNI, Histopathology, Image Stitching, Foundation Model

TL;DR: SemanticStitcher is an automated method that uses semantic features from visual foundation models to robustly stitch fragmented histopathology images into accurate whole-mount slides, outperforming existing boundary-based state-of-the-art methods.

Abstract: In histopathology, tissue samples are often larger than a standard microscope slide, making stitching of multiple fragments necessary to process entire structures such as tumors. Automated stitching is a prerequisite for scaling analysis, but is challenging due to possible tissue loss during preparation, inhomogeneous morphological distortion, staining inconsistencies, missing regions due to misalignment on the slide, or frayed tissue edges. This limits state-of-the-art stitching methods using boundary shape matching algorithms to reconstruct artificial whole mount slides (WMS). Here, we introduce SemanticStitcher using latent feature representations derived from a visual histopathology foundation model to identify neighboring areas in different fragments. Robust pose estimation based on a large number of semantic matching candidates derives a mosaic of multiple fragments to form the WMS. Experiments on three different histopathology datasets demonstrate that SemanticStitcher yields robust WMS mosaicing and consistently outperforms the state of the art in correct boundary matches.

Submission Number: 22