High-Resolution Medical Image Translation via Patch Alignment-Based Bidirectional Contrastive Learning

Xinyue Li

Abstract:

Pathology image assessment plays a crucial role in disease
diagnosis and treatment. In this study, we propose a Patch alignmentbased
Paired medical image-to-image Translation (PPT) model that
takes the Hematoxylin and Eosin (H&E) stained image as input and
generates the corresponding Immunohistochemistry (IHC) stained image
in seconds, which can bypass the laborious and time-consuming procedures
of IHC staining and facilitate timely and accurate pathology
assessment. First, our proposed PPT model introduces FocalNCE loss in
patch-wise bidirectional contrastive learning to ensure high consistency
between input and output images. Second, we propose a novel patch
alignment loss to address the commonly observed misalignment issue in
paired medical image datasets. Third, we incorporate content and frequency
loss to produce IHC stained images with finer details. Extensive
experiments show that our method outperforms state-of-the-art methods,
demonstrates clinical utility in pathology expert evaluation using
our dataset and achieves competitive performance in two public breast
cancer datasets. Lastly, we release our H&E to IHC image Translation
(HIT) dataset of canine lymphoma with paired H&E-CD3 and H&EPAX5
images, which is the first paired pathological image dataset with
a high resolution of 2048 × 2048. Our code and dataset are available at
https://github.com/coffeeNtv/PPT.