Advancing H&E-to-IHC Virtual Staining with Task-Specific Domain Knowledge for HER2 Scoring

Qiong Peng, Weiping Lin, Yihuang Hu, Ailisi Bao, Chenyu Lian, Weiwei Wei, Meng Yue, Jingxin Liu, Lequan Yu, Liansheng Wang

Abstract: The assessment of HER2 expression is crucial in diagnosing breast cancer. Staining pathological tissues with immunohistochemistry (IHC) is a critically pivotal step in the assessment procedure, while it is expensive and time-consuming. Recently, generative models have emerged as a novel paradigm for virtual staining from hematoxylin-eosin (H&E) to IHC. Unlike traditional image translation tasks, virtual staining in IHC for HER2 scoring requires greater attention to regions like nuclei and stained membranes, informed by task-specific domain knowledge. Unfortunately, most existing virtual staining methods overlook this point. In this paper, we propose a novel generative adversarial network (GAN) based solution that incorporates specific knowledge of HER2 scoring, i.e., nuclei distribution and membrane staining intensity. We introduce a nuclei density estimator to learn the nuclei distribution and thus facilitate the cell alignment between the real and generated images by an auxiliary regularization branch. Moreover, another branch is tailored to focus on the stained membranes, ensuring a more consistent membrane staining intensity. We collect RegH2I, a dataset comprising 2592 pairs of registered H&E-IHC images and conduct extensive experiments to evaluate our approach, including H&E-to-IHC virtual staining on internal and external datasets, nuclei distribution and membrane staining intensity analysis, as well as downstream tasks for generated images. The results demonstrate that our method achieves superior performance than existing methods. Code and dataset are released at https://github.com/balball/TDKstain.