Correlation via Synthesis: End-to-end Image Generation and Radiogenomic Learning Based on Generative Adversarial NetworkDownload PDF

Published: 18 Apr 2020, Last Modified: 05 May 2023MIDL 2020Readers: Everyone
Abstract: Radiogenomic map linking image features and gene expression profiles has great potential for non-invasively identifying molecular properties of a particular type of disease. Conventionally, such map is produced in three independent steps: 1) gene-clustering to metagenes, 2) image feature extraction, and 3) statistical correlation between metagenes and image features. Each step is separately performed and relies on arbitrary measurements without considering the correlation among each other. In this work, we investigate the potential of an end-to-end method fusing gene code with image features to generate synthetic pathology image and learn radiogenomic map simultaneously. To achieve this goal, we develop a multi-conditional generative adversarial network (GAN) conditioned on both background images and gene expression code, synthesizing the corresponding image. Image and gene features are fused at different scales to ensure both the separation of pathology part and background, as well as the realism and quality of the synthesized image. We tested our method on non-small cell lung cancer (NSCLC) dataset. Results demonstrate that the proposed method produces realistic synthetic images, and provides a promising way to find gene-image relationship in a holistic end-to-end manner.
TL;DR: Correlation via Synthesis: Image Generation and Radiogenomic Learning
Track: full conference paper
Keywords: Image synthesis, Radiogenomic learning, Multi-conditional GAN
Paper Type: methodological development
Source Latex: zip
Presentation Upload: zip
Presentation Upload Agreement: I agree that my presentation material (videos and slides) will be made public.
10 Replies