CT2RGB: Seeing The True Colors of Computed Tomography Images

Hung-Ju Liao, Jin-Siang Lin, Huan-Chih Wang, Min Sun

Keywords: Generative Model, Image-to-Image Translation, CT Image, Image Colorization

Abstract: Human anatomy plays a pivotal role in modern medicine as many diseases or pathological conditions take the form of anatomical derangements. Nowadays, state-of-the-art medical imaging technologies such as Computed Tomography (CT) are widely used to evaluate these conditions in a noninvasive fashion. However, these medical images are typically displayed as grayscale images, which require a well-trained professional to interpret. Color cues, which are sometimes essential to facilitate diagnosis and preoperative planning, are clearly lacking in these images. We hereby propose a framework to bridge the gap between CT images and cross-sectional cryosection represented by an RGB image. We formulate the problem as an image-to-image translation task, where both the CT and structural information corresponding to critical contours (i.e., fat and bone border) are the inputs, and the cross-sectional RGB image is the output. We train our model with an adversarial training scheme to overcome the lack of paired training input and output data. Moreover, two procedures are designed to force training focusing on the interior of the body and critical contours. Specially, we compute Sobel image gradient in the ranges of CT values corresponding to fat and bone. Although our model is trained on a small dataset due to a lack of training data, our approach is designed to generate realistic cross-sectional color images given unseen CT images. Our experimental results demonstrate the effectiveness of our proposed framework in both quantitative and qualitative aspects for both within the dataset and cross-dataset settings. To investigate the proposed framework's applicability, we also acquire feedback from both doctors and general people by conducting a human perceptual study. The feedback shows a strong potential for educational and clinical communication purposes between doctors and students, and doctors and patients, respectively.

Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.

Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.

Paper Type: both

Primary Subject Area: Image Synthesis

Secondary Subject Area: Application: Radiology

Confidentiality And Author Instructions: I read the call for papers and author instructions. I acknowledge that exceeding the page limit and/or altering the latex template can result in desk rejection.