Enhancing pulmonary nodule detection via cross-modal alignment

Yumeng Zhu, Yi Xu, Bingbing Ni, Jie Zhang, Xiaokang Yang

Abstract: Lack of large available datasets fully annotated is a fundamental bottleneck in pulmonary nodule detection, especially when the sensing equipment and the corresponding computed tomography (CT) images obtained are device dependent. This work presents a novel cross modal scheme, pursuing modal alignment, to facilitate our aggregate channel detector training. Named as multi-class cycle-consistent adversarial network (CycleGAN), our proposed framework utilizes a generative adversarial model to transfer nodule morphological characteristics from source modal to target modal, and we propose an end to end objective function to unify the transfer and detection procedures. The outputs of the two parts are combined with a dedicated fusion method for final classification. Extensive experimental results on 1948 scans of the private dataset demonstrate the proposed modal transfer method is very effective in data augmentation.