A Segmentation-Assisted Model for Universal Lesion Detection with Partial Labels

Fei Lyu, Baoyao Yang, Andy J. Ma, Pong C. Yuen

Abstract: Developing a Universal Lesion Detector (ULD) that can detect various types of lesions from the whole body is of great importance for early diagnosis and timely treatment. Recently, deep neural networks have been applied for the ULD task, and existing methods assume that all the training samples are well-annotated. However, the partial label problem is unavoidable when curating large-scale datasets, where only a part of instances are annotated in each image. To address this issue, we propose a novel segmentation-assisted model, where an additional semantic segmentation branch with superpixel-guided selective loss is introduced to assist the conventional detection branch. The segmentation branch and the detection branch help each other to find unlabeled lesions with a mutual-mining strategy, and then the mined suspicious lesions are ignored for fine-tuning to reduce their negative impacts. Evaluation experiments on the DeepLesion dataset demonstrate that our proposed method allows the baseline detector to boost its average precision by 13%, outperforming the previous state-of-the-art methods.