Multi-Scale Feature Pyramids for Weakly Supervised Thoracic Disease Localization

Yang Ma, Andy J. Ma, Youngsun Pan, Xi Chen

Abstract: Automatic localization of thoracic diseases has a wide range of applications which can assist radiologists for more efficient and better diagnosis. However, it is still a challenging task to locate the diseases accurately since strong location annotation may not be available and different thoracic diseases may vary in size greatly. In this paper, we propose a novel multi-scale feature pyramids model for weakly supervised disease localization on chest X-ray images. Our model leverages the multi-scale feature maps to learn location representation of lesions by fusing heatmaps generated from all these feature maps. Instead of linear combination of heatmaps, we reconFigure multi-scale feature maps with an Feature Pyramids Network (fpn) structure first. The FPN we conducted is a nonlinear combination of feature hierarchy, adding highly-nonlinear patterns in feature maps, enriching the representation space of heatmaps. Experiments on the ChestXray14 dataset show that the proposed method can significantly improve the 10-calization performance of small-sized diseases (nodules and masses) with competitive localization performance of large-sized diseases (cardiomegaly and pneumonia). Averagely, our method is superior to the state-of-the-art weakly supervised localization algorithms on the dataset, ChestXray14.