Volume-based Performance not Guaranteed by Promising Patch-based Results in Medical Imaging

Abhishek Moturu, Sayali Joshi, Andrea S. Doria, Anna Goldenberg

Keywords: MRI, medical, imaging, tumour, segmentation, deep learning, augmentations, synthetic data, patches, pediatric, screening, cancer

TL;DR: strong patch-based performance does not guarantee strong volume-based performance in medical imaging

Abstract: Whole-body MRIs are commonly used to screen for early signs of cancer. In addition to the small size of tumours at onset, variations in individuals, tumour types, and MRI machines increase the difficulty of finding tumours in these scans. Using patches, rather than whole-body scans, to train a deep-learning-based segmentation model with a custom compound patch loss function, several augmentations, and additional synthetically generated training data to identify areas where there is a high probability of a tumour provided promising results at the patch-level. However, applying the patch-based model to the entire volume did not yield great results despite all of the state-of-the-art improvements, with over 50% of the tumour sections in the dataset being missed. Our work highlights the discrepancy between the commonly used patch-based analysis and the overall performance on the whole image and the importance of focusing on the metrics relevant to the ultimate user $-$ in our case, the clinician. Much work remains to be done to bring state-of-the-art segmentation to the clinical practice of cancer screening.