Multi-Scale Task Multiple Instance Learning for the Classification of Digital Pathology Images with Global Annotations

Niccolò Marini, Sebastian Otálora, Francesco Ciompi, Gianmaria Silvello, Stefano Marchesin, Simona Vatrano, Gianziana Buttafuoco, Manfredo Atzori, Henning Müller

Keywords: Multi-Scale Multiple Instance Learning, Multiple Instance Learning, Multi-scale approach, Computational pathology

TL;DR: A multi-scale task multiple instance learning to classify WSIs

Abstract:

Whole slide images (WSIs) are high-resolution digitized images of tissue samples, stored including different magnification levels.
WSIs datasets often include only global annotations, available thanks to pathology reports.
Global annotations refer to global findings in the high-resolution image and do not include information about the location of the regions of interest or the magnification levels used to identify a finding.
This fact can limit the training of machine learning models, as WSIs are usually very large and each magnification level includes different information about the tissue.
This paper presents a Multi-Scale Task Multiple Instance Learning (MuSTMIL) method, allowing to better exploit data paired with global labels and to combine contextual and detailed information identified at several magnification levels.
The method is based on a multiple instance learning framework and on a multi-task network, that combines features from several magnification levels and produces multiple predictions (a global one and one for each magnification level involved).
MuSTMIL is evaluated on colon cancer images, on binary and multilabel classification.
MuSTMIL shows an improvement in performance in comparison to both single scale and another multi-scale multiple instance learning algorithm, demonstrating that MuSTMIL can help to better deal with global labels targeting full and multi-scale images.