How Effective Can Dropout Be in Multiple Instance Learning ?

Wenhui Zhu, Peijie Qiu, Xiwen Chen, Zhangsihao Yang, Aristeidis Sotiras, Abolfazl Razi, Yalin Wang

TL;DR: The MIL methods need a specific Dropout.

Abstract: Multiple Instance Learning (MIL) is a popular weakly-supervised method for various applications, with a particular interest in histological whole slide image (WSI) classification. Due to the gigapixel resolution of WSI, applications of MIL in WSI typically necessitate a two-stage training scheme: first, extract features from the pre-trained backbone and then perform MIL aggregation. However, it is well-known that this suboptimal training scheme suffers from "noisy" feature embeddings from the backbone and inherent weak supervision, hindering MIL from learning rich and generalizable features. However, the most commonly used technique (i.e., dropout) for mitigating this issue has yet to be explored in MIL. In this paper, we empirically explore how effective the dropout can be in MIL. Interestingly, we observe that dropping the top-k most important instances within a bag leads to better performance and generalization even under noise attack. Based on this key observation, we propose a novel MIL-specific dropout method, termed MIL-Dropout, which systematically determines which instances to drop. Experiments on five MIL benchmark datasets and two WSI datasets demonstrate that MIL-Dropout boosts the performance of current MIL methods with a negligible computational cost. The code is available at \url{https://github.com/ChongQingNoSubway/MILDropout}.

Lay Summary:

Can dropout improve Multiple Instance Learning (MIL) for complex tasks like whole slide image (WSI) classification? We explored this by testing how different dropout strategies affect MIL performance.

MIL is often used in weakly-supervised settings, where labels are given for groups of data points (bags), not individual instances. In digital pathology, this means assigning a diagnosis to an entire WSI made up of many smaller image tiles. Current MIL models use a two-stage process: extract features with a pre-trained model, then aggregate them. But this setup often leads to noisy features and overfitting.

We found that dropping the top-k most important instances in each bag—not just dropping randomly—leads to better performance and generalization. Based on this, we propose MIL-Dropout, a simple yet effective method that learns which instances to drop. Our experiments on benchmark and medical datasets show that MIL-Dropout improves existing models with almost no extra cost.

Link To Code: https://github.com/ChongQingNoSubway/MILDropout

Primary Area: Deep Learning

Keywords: Multiple Instance Learning, Dropout, Optimization, Whole Slide Image, Computer Vision

Submission Number: 2291