iDPA: Instance Decoupled Prompt Attention for Incremental Medical Object Detection

Huahui Yi, Wei Xu, Ziyuan Qin, Xi Chen, Xiaohu Wu, Kang Li, Qicheng Lao

TL;DR: iDPA improves incremental medical object detection by introducing instance-level prompt generation and decoupled prompt attention, achieving superior performance on ODinM-13 across various data settings.

Abstract: Existing prompt-based approaches have demonstrated impressive performance in continual learning, leveraging pre-trained large-scale models for classification tasks; however, the tight coupling between foreground-background information and the coupled attention between prompts and image-text tokens present significant challenges in incremental medical object detection tasks, due to the conceptual gap between medical and natural domains. To overcome these challenges, we introduce the iDPA framework, which comprises two main components: 1) Instance-level Prompt Generation (IPG), which decouples fine-grained instance-level knowledge from images and generates prompts that focus on dense predictions, and 2) Decoupled Prompt Attention (DPA), which decouples the original prompt attention, enabling a more direct and efficient transfer of prompt information while reducing memory usage and mitigating catastrophic forgetting. We collect 13 clinical, cross-modal, multi-organ, and multi-category datasets, referred to as ODinM-13, and experiments demonstrate that iDPA outperforms existing SOTA methods, with FAP improvements of f 5.44%, 4.83%, 12.88%, and 4.59% in full data, 1-shot, 10-shot, and 50-shot settings, respectively.

Lay Summary:

Modern AI models can perform well on specific medical tasks, but struggle to continuously learn new ones without forgetting previous knowledge — especially in complex object detection tasks across different diseases and imaging types. While recent prompt-based methods help AI models remember better, they are mainly designed for natural images and simple classification tasks.

We introduce a new method called iDPA, which helps models incrementally learn medical tasks more effectively. It does this in two ways: first, it creates custom prompts based on specific objects in medical images (like tumors or lesions); second, it restructures how prompts interact with the model, making learning more efficient and reducing memory costs.

We tested our method on 13 diverse medical datasets, covering different organs, image types, and clinical scenarios. Our approach consistently outperformed existing methods, especially when very few training examples were available. This makes iDPA a promising tool for building adaptable, general-purpose medical AI systems.

Primary Area: General Machine Learning->Transfer, Multitask and Meta-learning

Keywords: Continual Learning.+Medical Object Detection

Submission Number: 313