FundusAdapter: few-shot adaptation of fundus image foundation model for fundus image diagnosis

Yifan Chang, Zihang Jiang, Kun Zhang, S Kevin Zhou

Keywords: Few-shot adaption, Fundus image, Foundation model

Abstract: Fundus images exhibit significant source gaps, limiting the performance of foundation models across different scenarios. Due to the scarcity of labeled training data, few-shot adaptation is essential for effective diagnosis. However, existing few-shot adapters have primarily focused on global image features, which are insufficient for distinguishing fundus diseases that require detailed texture information. In this paper, we propose FundusAdapter, the first few-shot adaptation model of fundus image foundation model for fundus image diagnosis. By leveraging hierarchical feature extraction, FundusAdapter effectively integrates both global and local features, enhancing the detection of subtle lesions. The use of cross-attention and gate memory guidance improves the interaction between features, leading to more accurate adaptation. Our model achieves state-of-the-art performance on public fundus benchmarks. Code is available at https://github.com/Yifan-Chang/CrossFundus.

Submission Number: 18